BI EE Web Catalog security is an area that generally requires a lot of understanding especially while migrating Web Catalogs. There is not a lot available in the documentation that goes into detail on how .ATR files store security. Its a known fact that doing a file system copy of reports and dashboards from one web catalog to another can potentially corrupt the security. But unless we know how .ATR files store the security, it will be very difficult to know how corruption can happen just by doing a file system copy. This is what i covered as part of my Open World Presentation yesterday. You can download the full set of slides from here. I will quickly go through the important parts of the presentation here.

Binary Representation of Web Catalog Permissions:

In BI EE web catalog we can assign quite a lot of permissions(shown below) for each user/role. Its important to understand how the permissions are represented internally. BI EE follows a binary representation (similar to the unix folder permissions – 777,775 etc). BI EE uses a total of 16 bits to represent the Access Control List (basically the permissions) as shown below

So basically each permission setting like Full Control, Open, Modify etc will have a corresponding decimal representation which is given below.

Binary Representation of Accounts Structure:

While assigning users/roles to a catalog object, they are stored internally in a binary format. The screenshot below shows how they are actually stored.

As you see the above uses a total of 7 bits to represent accounts.

Decompiling .ATR files:

As i mentioned above, all the accounts, their corresponding permissions that are relevant to a catalog object are stored within the .ATR file. The above binary representation is also stored in the byte code of the .ATR file. But we need to know how to interpret the file and look at the contents without opening up catalog manager.

Understanding Application Role & Permissions Storage:

While assigning permissions to an application role on a catalog object, both the permission and the account get pushed inside the .ATR file. So to understand how they are stored internally lets open up the .ATR file of a sample folder called Permissions Test in a HEX editor.

As you see, the application roles are stored in ASCII within the .ATR file. But permissions are not stored in normal ASCII. Instead they are represented in HEX (basically the 2 bytes immediately after the Application Role names as shown below)

Understanding User & Web Catalog Groups Storage:

Unlike Application Roles which are stored in ASCII format, users and web catalog groups are stored in an encrypted format. So, it is not straightforward to understand how they are stored internally.

If you notice, the user analyst has a code of 5DCF9000EEDA1E1BFF0D99317CAD3C9. This is basically the code for the user. It is a reference to a file that exists under /system/security/accountids/xxx folder in the webcatalog. In addition to that there will also be a separate analyst user object under the /system/security/users/xxx folder.

If we now look at the HEX code of the encrypted file itself, you will notice that it has the analyst user reference as shown below.

The permissions are stored in the same format as the application roles (2 bytes after the encrypted code). Same will be the case for application roles as well.

In addition to the above, i had covered how user permissions get applied through application roles through various scenarios. I will not be covering that here. Instead you can download the slides directly from the link above.

My last session of this year’s Oracle Openworld 2011 in San Francisco was on systems management of OBIEE 11g. Compared to the 10g release of OBIEE, the 11g release is much more complex, is deployed in much more distributed-type environments, and has a bunch of new tools that you need to manage the system based around Enterprise Manager and WebLogic Server.

The slides for this session are available for download here : OBIEE 11g Systems Management Best Practices

The main idea for this presentation came from a blog post I wrote a few months ago, entitled “So How Does Enterprise Manager work, Within OBIEE 11g?”. In that blog post, I wrote about how Enterprise Manager, under the covers, uses the Java JMX MBeans to perform systems management tasks such as adding new system components to the cluster, uploading new repositories, or changing caching settings to enable, or disable it. Within the session, therefore, I talked about how Enterprise Manager is used primarily to manage the various components within an Oracle BI Domain, and how you could write scripts, like the one below, to manage the repository upload process and commit the changes to the domain.

If you’re interested in scripting like this, keep an eye out for the next edition of Oracle Magazine, which has an article by myself on this topic and details of the above script. For me now though, it’s off to the DW Global Leaders’ meeting, where I’m presenting a session with Stewart Bryson on Exadata and OBIEE, and what’s now possible with the new Exalytics BI Machine.

The third of my presentation at this year’s Oracle Openworld in San Francisco was in fact a collaboration with Andrejus Baranovskis, fellow ACE Director and specialist in Oracle Application Development Framework (ADF). The idea for this presentation came to us a while ago, when the 11g release of Oracle Business Intelligence provided a framework for embedding analyses, dashboards and other BI objects within ADF applications, and the ability to pass context (parameters) between these components.

The slides for this presentation are available for download here : OBIEE / ADF Integration using the Action Framework

This feature is best illustrated by the Oracle Fusion Applications, which bring together ADF, SOA and OBIEE to create composite, hybrid applications that have business intelligence weaved all through them.

So the thinking behind this session was to take the components and technologies that Oracle used for the Fusion applications, and use them to build something ourselves along these lines. If you’re an OBIEE customer, using ADF and JDeveloper to extend your BI solution has a number of advantages, including:

• Being able to extend your BI system to include transactional elements such as forms, data input and so on
• Ability to add collaboration, forums, communications, annotations to your system
• The ability to make use of ADF visualizations such as gantt charts, organizational charts and so on that haven’t yet made their way into OBIEE 11g proper

For ADF customers, adding BI into their project gives a number of advantages beyond the basic ADF DVT visualizations that you get with JDeveloper 11g:

• You get access to a proper metadata layer, and the ability to create calculations, hierarchies, and combine data across multiple data sources including files, applications, OLAP and relational
• You get a catalog to put the reports in, plus full security and permissioning to control what users can do with the reports
• You also get access to BI features such as the Action Framework, KPIs and Scorecard, BI Publisher, caching and so on

In the end we put a sample application together which showed off some of the main integration points, including passing parameters between the components, having OBIEE access JDeveloper-built web services, and accessing ADF data through the OBIEE metadata layer as a data source.

The main user of this feature up until now has been the Fusion Applications developers themselves, so some of the features are a bit “alpha” and the documentation is a bit sketchy at various points. You can see the slides for full details, but for example a key integration point is between BI analyses and ADF, where we want to pass context (referred to as a Qualified Data Reference, or QDR) between the components, but this is only possible through a new type of action that’s not actually documented, the ADF Contextual Event action.

Anyway, if you came to the session, thanks for that, and the slides are available for download as detailed above. Andrejus and I are also about to start documenting this process in full for a future article for the Oracle Technology Network, so keep an eye on the blog for a link to it when it gets published.

One of the key messages Oracle keep putting out about Exalytics, is that they had to provide a new user interface to make best use of the “speed of thought” analytics provided by the product. In particular, a few key new areas were highlighted that were made possible by the in-memory processing provided by Exalytics:

• Auto-suggest for prompts
• Auto-complete for prompts
• “Go-less” prompts
• High-density visualizations incl. microcharts
• Recommendation engine for visualizations

So what in practice does this mean? Oracle PR recently put a number of screenshots of the new user interface up on Flickr, so I’ll use these as examples.

This first screenshot is a good example of the “grid of charts” visualization that Exalytics provides, with each cell in the grid showing its own chart.

Now where I think Oracle are coming from here is the Edward Tufte-inspired sparkline idea, where a single visualization contains small, succinct micro-charts of information, to show stock movements over a time period, for example. I’m not sure what Exalytics is doing, in taking a regular chart and just miniaturising it, then placing it in a trellis of other charts, is quite what sparklines are all about, but it’s a good start and could be useful for certain types of analysis.

Note also the fact that the graph controls have now moved over to the left-hand side of the page, rather than under the chart as is currently the case for 11.1.1.5. I’ve never been a fan of having controls under the chart as you’re often short of screen space here, so putting them on the left by default is an improvement.

The second screenshot is I think better, and is much more “sparkline-like”.

I guess my only concern is whether this type of analysis would always be appropriate; Oracle are also making a big deal this week at Openworld about “big data”, and so you can see why analyses like these, displaying large sets of data using lots of small visualizations, would make sense. But my feeling is that the benefit of Exalytics will just be in making regular analyses run faster, and elminate any lag around drilling, analysis, calculations and so on.

For example, if you take this demo by Qlikview, surely the competitor that Oracle were most targeting with this release, their dashboards are far less cluttered and more like the type of dashboard we use now. My design approach with dashboards is to try and reduce complexity, by showing exceptions rather than the full data set, and providing data at a high-level and then the user drill-into more detail, so I’m not sure providing the whole data set, across the whole screen, and then letting the user pick out what’s important, is always the best approach.

But certainly, there are some types of analysis (typically, the “big data” analysis that Oracle are targeting it at) that would benefit from this, and I also guess that Exalytics, like the iPad, is probably something that makes more sense once you’ve had hands-on demo and tried it yourself. Here’s one more in a similar vein.

Not in the screenshots but demonstrated at Openworld were the auto-complete and auto-suggest features for prompts. The way this works is that prompts, that have a drop-down menu and free-form text box that let’s you search inline for matches, now auto-suggest matches as you type, a bit like the auto-suggest feature on Google. Similarly, when you create an analysis using the Exalytics release, you can ask Answers to suggest the most appropriate visualization based on some heuristics performed by the application, the data you’re reporting on and so on.

You can fine-tune this further by selecting a particular type of analysis you are looking to create (compare two numbers, for example), or just let Answers look at your data, the cardinality and so on, and suggest the best visualization for that. Sounds interested, but of course we’ll need to see how well it works in practice (i.e., is it Watson, or Clippy?)

It sounds like (though licensing and packaging weren’t discussed), that some or all of these visualizations will be Exalytics-only, partly because they will be so I/O intensive that you need an in-memory engine to provide the data, and partly I guess to drive sales of Exalytics. I think we’ll hear more about what’s in, and what’s out, of 11.1.1.6 depending on whether you use Exalytics or not, and I think this may also apply to the TimesTen and in-memory Essbase server that’s also been talked about for this release.

Something we’ve not got screenshots of but that also looked interesting, was the new UI for the Mobile product. The 11.1.1.5 release of OBIEE 11g introduced a new iPad client, which though good was a bit limiting and also suffered from a few usability issues where it hadn’t properly adopted the native UI elements for IOS. This new release takes a fresh look at this, includes a coverflow-style dashboard picker, and implements prompts properly by using native IOS controls. I didn’t get any hands-on time myself, but this looks like a good upgrade on the initial, 11.1.1.5 release.