Category Archives: Rittman Mead
An Alternative Front-End for OBIEE using Web Services and d3
Working with OBIEE, it is common to get reequests to add visualisations or otherwise improve the UI. Typically this has been done by using narrative views to embed HTML and Javascript to provide data driven, custom visualisations. To cope with the former, Tom Underhill developed the Rittman Mead Visual Plugin Pack (RMVPP), a web based plugin to allow configurable visualisations to work natively in OBIEE through the use of narratives. This technology is uses configurable Javascript templates for each plugin. Each plugin has a render function which will receive the dataset from OBIEE as well as any associated information. The features of the plugin are then only limited by the ability of the JS developer and current web standards. Plugin developers could be further aided by sharing common functions for display and data manipulation, and external libraries can be included simply.
Above shows the process of creating a Sankey diagram using the RMVPP framework. The code then produces the corresponding narrative required to generate this visualisation automatically.
When RMVPP was being developed by Tom, I was investigating the use of web services in OBIEE. Several of us at Rittman Mead had used the exposed web services to interface with OBIEE to automate various tasks programmatically, such as regression testing and web catalogue auditing. A client had a requirement to execute an OBIEE query on a webpage outside of the OBIEE front end in order to power a fairly sophisticated 3JS application that was not rendering within OBIEE (via a narrative). It appeared that the shortcomings were due to attempting to render the WebGL within the complex div and table structure of an OBIEE page. Soon, I was able to use the web services client side, in Javascript (so long as the page was hosted on the same server as OBIEE) as opposed to server side in Python or similar. Combining this breakthrough with Tom’s RMVPP framework seemed like the next logical step – as it would allow us to totally modify the OBIEE front end whilst retaining the benefits of using the BI Server like security and physical query generation.
An Alternative Front-end for OBIEE
The following section will describe and show some of the features I have developed in my alternative to the OBIEE front end. It uses the plugin framework from RMVPP but with a custom interface that works differently to OBIEE.
Creating a Visualisation
Below is a screencapture showing creation of a basic bar chart.
The column mapping tab is configurable by the plugin developer and specifies the parameters that each visualisation will use. As a user, OBIEE presentation columns (displayed on the left) are mapped into these parameters. A flag can be set on a parameter to allow multiple columns to be chosen for that parameter. The plugin developer can then use the resulting data set accordingly. The columns selected here are effectively the Criteria from vanilla OBIEE as they determine which columns are used in the logical SQL. The parameter mapping allow the developer to use the correct columns from the data set in the render function.
The configuration screen shown there is customisable by the plugin designer, allowing developers to specify a list of options and UI elements that get fed back into the render function for the visualisation. UI options include:
- Textboxes
- Dropdowns
- Radio Buttons
- Colour Pickers
Filtering
The next screen capture shows a Sankey chart and applies a filter to the query. The filter mechanism also has an sub search facility similar to vanilla OBIEE.
Conditional Formatting
The next example shows a pivot table being created and the conditional formatting functionality available.
The conditional format tab allows you to choose the columns to apply formatting to as well as the formatting rule that should be applied. Additionally, a plugin developer can specify custom conditional formats to be available for specific plugins. The heatmap formatting style is one such example for the pivot table.
Adding and Editing Columns
Similar to vanilla OBIEE, columns formats and formulae can be edited and added for visualisations.
Saving/Loading
Visualisations (and dashboard pages) can be saved and loaded to and from the web catalogue. This means that security can be applied to them in the normal way.
They are saved to OBIEE as analyses with Static Text views in them containing the necessary information to build the visualisation with my API. OBIEE however, will not be able to render these properly and I’ve stopped developing that functionality for a number of reasons I won’t go into in this post. The screencapture shows a brief glimpse of the webcat explorer in my app, which is basic, but filters out any vanilla OBIEE objects, only allowing you to view and open custom objects that were created by the app.
Dashboards
The interface allows the user to create dashboards from the same app, without having to save individual visualisations first.
After creating and configuring a visualisation, they can be temporarily stored. Once stored, the user can switch to dashboard mode, which will have a blank canvas, allowing the user to add any stored visaulisations. The edit button in dashboard mode will allow visualisations to be freely placed on the canvas.
Interactions
The interactivity framework is probably the feature with the biggest benefit over the vanilla system that I’ve added.
Plugin developers are able to create triggers as part of their visualisations. These triggers can be fired in most situations and tied to the data so that dynamic interactions between visualisations are possible. Users can specify what data should be passed to the target reaction. By default, every plugin can be hard filtered by an interaction, which modifies the query and goes back to the BI server. Plugin developers can also write their own reactions, allowing for interactivity only limited by the plugin designer’s web development capability.
Drilldowns
Drilldowns between custom dashboard pages can be defined using the same framework as the other interactions.
Users need only enter the webcat path for the target page in order for the drilldown to function. Each visualisation of the target dashboard will be automatically filtered (if the subject area matches) on the criteria specified by the interaction.
Dashboard Prompts
Some of the vanilla features in OBIEE are very useful, so I replicated the functionality in my app.
This shows the implementation of prompts to a dashboard page. Unlike OBIEE, it is not required to set an ‘is prompted’ filter on each target. Rather, the prompt will automatically filter any visualisations on the page that are from the same subject area.
Column Selectors
Column selectors can be added as either dropdowns or radio buttons.
The user chooses an array of columns to be in the column selector. If any visualisations in the dashboard contain a column in this array, it will be swapped when the column selector is updated.
Map Visualisations
Two of the visualisation plugins I have made are map based, for geographical data.
Both use LeafletJS, an open source Javascript mapping library. The first, is a bubble chart which relies on longitude and latitude being stored in the database and exposed via OBIEE. The second is a choropleth which requires a TopoJSON file of the regions in order to function. Additionally, the json file needs to have an attribute attached to each region which will act as a key to join it onto OBIEE data. So that region attribute needs to have a matching key exposed via OBIEE.
Summary
So that wraps up the major features of the app. I don’t know if it’s going to be useful, if there’s any mileage in further development or any of that, but was an interesting PoC while it lasted. If anyone has any comments, ideas, features or just want to know more about the app and how it works, feel free to comment.
The post An Alternative Front-End for OBIEE using Web Services and d3 appeared first on Rittman Mead Consulting.
Driving OBIEE User Engagement with Enhanced Usage Tracking for OBIEE
Measuring and monitoring user interactions and behaviour with OBIEE is a key part of Rittman Mead’s User Engagement Service. By understanding and proving how users are engaging the system we can improve the experience for the user, driving up usage and ensuring maximum value for your OBIEE investment. To date, we have had the excellent option of Usage Tracking for finding out about system usage, but this only captures actual dashboard and analysis executions. What I am going to discuss in this article is taking Usage Tracking a step further, and capturing and analysing every click that that the user makes. Every login, every search, every report build action. This can be logged to a database such as Oracle, and gives us Enhanced Usage Tracking!
Why?
Because the more we understand about our user base, the more we can do for them in terms of improved content and accessibility, and the more we can do for us, the OBIEE dev/sysadmin, in terms of easier maintenance and better knowledge of the platform for which we are developing.
Here is a handful of questions that this data can answer – I’m sure once you see the potential of the data you will be able to think of plenty more…
How many users are accessing OBIEE through a mobile device?
Maybe you’re about to implement a mobile strategy, perhaps deploying MAD or rolling out BI Mobile HD. Wouldn’t it be great if you could quantify its uptake, and not only that but the impact that the provision of mobile makes on the general user engagement levels of your OBIEE user base?
Perhaps you think your users might benefit from a dedicated Mobile OBIEE strategy, but to back up your business case for the investment in mobile licences or time to optimise content for mobile consumption you want to show how many users are currently accessing full OBIEE through a web browser on their mobile device. And not only ‘a mobile device’, but which one, which browser, and which OS. Enhanced Usage Tracking data can provide all this, and more.
Which dashboards get exported to Excel the most frequently?
The risks that Excel-marts present are commonly discussed, and broader solutions such as data-mashup capabilities within OBIEE itself exist – but how do you identify which dashboards are frequently exported from OBIEE to Excel, and by whom? We’ve all probably got a gut-instinct, or indirect evidence, of when this happens – but now we can know for sure. Whilst Usage Tracking alone will tell us when a dashboard is run, only Enhanced Usage Tracking can show what the user then did with the results:
What do we do with this information? It Depends, of course. In some cases exporting data to Excel is a – potentially undesirable but pragmatic – way of getting certain analysis done, and to try to prevent it unnecessarily petulant and counterproductive. In many other cases though, people use it simply as a way of doing something that could be done in OBIEE but they lack the awareness or training in order to do it. The point is that by quantifying and identifying when it occurs you can start an informed discussion with your user base, from which both sides of the discussion benefit.
Precise Tracking of Dashboard Usage
Usage Tracking is great, but it has limitations. One example of this is where a user visits a dashboard page more than once in the same session, meaning that it may be served from the Presentation Services cache, and if that happens, the additional visit won’t be recorded in Usage Tracking. By using click data we can actually track every single visit to a dashboard.
In this example here we can see a user visiting two dashboard pages, and then going back to the first one – which is captured by the Enhanced Usage Tracking, but not the standard one, which only captures the first two dashboard visits:
This kind of thing can matter, both from an audit point of view, but also a more advanced use, where we can examine user behaviour in repeated visits to a dashboard. For example, does it highlight that a dashboard design is not optimal and the user is having to switch between multiple tabs to build up a complete picture of the data that they are analysing?
Predictive Modelling to Identify Users at Risk of ‘Churn’
Churn is when users disengage from a system, when they stop coming back. Being able to identify those at risk of doing this before they do it can be hugely valuable, because it gives you opportunity to prevent it. By analysing the patterns of system usage in OBIEE and looking at users who have stopped using OBIEE (i.e. churned) we can then build a predictive model to identify those with similar patterns of usage but are still active.
Measures such as the length of time it takes to run the first dashboard after login, or how many dashboards are run, or how long it takes to find data when building an analysis, can all be useful factors to include in the model.
Are any of my users still accessing OBIEE through IE6?
A trend that I’ve seen in the years working with OBIEE is that organisations are [finally] moving to a more tolerant view on web browsers other than IE. I suppose this is as the web application world evolves and IE becomes more standards compliant and/or web application functionality forces organisations to adopt browsers that provide the latest capabilities. OBIEE too, is a lot better nowadays at not throwing its toys out of the pram when run on a browser that happens to have been written within the past decade.
What’s my little tirade got to do with enhanced usage tracking? Because as those responsible for the development and support of OBIEE in an organisation we need to have a clear picture of the user base that we’re supporting. Sure, corporate ‘standard’ is IE9, but we all know that Jim in design runs one of those funny Mac things with Safari, Fred in accounts insists on Firefox, Bob in IT prides himself on running Konquerer, and it would be a cold day in hell before you prise the MD’s copy of IE5 off his machine. Whether these browsers are “supported” or not is only really a secondary point to whether they’re being used. A lot of the time organisations will take the risk on running unsupported configurations, consciously or in blissful ignorance, and being ‘right’ won’t cut it if your OBIEE patch suddenly breaks everything for them.
Enhanced Usage Tracking gives us the ability to analyse browser usage over time:
as well as the Enhanced Usage Tracking data rendered through OBIEE itself, showing browser usage in total (nb the Log scale):
It’s also easy to report on the Operating System that users have:
Where are my users connecting to OBIEE from?
Whilst a lot of OBIEE deployments are run within the confines of a corporate network, there are those that are public-facing, and for these ones it could be interesting to include location as another dimension by which we analyse the user base and their patterns of usage. Enhanced Usage Tracking includes the capture of a user’s IP, which for public networks we can easily lookup and use the resulting data in our analysis.
Even on a corporate network the user’s IP can be useful, because the corporate network will be divided into subnets and IP ranges, which will usually have geographical association to them – you just might need to code your own lookup in order to translate 192.168.11.5 to “Bob’s dining room”.
Who deleted this report? Who logged in? Who clicked the Do Not Click Button?
The uses for Enhanced Usage Tracking are almost endless. Any user interaction with OBIEE can now be measured and monitored.
A frequent question that I see on the OTN forums is along the lines of “for audit purposes, we need to know who logged in”. Since Usage Tracking alone won’t capture this directly (although the new init block logging in > 11.1.1.9 probably helps indirectly with this) this information usually isn’t available….until now! In this table we see the user, their session ID, and the time at which they logged in:
What about who updated a report last, or deleted it? We can find that out too! This simple example shows some of the operations in the Presentation Catalog recorded as clear as day in Enhanced Usage Tracking:
Want to know more? We’d love to tell you more!
Measuring and monitoring user interactions and behaviour with OBIEE is a key part of Rittman Mead’s User Engagement Service. By understanding and proving how users are engaging the system we can improve the experience for the user, driving up usage and ensuring maximum value for your OBIEE investment.
If you’d like to find out more, including about Enhanced Usage Tracking and getting a free User Engagement Report for your OBIEE system, get in touch now!
The post Driving OBIEE User Engagement with Enhanced Usage Tracking for OBIEE appeared first on Rittman Mead Consulting.
Rittman Mead at UKOUG Tech’15 Conference, Birmingham
This week Rittman Mead are very pleased to be presenting at the UK Oracle User Group’s Tech’15 Conference in Birmingham, delivering a number of sessions around OBIEE, Data Integration, Cloud and Big Data.
If you’re at the event and you see any of us in sessions, around the conference or during our talks, we’d be pleased to speak with you about your projects and answer any questions you might have. Here’s the list our speaking slots over the four days of the event, and I’ll update the list with links to presentation downloads as they become available over the event.
- “Adding a Data Reservoir to Your Oracle Data Warehouse for Customer 360-Degree Analysis” – Mark Rittman, Sunday 6th December 2015
- “Smarter Regression Testing for OBIEE” – Robin Moffatt, Tuesday 8th December 2015
- “Deploying Full OBIEE Systems into Oracle Cloud” – Mark Rittman, Tuesday 8th December 2015
- “User Adoption and Retention in OBIEE for the IT Department” – Robin Moffatt, Wednesday 9th December 2015
- “Putting Oracle Big Data to Work – Three Real-World Case Studies From Rittman Mead’s Big Data Delivery Team” – Mark Rittman, Wednesday 9th December 2015
- “ODI Lifecycle and Data Governance” – Jerome Francoisse, Tuesday 8th December
- “Migration From Oracle Warehouse Builder to Oracle Data Integrator – Eurocontrol Case Study” – – Jerome Francoisse, Wednesday 9th December
In addition, if you’re interested in the OBIEE user adoption and retention area that Robin talks about in his Wednesday session, Rittman Mead have a User Engagement service using some of the tools and techniques that Robin talked about (datasheet here) and we’d be pleased to talk to you about how you can increase user engagement, adoption and retention for your OBIEE system. Other than that, come and speak to us if you see us at the Birmingham ICC, and look forward to more content on these areas in the New Year!
The post Rittman Mead at UKOUG Tech’15 Conference, Birmingham appeared first on Rittman Mead Consulting.
Oracle OpenWorld 2015 Roundup Part 3 : Oracle 12cR2 Database Sharding, Analytic Views and Essbase 12c
With the UKOUG conference starting tomorrow I thought it about time I finished off my three-part post-OOW 2015 blog series, with a final post on some interesting announcements around Oracle Database and Essbase. As a reminder the other two posts were on OBIEE12c and the new Data Visualisation Cloud Service, and Data Integration and Big Data. For now though let’s look first at two very significant announcements about future 12cR2 functionality – database sharding and Analytic Views.
Anyone who’s been involved in Oracle Data Warehousing over the years will probably be aware of the shared-everything vs. shared-nothing architecture debate. Databases like Oracle Database were originally designed for OLTP workloads with the optimal way to increase capacity being to buy a bigger server. When RAC (Real Application Clusters) came along the big selling point was a single shared database instance spread over multiple nodes, making application development easy (no real changes) but with practical limits as to how big that cluster can get – due to the need to synchronise shared memory across all nodes in the cluster, and network bottleneck caused by compute and storage being spread across the whole cluster, not co-located as we get with Hadoop and HDFS, for example.
Shared-nothing databases such as Netezza, for example, take a different approach and “shard” the database instance over multiple nodes in the cluster so that processing and storage are co-located on the same node for particular ranges of data. This gives the advantage of much greater scalability that a shared-nothing approach (again, this is why Hadoop uses a similar approach for its massively-clustered distributed compute approach) but with the drawback of having to consider data locality when writing ETL and other code; at worst it means data loading and processing needs to be rewritten when you add more nodes and re-shard the database, and it also generally precludes OLTP work and consequently mixed-workloads on the same platform.
But if it’s just data warehousing you want to do, you don’t really care about mixed workloads and its generally considered that shared-nothing and sharding is what you need if you want to get to very-large scale data warehousing, such that Oracle went partly down the shared-nothing route with Exadata and push-down of filtering, projection and other operations to storage nodes thereby adding an element of data locality and reducing the network throughput between storage and compute.
But both types of database are loosing out to Hadoop for very, very large datasets with Hadoop distributed compute approach designed right from the start for large distributed workloads at the expense of not supporting OLTP at all and, at least initially, all intermediate resultsets being written to disk. For those types of workloads and database size Oracle just wasn’t an option, but a certain top their of Oracle’s data warehousing customers wanted to be able to scale to hundreds or thousands of nodes and most of them have ULAs, so cost isn’t really a limiting factor; for those customers, Oracle announced that the 12c Release 2 version of Oracle Database would support sharding … but with warnings it’s for sophisticated and experienced customers only.
Oracle are positioning what they’re referring to as “Oracle Elastic Sharding” as for both scaling and fault-tolerance, with up to 1,000 nodes supported and with data routed to particular shards through use of a “sharding key” passed the connection pool.
Sharding in 12c Release 2 was described to me as a featured aimed to the “top 5%” of Oracle customers where price isn’t the issue but they want Oracle to scale to the size of cluster supported by Hadoop and NoSQL. Time will tell how well it’ll work and what it’ll cost, but it certainly completes Oracle’s journey from strict shared-everything for data warehousing to more-or-less shared nothing, if you want to go down that extreme-scalability route.
The other announcement from the Oracle Database side was the even-more-unexpected “Analytic Views”. A clue came from who was running the session – Bud Endress, of Oracle Express / Oracle OLAP fame and more recently, the Vector Group By feature in the In-Memory Option – but what we got was a lot more than Oracle OLAP re-imagined for in-memory; instead what Oracle are looking to do is bring the business metadata and calculation layers that BI tools use right into the database, provide an MDX query interface over it, simplify SQL so that you just select measures, attributes and hierarchies – and then optimise the whole thing so it runs in-memory if you have that option licensed.
Its certainly an “interesting” goal with considerable overlap with OBIEE’s BI Server and Essbase Server, but the goal of bringing all this functionality closer to the data and available to all tools is certainly ambitious, if it gets traction it should bring business metadata layers and simpler queries to a wider audience – but the fact that it seems to be being developed separately to Oracle’s BI and Essbase teams means it probably won’t be subsuming Essbase or the BI Server’s functionaliy.
The last area I wanted to look at was Essbase. Essbase Cloud Service was launched at this event with it positioned as a return to Essbase’s roots as a tool you could use in the finance department without requiring IT’s help, except this time it’s because Essbase is running as a service in the cloud rather than on an old PC under your desk. What was particularly interesting though is that the version of Essbase being used in the cloud is the new 12c version, that replaces some of the server components (the Essbase Agent, but not the core Essbase Server part) with new Java components that presumably fit better with Oracle’s cloud infrastructure and also support greater levels of concurrency
Apart from the announcement of a future ability to link to R libraries, the other really interesting part of Essbase 12c is that for now the only on-premise version of it is as part of OBIEE12c, and it’ll have a very fixed role there as a pure query accelerator for OBIEE’s BI Server – perhaps the answer to Qlikview and Tableau’s in-memory column-store caches. Essbase as part of an OBIEE12c store doesn’t work with Essbase Studio or any of the other standard Essbase tools, but instead has a new Essbase Business Intelligence Acceleration Wizard that deploys Hybrid ASO/BSO Essbase cubes directly from the OBIEE BI Server and RPD.
Coupled with the changes to Essbase announced a couple of years ago at Openworld 2013 designed to improve compatibility with OBIEE, this co-located version of Essbase seems to have completed it’s transformation into the BI Server mid-tier aggregate cache layer of choice that started back with the 11.1.1.6.2 BP1 version of OBIEE – but it does mean this version can’t be used for anything else, even custom Essbase cubes you load and design yourself. Interesting developments across both database server products though, and that wraps up my overview of OOW2015 announcements. Next stop – UKOUG Tech’15 in Birmingham, where I’ve just arrived ready for my masterclass session in tomorrow’s Super Sunday event – on data reservoirs and Customer 360 on Oracle Big Data Appliance.
The post Oracle OpenWorld 2015 Roundup Part 3 : Oracle 12cR2 Database Sharding, Analytic Views and Essbase 12c appeared first on Rittman Mead Consulting.
Action Links in OBIEE 12c – Part 1
Introduction
With the release of OBIEE 12c, let’s take a look at Action Links and how things may be different compared to the previous release, 11g. Over this three part blog series, we’re going to cover the more popular link types, which are navigating to BI content, and navigating to a web page. However, to sweeten the deal, I’ll also include some tricks for your tool belt which well enable you to do the following:
- Navigate to a target report, while filtering it on parameters chosen in the source
- Pass filter parameters via the GoURL syntax from a source report to another, target report
- Become familiar with the GoURL structure and how to apply it to your business case
In the first installment of this three part series, we’re going look at how to navigate to other reports and dashboards in your catalog through the ‘Navigate to BI Content’ action. This will set you up for parts 2 and 3, wherein we show you some tricks using Action Links.
1. The Action Link UI
By now, there are likely lots of blogs talking about the new look and features of OBIEE 12c, so we can keep this bit short. Suffice to say it got a much needed face lift, with both changes in overall skinning and in its portfolio of icons. While this change in graphics may induce a bit of frustration on part of the developer, I believe this approach to design will end up being a good long term strategy to handle later releases of the product as trends in UX seem to have their feet firmly planted in the stripped down, the clean, and the subdued. Even with this shift, however, the basic processes and series of steps to implement most any of the features in Answers remains the same, Action Links being no different. Just follow these simple steps below to set up your Action Link! After you’ve got a hold of the basics, look to future posts in this series for some tips and tricks using Action Links.
In chosen column, go to the Column Properties menu:
Next, click on the Interaction tab:
Select ‘Action Links’ as the Primary Interaction value and click on the ‘+’ icon. This will display another dialogue box where we will set up the actual properties of the Action Link. Click on the running man icon (this little guy seems to be more intuitive than the green gear):
2. Navigate to BI Content
For the first example, we’re going to select the ‘Navigate to BI Content’ option. This simply allows us to go to another report or dashboard, as though you were clicking on a link in a web page. To implement this on your report, simply follow the steps above and then refer to the steps below.
After clicking on the running man icon, select the ‘Navigate to BI Content’ option. This will be followed by a dialogue box allowing you to select the object to which you want to navigate.
Confirm your selection and then click ‘OK’, not once, not twice, but thrice, at which point you’re taken back to the Criteria tab. From now on, this column will take you to the selected report.
And that’s it! Take a look back here for part 2 on Action Links in OBIEE 12c, which will outline a neat technique on how to implement what’s called a ‘driving document’ to filter values between disparate reports using the navigate action.
The post Action Links in OBIEE 12c – Part 1 appeared first on Rittman Mead Consulting.
