Tag Archives: Oracle BI Suite EE
Introduction to the BI Apps 11.1.1.7.1 – Product Architecture & New Configuration Tools
In my previous posting in this series, I looked at the new 11.1.1..7.1 release of the Oracle BI Applications at a high-level, and talked about how this new release uses ODI as the embedded ETL tool instead of Informatica PowerCenter. Support for Informatica will come with patch set 2 (PS2) of BI Apps 11.1.1.7.x giving customers the choice of which ETL to use (with the caveat that customers upgrading from 7.9.x will typically have to stick with Informatica unless they want to completely re-implement using ODI), but for this initial release at least, ODI and some new Fusion Middleware tools take over from Informatica and the DAC, giving us what could well be a much simpler architecture for supplying the underlying data for the BI Apps dashboards.
In this posting then, I’m going to take a closer look at this new product architecture, and I’ll follow it with a more detailed look at how the various bits of ODI functionality replace the workflows, mappings, transformation operators and execution plans provided in earlier releases by Informatica and the DAC. For anyone familiar with the previous, 7.9.x versions of the BI Applications, the architecture diagram below shows the five tiers that this product typically implemented; tiers for the source data and data warehouse/repository databases, an ETL tier for Informatica and the DAC server, then two more tiers for the OBIEE application server and the client web browser.
Communication between the tiers was – to put it politely – “loosely coupled”, with DAC task names corresponding with Informatica workflow names, each workflow containing a single mapping, and all of the connections and sources having to be named “just so”, so that every part of the stack could communicate with all the others. It worked, but it was a lot of work to implement and configure, and once it was up and running in most cases customers were scared to then change it, in case a name or a connection got out of sync and everything then stopped working. Plus – Informatica skills are scarce in the Oracle world, and the DAC is an extra piece of technology that few DBAs really understood properly.
The 11.1.1.7.1 release of the BI Apps simplifies this architecture by removing the separate ETL tier, and instead using Oracle Data Integrator as the embedded ETL tool, with its server functions running as JEE applications within the same WebLogic domain as OBIEE 11g, giving us the overall architecture in the diagram below.
Now anyone who read my series of posts back in 2009 on the 7.9.5.2 release of the BI Apps, which also used ODI as the embedded ETL tool, will know that whilst ODI 10g could do the job of loading data into the BI Apps data warehouse, it lacked the load orchestration capabilities of Informatica and the DAC and wasn’t really set up to dynamically generate what have become, in ODI 11g, load plans. BI Apps 7.9.5.2 turned-out to be a one-off release and in the intervening years Oracle have added the aforementioned load plans along with other functionality aimed at better supporting the BI Apps, along with two new JEE applications that run in WebLogic to replace the old DAC. These new applications, along with the ODI JEE agent, ODI Console and the ODI SDK, are shown in the more detailed BI Applications 11.1.1.7.1 logical architecture diagram shown below.
Oracle BI Applications 11.1.1.7.1 has two main product tiers to it, made up of the following components:
- The Middleware (BI and ETL) tier; a WebLogic domain and associated system components, comprising BI components delivered as part of OBIEE 11.1.1.7 (including Essbase and related applications) as one managed server, and another managed server containing ODI Java components, including three new BI Apps-related ones; Configuration Manager, Functional Setup Manager, and ODI Load Plan Generator
- The Database (DW and Repositories) tier; for the time-being, Oracle only, and comprising a data warehouse schema (staging + performance layer), and a repository database containing the OBIEE repository schemas plus new ones to hold the ODI repository and other ETL/configuration metadata used for configuring your system.
Essbase at this stage is installed, but not used for the main BI applications, and all of it uses Fusion Middleware security (application roles and policies) along with the WebLogic Embedded LDAP server to hold users and groups. A special version of RCU is used to set up the new BI Apps-related schemas, and import data into them using Oracle database export files, so that the ODI repository, metadata tables and so forth are all populated prior to the first load taking place. Enterprise Manager Fusion Middleware Control is still used to manage and monitor the overall platform, but there’s now an entry for ODI along with Essbase, the latter of course being delivered as part of the 11.1.1.7 OBIEE platform release.
In the next posting in the series we’ll take a closer look at how ODI uses its JEE agent and mappings imported into its repository to load the BI Apps data warehouse, but what about the two new web-based configuration tools, Oracle BI Applications Configuration Manager (BIACM) and Oracle BI Applications Functional Setup Manager (FSM) – what do they do?
After you install OBIEE 11.1.1.7 and then the BI Applications 11.1.1.7.1, the BI Apps installer extends the BI domain to include FSM, BIACM and the ODI Load Plan Generator, along with some other supporting applications and libraries required for the full product. Load Plan Generator works behind the scenes to build new load plans in a similar way to the Execution Plan “Build” feature in the DAC, and the two web-based tools perform the following functions:
- Oracle BI Applications Configuration Manager performs system-wide setup tasks such as defining sources, selecting BI Apps modules and performing other, “one-only” tasks similar to the Setup feature in the DAC Console.
- Oracle BI Applications Functional Setup Manager is then used to list out, and track progress against, the various tasks required to configure the BI Applications modules, or “Offerings”, that you selected in the Configuration Manager
Most importantly though, these tools connect directly through to the DAC repository, so data sources you set up here will get pushed down to ODI as data servers in the ODI master repository; load plans you set up to, as in the screenshot below, load configuration tables, are ODI load plans and you can track their progress either from within ODI, or from within these applications themselves.
I haven’t had a chance to properly “diff” the RPD used in BI Apps 11.1.1.7.1 with the previous 7.9.x ones, or do a similar exercise for the underlying database data model, but on first glance the new RPD is at least recognisable, albeit with new sources and subject areas for the Fusion Apps, Oracle Transactional BI (OTBI), Real-Time Decisions and the like. The web catalog also looks familiar, but also has new content around the new applications along with additional content for the existing ones.
So, we’re at the point now where can start to think about loading data into the BI Apps data warehouse, and in tomorrows post we’ll take a look at what’s involved in a BI Apps 11.1.1.7.1 ETL load, and also look into how GoldenGate can now be used to extract and stage data prior to loading via ODI. Back tomorrow…
Introduction to the BI Apps 11.1.1.7.1 – Release Overview
The long-awaited and anticipated 11.1.1.7.1 (PS1) release of the Oracle BI Applications became available early last week, with the software and documentation available for download on OTN. Over the next few blog posts, I’ll be taking an in-depth look at this new release, starting today with an overview of what’s new and any limitations in his initial version, and then over the next few posts taking a look at the product architecture, how it uses Oracle Data Integrator instead of Informatica to do the data loads, and what new content the 11g dashboards contain. For a bit of background into this release you’re best off taking a look at a series of posts I put together towards the end of last year on the BI Apps product roadmap, and I’ll recap on those posts a bit in this one as I go through at a high level what’s in this release.
Although the focus in BI Apps 11.1.1.7.1 is on ODI as the ETL tool, this new release actually delivers a whole new product architecture along with new dashboards, new content, and a new security framework. In addition, there’s now an option to use Oracle GoldenGate to create a new layer in the BI Apps data warehouse data architecture that replicates source data into the warehouse environment, giving you the ability to run the more large-scale ETL processes when you like, rather than when there’s an ETL window for the source systems.
Let’s start off though with a summary of what’s new from a functional perspective, and also what limitations there are for this first release in terms of sources, scope and so forth. BI Apps 11.1.1.7.1 delivers the following set of new features and capabilities:
- Oracle Data Integrator as the embedded ETL tool, along with a whole new FMW11g-centric architecture and set of utilities
- Two new analytic applications - Student Information Analytics, and Indirect Spend Planning
- New content for existing analytic applications including Financial Analytics, HR, Projects, CRM and Procurement & Spend
- Dashboards that are now written for OBIEE 11g rather than 10g, including 11.1.1.7 visualisations such as performance tiles
Now although, in general terms, BI Apps 11.1.1.7.1 covers all (or most…) of the existing analytic application modules along with all of the 7.9.x-era sources (EBS, PeopleSoft, JDE and Siebel), there are some important restrictions that you’ll need to be aware of when making any plans to use this new release, starting with upgrade paths (or lack of them):
- There’s no automatic upgrade path from BI Apps 7.9.x, and no automated migration routine to take you from Informatica to ODI; if you want BI Apps 11.1.1.7.1 now, you’ll have to reimplement rather than upgrade, or you can wait for BI Apps 11.1 PS2 which will support upgrades from earlier releases, but (important to note) keeps you on Informatica – any move from Informatica to ODI will need to be done yourself, as a re-implementation
- Only Oracle database sources and targets are supported in this initial release, in practice not a real issue for new implementations, but worth bearing in mind if you planned to use Teradata, for example, as your target data warehouse platform
- Oracle Fusion Applications aren’t supported as a source either, yet, so anyone using this will need to stay on BI Apps 11.1.1.6.x until an upgrade version becomes available
- A few edge-case analytic applications and sources aren’t supported in this release yet – Enterprise Asset Management, for example, is not yet supported for any source, whereas some other applications only support more recent PeopleSoft versions and not JDE, for example. As always, get the most up-to-date supported sources and applications list from Oracle before making any major investment in an implementation or upgrade project.
From a technical perspective though the major difference in this release, compared to the 7.9.6.x versions that preceded it, is the use of Oracle Data Integrator 11g as the embedded ETL tool rather than Informatica. To be clear, Informatica will still be supported as an ETL option for the BI Apps going well into the foreseeable future, but Informatica users will need to wait for the PS2 release due in the next twelve months or so before they can upgrade to the new 11g platform.
In addition and perhaps more importantly, it’s not envisaged that Informatica customer will move over to ODI unless they use the upgrade as an opportunity to re-implement their system now on ODI, moving across customisations themselves and essentially starting with a clean sheet of paper (which may not be a bad thing, if you’re thinking of tidying up your system following years of upgrades, customisations and so forth). What this does mean though is no DAC, no Informatica server and client tools, a new (and hopefully simpler) way of setting up and configuring your system, and in-theory a more closely-integrated set of tools all based around the modern, standards-based Fusion Middleware 11g architecture.
In this new world of ODI and the BI Apps, ODI load plans replace Informatica Workflows, whilst ODI packages and interfaces equate to Informatica mappings and maplets. The DAC is no more and is replaced by metadata within the ODI repository and other supporting schemas, with setup and configuration of the warehouse and ETL processes now carried out by two web-based tools, BI Applications Configuration Manager and Functional Setup Manager. The closer integration between these tool along with a chance for Oracle to re-think the BI Apps setup process should lead to easier configuration and customisations, but if you’re an Informatica developer it’s a whole new world, and the 11g platform makes a lot more use of Fusion Middleware platform functionality particularly around security and user provisioning.
So – all very exciting but quite daunting in terms of what needs to be learnt, and new processes that need to be thought through and put together before you can start making use of the new 11.1.1.7.1 feature set. We’ll start tomorrow then by taking a closer look at the BI Apps 11.1.1.7.1 technical architecture including the new configuration tools, and where ODI sits in the new product architecture, based on our first impressions of the product.
Agenda and Details for the Brighton RM BI Forum 2013
It’s Tuesday morning, and later today delegates will be starting to arrive in Brighton for the 5th Annual Rittman Mead BI Forum, running again at the Hotel Seattle down at Brighton Marina. Around seventy of Europe’s most experienced OBIEE, ODI and Essbase developers will be getting together to discuss techniques, share tips and take part in sessions led by some of the OBIEE world’s best speakers, all in our home town of Brighton. We’re also very pleased to be joined by several of the OBIEE and ODI product team members, as well as our special guest speaker, Toby Potter from Datasift. In this posting I’ll be setting out the detailed agenda for the three days, and don’t forget places are still (just) available for the Atlanta event, running the week after at the Georgia Tech Hotel & Conference Center.
Registration for the Brighton BI Forum officially opens at 5pm on Wednesday evening, but before that we’re running an optional one-day Oracle Data Integration Masterclass at the hotel venue led by myself, Stewart Bryson and Michael Rainey. The masterclass will start at 10am and go through until about 4.30pm, with the following estimated timings:
Day 1 : Optional Oracle Data Integration Masterclass, followed by Registration, Drinks and Keynote/Meal
10.00 – 11.00 : Welcome, and Introduction to Oracle Data Integrator 11g (Stewart Bryson)
11.00 – 11.15 : Morning Coffee
11.15 – 11.45 : ODI and the Oracle Reference Architecture for Information Management (Stewart Bryson)
11.45 – 12.45 : ODI and GoldenGate – A Perfect Match… (Michael Rainey)
12.45 – 13.30 : Lunch
13.30 – 14.30 : ODI and Hadoop, MapReduce and Big Data Sources (Mark Rittman)
14.30 – 15.30 : The Three R’s of ODI Fault Tolerance : Resuming, Restarting and Restoring (Stewart Bryson)
15.30 – 16.30 : Scripting and Automating ODI using Groovy and the ODI SDK (Michael Rainey)
Registration will then open at 5pm (17.00), with a drinks reception in the hotel bar between 6pm and 7pm (18.00 – 19.00), followed by the Oracle keynote and opening night meal in the hotel restaurant.
The main conference then opens at 8.45am on the Thursday morning, with the agenda for the first day looking like this:
Day 2 : Main Conference Sessions, Guest Speaker and Gala Meal
08.45 – 09.00 : Opening Remarks Mark Rittman, Rittman Mead
09.00 – 10.00 : “OBIEE SampleApp 11.1.1.7 functional highlights” (Philippe Lions, Oracle Corporation)
10.00 – 10.30 : Morning Coffee
10.30 – 11.30 : “OBI Performance Tuning – Real Customer Success Stories” (Antony Heljula, Peak Indicators Ltd)
11.30 – 12.30 : “Secrets of OBIEE implementation at LGI” (Marco Klaassens, Liberty Global)
12.30 – 13.15 : Lunch
13.15 – 13.30 : TED Session 1: ”Why I want to be working with Business Intelligence in 5 years time” (Jon Mead, Rittman Mead)
13.30 – 13.45 : TED Session 2 : “HA, DR, CFC, WTF?” (Mike Durran, Oracle Corporation)
13.45 – 14.00 : TED Session 3 : “Incrementally loading Exalytics using Notepad” (Antony Heljula, Peak Indicators Ltd)
14.15 – 15.15 : “Oracle Data Integrator 11g Best Practices. Busting your performance, deployment, and scheduling headaches.” (Uli Bethke/Maciek Kocon, Independent)
15.15 – 15.45 : Afternoon Tea/Coffee/Beers
15.45 – 16.45 : “New Developments in BI Multi-tenancy and Cloud” (Adam Bloom, Oracle Corporation)
After the first day’s presentations we’ll take a short break, and then convene again back in the conference room for our special guest speaker session, this year being provided by Toby Potter from Datasift, who’ll talk to us about social media and how it can be used within BI and analytic applications. Then, we’ll be picked-up by coach from outside the Hotel Seattle and taken to the venue for our gala meal, The Ginger Pig pub and restaurant in Hove, just down the road from where Jon and I live (though don’t all expect to come back afterwards, whatever Borkur and Ragnar tell you when you leave the pub.)
18:00 – 18.45 Guest Keynote Toby Potter, Datasift – “Social Data and Business Intelligence”
19:00 – 19.30 Depart for Restaurant
20:00 – 22:00 Gala Meal : The Ginger Pig, Hove
Don’t go too crazy on Thursday evening though, as we’re starting again early on Friday with sessions starting at 9am, going through until around 4.45 when we’ll close. Here’s Friday’s agenda:
Day 3 : Main Conference Sessions, and Close
9.00 – 10.00 : “The magic of aggregates” (Michael Wilcke, sumIT AG)
10.00 – 10.30 : Morning Coffee
10.30 – 11.30 : “Update on BI Metadata Modeler and Metadata Architecture” (Philippe Lions, Oracle)
11.30 – 12.30 : “Integrating Oracle BI, BPM and BAM 11g: The complete cycle of information” (Edelweiss Kammermann, Awen Consulting)
12.30 – 13.15 : Lunch
13.15 – 14.15 : Debate “Big Data – Hype, or the Future or Oracle BI/DW?”
14.15 – 15.15 : “Endeca – Looking beyond the general demos” (Adam Seed, Rittman Mead)
15.15 – 15.30 : Afternoon Tea/Coffee/Beers
15.30 – 16.30 : “Virtualizing Exalytics” (Mike Durran, Oracle Corporation)
16.30 – 16.45 : Closing Remarks (Mark Rittman)
You’ll notice we’ve brought back the popular “debate” section this year, with this year’s topic being “Big Data – Hype, or the Future of BI/DW?”. I’ll be looking for volunteers to argue the case for either of the two sides in the debate, so if you’ve got a view on whether big data is going to be the salvation of BI, whether it’ll turn us in to the COBOL programmers of the future, or whether its just a load of hot air (or you just like having an argument), let me know when you arrive and we’ll pull the debating teams together.
Other than that – have a safe journey over, bring something warm and waterproof as the weather is typically England in May, and see at least some of you in Brighton today and tomorrow!
Agenda and Details for the Brighton RM BI Forum 2013
It’s Tuesday morning, and later today delegates will be starting to arrive in Brighton for the 5th Annual Rittman Mead BI Forum, running again at the Hotel Seattle down at Brighton Marina. Around seventy of Europe’s most experienced OBIEE, ODI and Essbase developers will be getting together to discuss techniques, share tips and take part in sessions led by some of the OBIEE world’s best speakers, all in our home town of Brighton. We’re also very pleased to be joined by several of the OBIEE and ODI product team members, as well as our special guest speaker, Toby Potter from Datasift. In this posting I’ll be setting out the detailed agenda for the three days, and don’t forget places are still (just) available for the Atlanta event, running the week after at the Georgia Tech Hotel & Conference Center.
Registration for the Brighton BI Forum officially opens at 5pm on Wednesday evening, but before that we’re running an optional one-day Oracle Data Integration Masterclass at the hotel venue led by myself, Stewart Bryson and Michael Rainey. The masterclass will start at 10am and go through until about 4.30pm, with the following estimated timings:
Day 1 : Optional Oracle Data Integration Masterclass, followed by Registration, Drinks and Keynote/Meal
10.00 – 11.00 : Welcome, and Introduction to Oracle Data Integrator 11g (Stewart Bryson)
11.00 – 11.15 : Morning Coffee
11.15 – 11.45 : ODI and the Oracle Reference Architecture for Information Management (Stewart Bryson)
11.45 – 12.45 : ODI and GoldenGate – A Perfect Match… (Michael Rainey)
12.45 – 13.30 : Lunch
13.30 – 14.30 : ODI and Hadoop, MapReduce and Big Data Sources (Mark Rittman)
14.30 – 15.30 : The Three R’s of ODI Fault Tolerance : Resuming, Restarting and Restoring (Stewart Bryson)
15.30 – 16.30 : Scripting and Automating ODI using Groovy and the ODI SDK (Michael Rainey)
Registration will then open at 5pm (17.00), with a drinks reception in the hotel bar between 6pm and 7pm (18.00 – 19.00), followed by the Oracle keynote and opening night meal in the hotel restaurant.
The main conference then opens at 8.45am on the Thursday morning, with the agenda for the first day looking like this:
Day 2 : Main Conference Sessions, Guest Speaker and Gala Meal
08.45 – 09.00 : Opening Remarks Mark Rittman, Rittman Mead
09.00 – 10.00 : “OBIEE SampleApp 11.1.1.7 functional highlights” (Philippe Lions, Oracle Corporation)
10.00 – 10.30 : Morning Coffee
10.30 – 11.30 : “OBI Performance Tuning – Real Customer Success Stories” (Antony Heljula, Peak Indicators Ltd)
11.30 – 12.30 : “Secrets of OBIEE implementation at LGI” (Marco Klaassens, Liberty Global)
12.30 – 13.15 : Lunch
13.15 – 13.30 : TED Session 1: ”Why I want to be working with Business Intelligence in 5 years time” (Jon Mead, Rittman Mead)
13.30 – 13.45 : TED Session 2 : “HA, DR, CFC, WTF?” (Mike Durran, Oracle Corporation)
13.45 – 14.00 : TED Session 3 : “Incrementally loading Exalytics using Notepad” (Antony Heljula, Peak Indicators Ltd)
14.15 – 15.15 : “Oracle Data Integrator 11g Best Practices. Busting your performance, deployment, and scheduling headaches.” (Uli Bethke/Maciek Kocon, Independent)
15.15 – 15.45 : Afternoon Tea/Coffee/Beers
15.45 – 16.45 : “New Developments in BI Multi-tenancy and Cloud” (Adam Bloom, Oracle Corporation)
After the first day’s presentations we’ll take a short break, and then convene again back in the conference room for our special guest speaker session, this year being provided by Toby Potter from Datasift, who’ll talk to us about social media and how it can be used within BI and analytic applications. Then, we’ll be picked-up by coach from outside the Hotel Seattle and taken to the venue for our gala meal, The Ginger Pig pub and restaurant in Hove, just down the road from where Jon and I live (though don’t all expect to come back afterwards, whatever Borkur and Ragnar tell you when you leave the pub.)
18:00 – 18.45 Guest Keynote Toby Potter, Datasift – “Social Data and Business Intelligence”
19:00 – 19.30 Depart for Restaurant
20:00 – 22:00 Gala Meal : The Ginger Pig, Hove
Don’t go too crazy on Thursday evening though, as we’re starting again early on Friday with sessions starting at 9am, going through until around 4.45 when we’ll close. Here’s Friday’s agenda:
Day 3 : Main Conference Sessions, and Close
9.00 – 10.00 : “The magic of aggregates” (Michael Wilcke, sumIT AG)
10.00 – 10.30 : Morning Coffee
10.30 – 11.30 : “Update on BI Metadata Modeler and Metadata Architecture” (Philippe Lions, Oracle)
11.30 – 12.30 : “Integrating Oracle BI, BPM and BAM 11g: The complete cycle of information” (Edelweiss Kammermann, Awen Consulting)
12.30 – 13.15 : Lunch
13.15 – 14.15 : Debate “Big Data – Hype, or the Future or Oracle BI/DW?”
14.15 – 15.15 : “Endeca – Looking beyond the general demos” (Adam Seed, Rittman Mead)
15.15 – 15.30 : Afternoon Tea/Coffee/Beers
15.30 – 16.30 : “Virtualizing Exalytics” (Mike Durran, Oracle Corporation)
16.30 – 16.45 : Closing Remarks (Mark Rittman)
You’ll notice we’ve brought back the popular “debate” section this year, with this year’s topic being “Big Data – Hype, or the Future of BI/DW?”. I’ll be looking for volunteers to argue the case for either of the two sides in the debate, so if you’ve got a view on whether big data is going to be the salvation of BI, whether it’ll turn us in to the COBOL programmers of the future, or whether its just a load of hot air (or you just like having an argument), let me know when you arrive and we’ll pull the debating teams together.
Other than that – have a safe journey over, bring something warm and waterproof as the weather is typically England in May, and see at least some of you in Brighton today and tomorrow!
New Oracle Magazine Article on Integrating OBIEE 11g with Oracle WebCenter Portal: Spaces
Just a short note to say that my latest Oracle Magazine article is now online, and talks about integrating OBIEE 11g with Oracle WebCenter Portal: Spaces, the “spiritual successor” to Oracle Portal. “New Spaces for BI” introduces Portal: Spaces talks about the high-level benefits of integration, and then goes through a step-by-step example where BI content gets added to a Portal : Spaces team website.
Writing this article brought back a few memories, as I wrote the article for OTN back in 2005 on integrating Oracle Discoverer with Oracle Portal, with the same end-objective of creating a composite intranet-style web application that placed BI content alongside more “line-of-business” web portlets. The technology has come on a lot since the time of that article – apart from Discoverer, we were using Oracle Application Server 10g (pre-BEA and WebLogic), Portal itself and Oracle Database 10g, and whilst the approach in the other article worked, you didn’t have a lot of control over the layout and there was a weird mechanism in the background that showed you cached data, rather than live data in the Discoverer port lets. But the basic principal was the same though – register the Discoverer End-User Layer as a Portlet provider, as we do with OBIEE 11g’s Presentation Services catalog in the Oracle Magazine article, and then users can drag and drop their favourite BI content into their corporate portal.
Also, if you’re interested, I covered OBIEE 11g and WebCenter integration in a lot more detail back towards the end of 2012, with a series of postings on the blog starting with an introduction to Oracle WebCenter and WebCenter Portal, a more detailed look at OBIEE and WebCenter Portal : Spaces integration, and then finally a posting on leveraging the WebCenter Portal : Framework ADF toolkit with OBIEE 11g to programmatically bring together BI content and a portal framework as a custom ADF application.
