In my previous posting in this series, I looked at the new 11.1.1..7.1 release of the Oracle BI Applications at a high-level, and talked about how this new release uses ODI as the embedded ETL tool instead of Informatica PowerCenter. Support for Informatica will come with patch set 2 (PS2) of BI Apps 11.1.1.7.x giving customers the choice of which ETL to use (with the caveat that customers upgrading from 7.9.x will typically have to stick with Informatica unless they want to completely re-implement using ODI), but for this initial release at least, ODI and some new Fusion Middleware tools take over from Informatica and the DAC, giving us what could well be a much simpler architecture for supplying the underlying data for the BI Apps dashboards.

In this posting then, I’m going to take a closer look at this new product architecture, and I’ll follow it with a more detailed look at how the various bits of ODI functionality replace the workflows, mappings, transformation operators and execution plans provided in earlier releases by Informatica and the DAC. For anyone familiar with the previous, 7.9.x versions of the BI Applications, the architecture diagram below shows the five tiers that this product typically implemented; tiers for the source data and data warehouse/repository databases, an ETL tier for Informatica and the DAC server, then two more tiers for the OBIEE application server and the client web browser.

Communication between the tiers was – to put it politely – “loosely coupled”, with DAC task names corresponding with Informatica workflow names, each workflow containing a single mapping, and all of the connections and sources having to be named “just so”, so that every part of the stack could communicate with all the others. It worked, but it was a lot of work to implement and configure, and once it was up and running in most cases customers were scared to then change it, in case a name or a connection got out of sync and everything then stopped working. Plus – Informatica skills are scarce in the Oracle world, and the DAC is an extra piece of technology that few DBAs really understood properly.

The 11.1.1.7.1 release of the BI Apps simplifies this architecture by removing the separate ETL tier, and instead using Oracle Data Integrator as the embedded ETL tool, with its server functions running as JEE applications within the same WebLogic domain as OBIEE 11g, giving us the overall architecture in the diagram below.

Now anyone who read my series of posts back in 2009 on the 7.9.5.2 release of the BI Apps, which also used ODI as the embedded ETL tool, will know that whilst ODI 10g could do the job of loading data into the BI Apps data warehouse, it lacked the load orchestration capabilities of Informatica and the DAC and wasn’t really set up to dynamically generate what have become, in ODI 11g, load plans. BI Apps 7.9.5.2 turned-out to be a one-off release and in the intervening years Oracle have added the aforementioned load plans along with other functionality aimed at better supporting the BI Apps, along with two new JEE applications that run in WebLogic to replace the old DAC.  These new applications, along with the ODI JEE agent, ODI Console and the ODI SDK, are shown in the more detailed BI Applications 11.1.1.7.1 logical architecture diagram shown below.

Oracle BI Applications 11.1.1.7.1 has two main product tiers to it, made up of the following components:

• The Middleware (BI and ETL) tier; a WebLogic domain and associated system components, comprising BI components delivered as part of OBIEE 11.1.1.7 (including Essbase and related applications) as one managed server, and another managed server containing ODI Java components, including three new BI Apps-related ones; Configuration Manager, Functional Setup Manager, and ODI Load Plan Generator
• The Database (DW and Repositories) tier; for the time-being, Oracle only, and comprising a data warehouse schema (staging + performance layer), and a repository database containing the OBIEE repository schemas plus new ones to hold the ODI repository and other ETL/configuration metadata used for configuring your system.

Essbase at this stage is installed, but not used for the main BI applications, and all of it uses Fusion Middleware security (application roles and policies) along with the WebLogic Embedded LDAP server to hold users and groups. A special version of RCU is used to set up the new BI Apps-related schemas, and import data into them using Oracle database export files, so that the ODI repository, metadata tables and so forth are all populated prior to the first load taking place. Enterprise Manager Fusion Middleware Control is still used to manage and monitor the overall platform, but there’s now an entry for ODI along with Essbase, the latter of course being delivered as part of the 11.1.1.7 OBIEE platform release.

In the next posting in the series we’ll take a closer look at how ODI uses its JEE agent and mappings imported into its repository to load the BI Apps data warehouse, but what about the two new web-based configuration tools, Oracle BI Applications Configuration Manager (BIACM) and Oracle BI Applications Functional Setup Manager (FSM) – what do they do?

After you install OBIEE 11.1.1.7 and then the BI Applications 11.1.1.7.1, the BI Apps installer extends the BI domain to include FSM, BIACM and the ODI Load Plan Generator, along with some other supporting applications and libraries required for the full product. Load Plan Generator works behind the scenes to build new load plans in a similar way to the Execution Plan “Build” feature in the DAC, and the two web-based tools perform the following functions:

• Oracle BI Applications Configuration Manager performs system-wide setup tasks such as defining sources, selecting BI Apps modules and performing other, “one-only” tasks similar to the Setup feature in the DAC Console.
• Oracle BI Applications Functional Setup Manager is then used to list out, and track progress against, the various tasks required to configure the BI Applications modules, or “Offerings”, that you selected in the Configuration Manager

Most importantly though, these tools connect directly through to the DAC repository, so data sources you set up here will get pushed down to ODI as data servers in the ODI master repository; load plans you set up to, as in the screenshot below, load configuration tables, are ODI load plans and you can track their progress either from within ODI, or from within these applications themselves.

I haven’t had a chance to properly “diff” the RPD used in BI Apps 11.1.1.7.1 with the previous 7.9.x ones, or do a similar exercise for the underlying database data model, but on first glance the new RPD is at least recognisable, albeit with new sources and subject areas for the Fusion Apps, Oracle Transactional BI (OTBI), Real-Time Decisions and the like. The web catalog also looks familiar, but also has new content around the new applications along with additional content for the existing ones.

So, we’re at the point now where can start to think about loading data into the BI Apps data warehouse, and in tomorrows post we’ll take a look at what’s involved in a BI Apps 11.1.1.7.1 ETL load, and also look into how GoldenGate can now be used to extract and stage data prior to loading via ODI. Back tomorrow…

In my previous posting in this series, I looked at the new 11.1.1..7.1 release of the Oracle BI Applications at a high-level, and talked about how this new release uses ODI as the embedded ETL tool instead of Informatica PowerCenter. Support for Informatica will come with patch set 2 (PS2) of BI Apps 11.1.1.7.x giving customers the choice of which ETL to use (with the caveat that customers upgrading from 7.9.x will typically have to stick with Informatica unless they want to completely re-implement using ODI), but for this initial release at least, ODI and some new Fusion Middleware tools take over from Informatica and the DAC, giving us what could well be a much simpler architecture for supplying the underlying data for the BI Apps dashboards.

In this posting then, I’m going to take a closer look at this new product architecture, and I’ll follow it with a more detailed look at how the various bits of ODI functionality replace the workflows, mappings, transformation operators and execution plans provided in earlier releases by Informatica and the DAC. For anyone familiar with the previous, 7.9.x versions of the BI Applications, the architecture diagram below shows the five tiers that this product typically implemented; tiers for the source data and data warehouse/repository databases, an ETL tier for Informatica and the DAC server, then two more tiers for the OBIEE application server and the client web browser.

Communication between the tiers was – to put it politely – “loosely coupled”, with DAC task names corresponding with Informatica workflow names, each workflow containing a single mapping, and all of the connections and sources having to be named “just so”, so that every part of the stack could communicate with all the others. It worked, but it was a lot of work to implement and configure, and once it was up and running in most cases customers were scared to then change it, in case a name or a connection got out of sync and everything then stopped working. Plus – Informatica skills are scarce in the Oracle world, and the DAC is an extra piece of technology that few DBAs really understood properly.

The 11.1.1.7.1 release of the BI Apps simplifies this architecture by removing the separate ETL tier, and instead using Oracle Data Integrator as the embedded ETL tool, with its server functions running as JEE applications within the same WebLogic domain as OBIEE 11g, giving us the overall architecture in the diagram below.

Now anyone who read my series of posts back in 2009 on the 7.9.5.2 release of the BI Apps, which also used ODI as the embedded ETL tool, will know that whilst ODI 10g could do the job of loading data into the BI Apps data warehouse, it lacked the load orchestration capabilities of Informatica and the DAC and wasn’t really set up to dynamically generate what have become, in ODI 11g, load plans. BI Apps 7.9.5.2 turned-out to be a one-off release and in the intervening years Oracle have added the aforementioned load plans along with other functionality aimed at better supporting the BI Apps, along with two new JEE applications that run in WebLogic to replace the old DAC.  These new applications, along with the ODI JEE agent, ODI Console and the ODI SDK, are shown in the more detailed BI Applications 11.1.1.7.1 logical architecture diagram shown below.

Oracle BI Applications 11.1.1.7.1 has two main product tiers to it, made up of the following components:

• The Middleware (BI and ETL) tier; a WebLogic domain and associated system components, comprising BI components delivered as part of OBIEE 11.1.1.7 (including Essbase and related applications) as one managed server, and another managed server containing ODI Java components, including three new BI Apps-related ones; Configuration Manager, Functional Setup Manager, and ODI Load Plan Generator
• The Database (DW and Repositories) tier; for the time-being, Oracle only, and comprising a data warehouse schema (staging + performance layer), and a repository database containing the OBIEE repository schemas plus new ones to hold the ODI repository and other ETL/configuration metadata used for configuring your system.

Essbase at this stage is installed, but not used for the main BI applications, and all of it uses Fusion Middleware security (application roles and policies) along with the WebLogic Embedded LDAP server to hold users and groups. A special version of RCU is used to set up the new BI Apps-related schemas, and import data into them using Oracle database export files, so that the ODI repository, metadata tables and so forth are all populated prior to the first load taking place. Enterprise Manager Fusion Middleware Control is still used to manage and monitor the overall platform, but there’s now an entry for ODI along with Essbase, the latter of course being delivered as part of the 11.1.1.7 OBIEE platform release.

In the next posting in the series we’ll take a closer look at how ODI uses its JEE agent and mappings imported into its repository to load the BI Apps data warehouse, but what about the two new web-based configuration tools, Oracle BI Applications Configuration Manager (BIACM) and Oracle BI Applications Functional Setup Manager (FSM) – what do they do?

After you install OBIEE 11.1.1.7 and then the BI Applications 11.1.1.7.1, the BI Apps installer extends the BI domain to include FSM, BIACM and the ODI Load Plan Generator, along with some other supporting applications and libraries required for the full product. Load Plan Generator works behind the scenes to build new load plans in a similar way to the Execution Plan “Build” feature in the DAC, and the two web-based tools perform the following functions:

• Oracle BI Applications Configuration Manager performs system-wide setup tasks such as defining sources, selecting BI Apps modules and performing other, “one-only” tasks similar to the Setup feature in the DAC Console.
• Oracle BI Applications Functional Setup Manager is then used to list out, and track progress against, the various tasks required to configure the BI Applications modules, or “Offerings”, that you selected in the Configuration Manager

Most importantly though, these tools connect directly through to the ODI repository, so data sources you set up here will get pushed down to ODI as data servers in the ODI master repository; load plans you set up to, as in the screenshot below, load configuration tables, are ODI load plans and you can track their progress either from within ODI, or from within these applications themselves.

I haven’t had a chance to properly “diff” the RPD used in BI Apps 11.1.1.7.1 with the previous 7.9.x ones, or do a similar exercise for the underlying database data model, but on first glance the new RPD is at least recognisable, albeit with new sources and subject areas for the Fusion Apps, Oracle Transactional BI (OTBI), Real-Time Decisions and the like. The web catalog also looks familiar, but also has new content around the new applications along with additional content for the existing ones.

So, we’re at the point now where can start to think about loading data into the BI Apps data warehouse, and in tomorrows post we’ll take a look at what’s involved in a BI Apps 11.1.1.7.1 ETL load, and also look into how GoldenGate can now be used to extract and stage data prior to loading via ODI. Back tomorrow…

The long-awaited and anticipated 11.1.1.7.1 (PS1) release of the Oracle BI Applications became available early last week, with the software and documentation available for download on OTN. Over the next few blog posts, I’ll be taking an in-depth look at this new release, starting today with an overview of what’s new and any limitations in his initial version, and then over the next few posts taking a look at the product architecture, how it uses Oracle Data Integrator instead of Informatica to do the data loads, and what new content the 11g dashboards contain. For a bit of background into this release you’re best off taking a look at a series of posts I put together towards the end of last year on the BI Apps product roadmap, and I’ll recap on those posts a bit in this one as I go through at a high level what’s in this release.

Although the focus in BI Apps 11.1.1.7.1 is on ODI as the ETL tool, this new release actually delivers a whole new product architecture along with new dashboards, new content, and a new security framework. In addition, there’s now an option to use Oracle GoldenGate to create a new layer in the BI Apps data warehouse data architecture that replicates source data into the warehouse environment, giving you the ability to run the more large-scale ETL processes when you like, rather than when there’s an ETL window for the source systems.

Let’s start off though with a summary of what’s new from a functional perspective, and also what limitations there are for this first release in terms of sources, scope and so forth. BI Apps 11.1.1.7.1 delivers the following set of new features and capabilities:

• Oracle Data Integrator as the embedded ETL tool, along with a whole new FMW11g-centric architecture and set of utilities
• Two new analytic applications - Student Information Analytics, and Indirect Spend Planning
• New content for existing analytic applications including Financial Analytics, HR, Projects, CRM and Procurement & Spend
• Dashboards that are now written for OBIEE 11g rather than 10g, including 11.1.1.7 visualisations such as performance tiles

Now although, in general terms, BI Apps 11.1.1.7.1 covers all (or most…) of the existing analytic application modules along with all of the 7.9.x-era sources (EBS, PeopleSoft, JDE and Siebel), there are some important restrictions that you’ll need to be aware of when making any plans to use this new release, starting with upgrade paths (or lack of them):

• There’s no automatic upgrade path from BI Apps 7.9.x, and no automated migration routine to take you from Informatica to ODI; if you want BI Apps 11.1.1.7.1 now, you’ll have to reimplement rather than upgrade, or you can wait for BI Apps 11.1 PS2 which will support upgrades from earlier releases, but (important to note) keeps you on Informatica – any move from Informatica to ODI will need to be done yourself, as a re-implementation
• Only Oracle database sources and targets are supported in this initial release, in practice not a real issue for new implementations, but worth bearing in mind if you planned to use Teradata, for example, as your target data warehouse platform
• Oracle Fusion Applications aren’t supported as a source either, yet, so anyone using this will need to stay on BI Apps 11.1.1.6.x until an upgrade version becomes available
• A few edge-case analytic applications and sources aren’t supported in this release yet – Enterprise Asset Management, for example, is not yet supported for any source, whereas some other applications only support more recent PeopleSoft versions and not JDE, for example. As always, get the most up-to-date supported sources and applications list from Oracle before making any major investment in an implementation or upgrade project.

From a technical perspective though the major difference in this release, compared to the 7.9.6.x versions that preceded it, is the use of Oracle Data Integrator 11g as the embedded ETL tool rather than Informatica. To be clear, Informatica will still be supported as an ETL option for the BI Apps going well into the foreseeable future, but Informatica users will need to wait for the PS2 release due in the next twelve months or so before they can upgrade to the new 11g platform.

In addition and perhaps more importantly, it’s not envisaged that Informatica customer will move over to ODI unless they use the upgrade as an opportunity to re-implement their system now on ODI, moving across customisations themselves and essentially starting with a clean sheet of paper (which may not be a bad thing, if you’re thinking of tidying up your system following years of upgrades, customisations and so forth). What this does mean though is no DAC, no Informatica server and client tools, a new (and hopefully simpler) way of setting up and configuring your system, and in-theory a more closely-integrated set of tools all based around the modern, standards-based Fusion Middleware 11g architecture. 

In this new world of ODI and the BI Apps, ODI load plans replace Informatica Workflows, whilst ODI packages and interfaces equate to Informatica mappings and maplets. The DAC is no more and is replaced by metadata within the ODI repository and other supporting schemas, with setup and configuration of the warehouse and ETL processes now carried out by two web-based tools, BI Applications Configuration Manager and Functional Setup Manager. The closer integration between these tool along with a chance for Oracle to re-think the BI Apps setup process should lead to easier configuration and customisations, but if you’re an Informatica developer it’s a whole new world, and the 11g platform makes a lot more use of Fusion Middleware platform functionality particularly around security and user provisioning.

So – all very exciting but quite daunting in terms of what needs to be learnt, and new processes that need to be thought through and put together before you can start making use of the new 11.1.1.7.1 feature set. We’ll start tomorrow then by taking a closer look at the BI Apps 11.1.1.7.1 technical architecture including the new configuration tools, and where ODI sits in the new product architecture, based on our first impressions of the product.