Oracle 11g and OBIEE Data Warehousing and Analytics

Analytics and DW Using Oracle 11g and OBIEE 10g Mark Rittman, Director, Rittman Mead Consulting

Who Am I? • Oracle BI&W Architecture and Development Specialist • Co-Founder of Rittman Mead Consulting • Oracle BI&W Project Delivery Specialists • 10+ years with Discoverer, OWB etc • Oracle ACE Director, ACE of the Year 2005 • Writer for OTN and Oracle Magazine • Longest-running Oracle blog • http://www.rittmanmead.com/blog • Chair of UKOUG BIRT SIG • Co-Chair of ODTUG BI&DW SIG • Second year of OU BI Masterclasses • 18 countries visited in 2006-7

Rittman Mead Consulting • Oracle BI&DW Project Specialists • Consulting, Training, Support • Works with you to ensure OBIEE project success • Small, focused team • OWB, Oracle BI, DW technical specialists • Clients in the UK, Europe, USA

The Oracle 11g Data Warehousing Platform • The latest release of the Oracle database • Enterprise Edition contains a large number of data warehousing features • Partitioning • Bitmap indexes • Star transformations • Compression • Options for in-database OLAP and Data Mining • Oracle Warehouse Builder for database-centric ETL and Data Quality

The Oracle 10g Business Intelligence Platform • Based around Oracle Business Intelligence Enterprise Edition • Dashboards, Answers, Delivers, Oracle BI Server • Complete set of web-based query, reporting and analysis tools • Based on Siebel Analytics • Heterogeneous database access • Logical Business Model • Data integration capabilities • Add-ons for data integration, real-time decisions

Oracle Fusion Middleware • Middleware architecture for Oracle products • SOA-enabled J2EE environment • Security, BI, user interaction, systems management, tools, BPM

Hyperion Financial Management Tools • Financial Management tools • Hyperion Planning, Hyperion Performance Scorecard • Hyperion Capital Expenses Planning, Hyperion Workforce Planning • Essbase OLAP Server • Hyperion System 9 infrastructure • ex-Brio reporting and dashboard tools

The Future Oracle BI&DW Platform • Leverages the analytic capabilities of the Oracle database • In-database OLAP, data mining, partitioning, analytic queries • Scalable store of detail-level data for analysis • Business logic and metadata layer provided by Oracle BI Server (OBIEE) • Semantic model, calculations, time-series functions, hierarchies • Oracle BI Presentation Server (OBIEE) plus Hyperion EPM tools provide Presentation Layer • Three-tier BI&DW architecture leverages Oracle RDBMS and OBIEE/Hyperion toolset

The Data Services Layer • Based around a core Oracle Data Warehouse • Or several federated data warehouses • Data Warehouse has three tiers • Staging • Atomic/Normalized • Performance/Dimensional • ETL process for loading DW • OWB (default ETL tool) • ODI (heterogenous sources /targets) • Data Services Layer also provides real-time and SOA feeds to OBIEE • For applications not yet in DW • For real-time reports • For transitory requirements

The Business Logic Layer • Contains an integrated, logical model of the business • Provides an abstraction layer for the business • Can be auto-generated by the ETL tool • OWB 11gR2 • Maintained by the business • Can connect to multiple sources in the data services layer • The Oracle data warehouse • Direct application access • OLAP cubes • Relational databases

The Presentation Layer • Data access layer, query tools, interfaces, MS Office integration • Oracle BI Answers, Dashboard, BI Publisher • BI Office integration • SOAP (SOA) interface • ODBC interface (BO, Cognos etc) • Hyperion tools • May access Essbase directly, by-passing Business Logic Layer

Analytics and DW Using Oracle 11g and OBIEE 10g • An in-depth look at the technologies behind Oracle’s future BI&DW platform • What’s new in Oracle Database 11g for data warehousing • How the Oracle BI Server provides the business logic for your BI&DW platform • Creating the Enterprise Semantic Model, how the technology works and what are its strengths and weaknesses • Where to place calculations, summaries and hierarchies in your architecture • How Oracle BI Presentation Server provides dashboards and reports (and what it’s missing in the current release) • Where the Hyperion tools fit in • What’s new in Oracle OLAP 11g, and an introduction to Oracle Essbase

Seminar Agenda • Introduction • ETL and DW development using Oracle 11g and OWB 11g Coffee • Creating the BI foundation later with OBIEE BI Server Lunch • Reporting and Dashboards using OBIEE Presentation Server Coffee • Analytics and OLAP Analysis using Oracle OLAP and Oracle Essbase Close

Seminar Material • Presentation slides are available in PDF format • http://www.rittmanmead.com/presentations • Leave a business card and I will forward on copies, answer questions • Visit http://www.rittmanmead.com for more OBIEE material

Participant Introduction • The most important people in this room are you • It helps if I know a bit about you • Your Oracle BI experience • Your course objectives • Where you work • Introduce yourself to me, and to the other seminar participants

Seminar Participation • Questions are welcome at any time • During the session • During breaks • At the end of the day • Hopefully I’ll know the answer • If not, I’ll get back to you • Your feedback is appreciated • What worked well, what could be better • Flow of slides and product coverage • Fill in the evaluation forms (Oracle and mine)

Analytics and DW Using Oracle 11g and OBIEE 10g Mark Rittman, Director, Rittman Mead Consulting

Analytics and DW Using Oracle 11g and Oracle BIEE Lesson 1 : ETL and DW development using Oracle 11g and OWB 11g (and ODI)

Objectives • Understand the data warehousing features in the Oracle database • See what’s new in Oracle Database 11g for data warehousing • Partitioning • Cube Organized Materialized Views • See what’s new in OWB11g • SCD2 Handling that works (sort of) • DML Error Logging • Get an introduction to ODI 10.1.3.4 • Web Services functionality

The Oracle 11g Data Warehousing Platform • Oracle Database 11g • Oracle Warehouse Builder • Oracle OLAP Option • Oracle Data Mining Option • Partitioning Option

The Future Oracle BI&DW Platform • Leverages the analytic capabilities of the Oracle database • In-database OLAP, data mining, partitioning, analytic queries • Scalable store of detail-level data for analysis • Business logic and metadata layer provided by Oracle BI Server (OBIEE) • Semantic model, calculations, time-series functions, hierarchies • Oracle BI Presentation Server (OBIEE) plus Hyperion EPM tools provide Presentation Layer • Three-tier BI&DW architecture leverages Oracle RDBMS and OBIEE/Hyperion toolset

The Data Services Layer • Based around a core Oracle Data Warehouse • Or several federated data warehouses • Data Warehouse has three tiers • Staging • Atomic/Normalized • Performance/Dimensional • ETL process for loading DW • OWB (default ETL tool) • ODI (heterogenous sources /targets) • Data Services Layer also provides real-time and SOA feeds to OBIEE • For applications not yet in DW • For real-time reports • For transitory requirements

Oracle 11g Enterprise Edition Code Features • Bitmap Indexes • Materialized Views • Star Transformations • Cost-Based Optimizer • Oracle Warehouse Builder Core Functionality

Oracle 11g Data Warehousing Options • Partitioning • OLAP • Data Mining • OWB Data Quality Option • OWB Enterprise ETL Option • OWB ERP Connectors

Partitioning • Use table partitions to • make large (fact) tables more manageable • ‘age out’ old data by dropping partitions • add new partitions to receive new data • set unchanging tablespaces to READ ONLY to optimize database backup • speed data load by using partition exchange • can reduce query time by • partition elimination (pruning) • parallel query

Annual Data

Jan

Mar Feb

Oracle OLAP Option • Adds multi-dimensional data types and a multi-dimensional engine • Stores data in indexed arrays • Adds an OLAP API and an SQL access layer • (Can be) Faster than star schemas for loading and querying • Supports time-series and dimensional queries • Based on Express Server technology • Provides consistently-fast access to summarized data

Oracle Data Mining Option • In-database data mining engine • Accessed via Java and PL/SQL APIs • Wide range of algorithms • Anomaly detection • Attribute importance • Association rules • Clustering • Classification & regression • Nonnegative matrix factorization • Structured & unstructured data (text mining) • BLAST (life sciences similarity search algorithm)

New Features in Oracle OLAP 11g • • • •

• • • •

The latest release of the Oracle Database OLAP Option Continues the objective of embedding OLAP in the database engine SQL SELECT access Cubes stored in Analytic Workspaces • Runs faster in 11g, uses SecureFiles instead of LOBs Metadata in the Oracle Data Dictionary Automatic creation of views over AW objects • Optimizer now “AW view aware” CUBE_TABLE for easier direct SQL access CUBE access now detailed in execution plans

Cube Organized Materialized Views • Transparently enhance the query performance of BI applications • Data is managed in an Oracle cube • Fast refresh • Fast query • Manage a single cube instead of 100’s of individual MVs • Applications query the base relational tables • Oracle automatically rewrites the queries to use the OLAP cube • Access to the summary data in the OLAP cube is fully transparent

Positioned As the Oracle DW Enhancer • • • •

Transparent integration with SQL Swap-in replacement for Materialized Views Refreshed and loaded in the same way as Materialized Views Either accessed directly using SQL views, or indirectly through query rewrite

OLAP Cubes Now Feature in Explain Plans • Cube operations now feature in execution plans • CUBE_SCAN shows data being retrieved from a Cube Organized MV explain plan for select sum( u.units), c.market_segment_dsc, t.year_dsc, p.family_id from units_fact u, customer_dim c, time_dim t, product_dim p where u.ship_to_id = c.ship_to_id and u.month_id = t.month_id and u.item_id = p.item_id group by c.market_segment_dsc, t.year_dsc, p.family_id; PLAN_TABLE_OUTPUT ----------------------------------------Plan hash value: 3517365018 ------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 19 | 2375 | 37 (6)| 00:00:01 | | 1 | HASH GROUP BY | | 19 | 2375 | 37 (6)| 00:00:01 | |* 2 | HASH JOIN | | 19 | 2375 | 36 (3)| 00:00:01 | |* 3 | HASH JOIN | | 19 | 2071 | 33 (4)| 00:00:01 | |* 4 | CUBE SCAN PARTIAL OUTER| CB$UNITS | 20 | 2000 | 29 (0)| 00:00:01 | | 5 | TABLE ACCESS FULL | TIME_DIM | 96 | 864 | 3 (0)| 00:00:01 | | 6 | TABLE ACCESS FULL | CUSTOMER_DIM | 61 | 976 | 3 (0)| 00:00:01 | -------------------------------------------------------------------------------------------

Demonstration Cube Organized Materialized Views

New In 11g - Partitioning Improvements • Interval Partitions • Automatically creates partitions as they are needed • Saves scripting the creation of new partitions each day/week/month • Useful as long as you’re not using Partition Exchange Loading

New in 11g - Virtual Columns • Define a column based on other columns in the table • Takes up no storage space • Equivalent to calculated measures in Oracle OLAP • Partitioning can be based on these virtual columns

New in 11g - REF Partitioning • Partition a table based on a column in another table • Partition an detail table by the master table primary key • Useful, but actually not relevant for star schemas • The wrong way around, we want to partition the dimensions by the fact keys

New in 11g - More Partitioning Combinations • Two-dimensional partitioning schemes • Partitions and sub-partitions • Partition fact table primarily by date, then create list sub-partitions • Some combinations were available in 9i, 10g • Many more available in 11g

Oracle Warehouse Builder • Oracle’s Full DW Lifecycle tool • Part of the Database Technology Stack • Now at version 11g Release 1 • Relational and OLAP Data Modeling • Data Mapping • Process Flows • Data Profiling, Auditing and Error Handling • Metadata Management • Change Management • ERP Connectivity

New in OWB11g - DML Error Logging • Table operators in mappings can have Error Tables configured • Erroneous rows (from 10.2 DB onwards) moved into error table • Uses DML Error Logging • Allows direct-path (set-based) loading without error roll-back • Traps constraint errors, others still fail • Free option, no dependency on Data Quality Option • Actually faster than (row-based) Data Rules

Demonstration DML Error Logging in OWB11g / 10.2.0.3

New in OWB11g - SCD2 Handling (That Works) • SCD2 handling now works in OWB11gR1 (and OWB 10.2.0.3) • Changes to levels above base level now handled properly • Hierarchy versioning • All attributes in a row with trigger attributes must now also be triggers • No ability to mix SCD1 and SCD2 in a single dimension • Watch out for performance though • Performance bug in 10.2.0.3 and 11gR1, look for upcoming patch

Recently Available - Experts on OTN • XML Input, AW Import, Dimension Derivation Experts etc • Mapping from view/SQL SELECT SELECT   DECODE(GROUPING(channel_desc) , 1, 'Multi-channel sum' , channel_desc) AS Channel , DECODE (GROUPING (country_iso_code) , 1, 'Multi-country sum' , country_iso_code) AS Country , TO_CHAR(SUM(amount_sold) , '9,999,999,999') SALES$ FROM sales , customers , times , channels , countries WHERE sales.time_id=times.time_id AND sales.cust_id=customers.cust_id AND customers.country_id = countries.country_id AND sales.channel_id= channels.channel_id AND channels.channel_desc IN ('Direct Sales', 'Internet') AND times.calendar_month_desc= '2000-09' AND country_iso_code IN ('GB', 'US') GROUP BY CUBE(channel_desc, country_iso_code);

Alternative Data Warehouse Data Sources • Web Services • Advanced Queues and Change Data Capture Streams • Enterprise Service Bus • Business Activity Monitoring

Web Service Data Sources • Many organizations are adopting SOA architectures • Data can now arrive through services and events, as well as from databases • ETL process need to be SOA-aware • Web services and business events are now potentially DW data sources

SOA Architecture Business Activity Monitoring

BPEL Orchestration

Web Services Manager Declarative Rules Engine Enterprise Service Bus

ETL Process Mappings

Metadata

Bulk Data Processing

Advanced Queues and CDC Streams • Trickle feed of incremental data • Arrives via queues, streams • Often need to process in real-time • Some platforms (i.e. Oracle RDBMS) have native support • ETL tools need to be CDC-aware Initial bulk load CDC for synchron ization

Incremental load of new data

CDC for loopback synchron ization

DW Target

DW Sources

CDC

-------------

CDC

Metadata

Oracle Data Integrator • Comprehensive data integration platform • Extraction, transformation and load of data, to and from databases, events and Web services • Batch and real-time loading • Java-based, hot-pluggable, database independent

Oracle Data Integrator Architecture

Extensible Knowledge Modules • Provided “out of the box”, and are user extensible • Written for specific source and target platforms • User created example : Oracle 10g Data Pump

Event and Service-Based Integration • Integrates data from events (files arrive, data arrives on queue) and from services (SOA, Web services) • Perform integration in real-time (Enterprise Information Integration) Oracle Data Integrator • Reflects today’s reality that Event Conductor Service Conductor not all data lives in databases Event-oriented Service-oriented • Integrates with Integration Integration Oracle SOA Suite Metadata Declarative Design

Data-oriented Integration Data Conductor

ODI Web Service Functionality • Data Services • Publish data stores (tables, views etc) as web services • ODIInvokeWebService tool • Request data from a web service as part of a package • ODI Public Web Services • Invoke an ODI scenario through a web service call Oracle Data

Event-oriented

Service-

Metadata Declarative Design

ODI Data Services • Publish a data store as a web service • Insert, Update, Select, Delete • List (multiple requests in one) • Inbuilt interface to Apache Axis2 1.2

ODIInvokeWebService Tool • Request data from a web service • Called as a package step (in OWB terms, a process flow) • Data is fetched into an XML document • XML document then processed and loaded into a table • Most suited to use in initial data preparation, then loaded into staging table for access by ODI interface

ODI Public Web Services • Invoke an scenario (a published ODI package) via a web service call • Call ODI functionality from an application, a BPEL process • Add as a partner link in the JDeveloper BPEL editor • Add an Invoke activity to call the InvokeScenario operation • Add an Assign activity to set operation parameters (repository, scenario etc)

Possible Web Services & ODI Scenarios • To provide bulk processing functionality to a business process • BPEL orchestration of an ETL process • ODI Public Web Services • Service-based access to data mart or data warehouse data • Data Services • Gather data from a web service prior to an ETL operation • Call a human workflow process as part of an ETL control routine • ODIInvokeWebService tool

Oracle Data Integrator and OWB • Complements OWB by adding advanced data integration functions • Works in the staging/integration layer • Pre-build packaged solutions for CDC, Web Services etc • Supports event-based, service-based and real-time integration • Complements the BI Server by handling large data volumes and data marts • Provides heterogeneous functionality - OWB may be more appropriate for Oracle data warehouses • Create offline store (data mart) with history tables (SCD etc) • Perform complex integration tasks • Integrate SOA and event-based data • Adds the ability to target heterogeneous platforms • Supports real-time data integration

“Next-Generation” DW Objectives • Incorporate data feeds from multiple sources • Database tables • Files • Services • Events • Create a consolidated, integrated reporting database • Incorporates batch-loaded and real-time data • Uses database-centric and heterogeneous tools • Tackle integration and standardization issues in a single place • Provide an integrated store of data, indexed and optimized

Demonstration ODI, Service and Event-Based Data Integration

Summary • Oracle Database 11g comes with a number of new data warehousing features • Partitioning improvements • Cube Organized Materialized Views • OWB11g features bug fixes, enhancements and support for 11g • SCD2 handling that works (watch performance though) • Support for DML Error Logging • Data Warehousing in the “Fusion” era is no longer just about extracting from, and loading to, Oracle databases • ODI provides access to heterogeneous data sources • Non-Oracle databases • Web services • Events

Analytics and DW Using Oracle 11g and Oracle BIEE Lesson 1 : ETL and DW development using Oracle 11g and OWB 11g (and ODI)

Analytics and DW Using Oracle 11g and Oracle BIEE Lesson 2 : Creating the Business Logic Layer with OBIEE BI Server

Oracle BI Suite Enterprise Edition Plus • • • • • •

Oracle “Next-Generation” BI tools platform Based on Siebel Analytics Originally developed by nQuire Standards (J2EE) based Heterogeneous connectivity Server and Web-based • Oracle BI Server • Oracle Presentation Server • Oracle BI Answers • Oracle BI Delivers • Oracle BI Administration • Job Manager, Catalog Manager etc

The Future Oracle BI&DW Platform • Leverages the analytic capabilities of the Oracle database • In-database OLAP, data mining, partitioning, analytic queries • Scalable store of detail-level data for analysis • Business logic and metadata layer provided by Oracle BI Server (OBIEE) • Semantic model, calculations, time-series functions, hierarchies • Oracle BI Presentation Server (OBIEE) plus Hyperion EPM tools provide Presentation Layer • Three-tier BI&DW architecture leverages Oracle RDBMS and OBIEE/Hyperion toolset

The Business Logic Layer • Contains an integrated, logical model of the business • Provides an abstraction layer for the business • Can be auto-generated by the ETL tool • OWB 11gR2 • Maintained by the business • Can connect to multiple sources in the data services layer • The Oracle data warehouse • Direct application access • OLAP cubes • Relational databases

OBIEE 10.1.3.2 Architecture Web Server (IIS, Tomcat, Websphere, iPlanet)

Oracle BI Publisher

Oracle BI Presentation Services

Delivery Server

Web Catalog Service

Oracle Interactive Dashboards

Data Logic

Oracle Answers

User Profiling, Security and Session Mngmt

XML Framework

Cache Services (Web) & Connection Mngmt

XML, HTML, XLS, PDF, TXT over HTTP/HTTPS

Javascript for Usability & Interactivity

HTML, SOAP over HTTP/HTTPS

External Applications and Portals

ODBC over TCP/IP (SSL)

Scheduling/Event Services

TCP/IP (SSL)

Oracle BI Server Logical SQL ODBC/JDBC (Logical Business Model)

Load Balancer

Agent Execution Logic

Session Management

Device Adaptive Content

Intelligent Request Generation

Query Govern.

Externalized Authentication LDAP

Logical Request Generation

Security Services

Oracle BI Cluster Controller

Cache Services

Navigator Multi-Pass / Sub-Request Logic Fragmentation Optimization Aggregate Navigator

DB Authentication Custom Authenticator

Optimized Query Rewrites Execution Engine

Data Source Adapters ODBC, CLI, OCI, XML, MDX Analytical and Operational Data Sources

Metadata Interchange

Oracle Delivers Server

System / Perf Monitoring

XSL

TCP/IP (SSL)

SOAP Web Services, XML and URL Interface

Layout Interfaces

Web Browser

SAW Bridge (J2EE/ISAPI)

Oracle BI Administration Metadata Management Services Multi-User Development Services Metadata Documentation Services Server Management Services

Oracle BI Server • Calculation and Data Integration Engine • “Virtual Data Warehouse” • Allows reporting across multiple sources • Data accessed though metadata layer • Connects to RDBMS and OLAP data • Oracle Database (including Oracle OLAP) • IBM DB2 • Microsoft SQL Server • Teradata • Microsoft Analysis Services (OLAP) • SAP BW 'info cubes' • Security, Summary Management • Translates incoming “logical SQL” in to optimized, physical SQL

Oracle BI Presentation Server • • • • •

“BI Application Server” Single application that provides Oracle BI Answers, Dashboards Metadata stored in Web Catalog User Profiling, Security and Session Management SOAP Web Services, XML and URL Interface

Web Server (IIS, Tomcat, Websphere, iPlanet)

Oracle BI Presentation Services SOAP Web Services, XML and URL Interface Oracle Interactive Dashboards

Web Browser

SAW Bridge (J2EE/ISAPI)

Oracle Answers

User Profiling, Security and Session Mngmt Cache Services (Web) & Connection Mngmt

TCP/IP (SSL) Web Catalog Service XML Framework

XML, HTML, XLS, PDF, TXT over HTTP/HTTPS

Javascript for Usability & Interactivity

HTML, SOAP over HTTP/HTTPS

External Applications and Portals

Oracle BI Delivers Server • • • • •

Scheduling and distribution Uses iBots (“Intelligent Bots”) Integration (through Java) with BPEL iBot chaining, calling from BPEL, calling BPEL Permits alerts, through execution of Answers reports • If report returns rows, alert triggered • Alert appears on dashboard, via email, SMS, cell phone • Can trigger narrative and Oracle Delivers Server conditionally-appearing reports Scheduling/Event Services Agent Execution Logic Device Adaptive Content

Oracle BI Publisher Server • New in 10.1.3.2, based on Oracle XML Publisher • Separate application, integrated with Oracle BI Presentation Server • Initially usernames and passwords integrated • Thereafter separately administered • Provides high-fidelity layout, positioning of data, charts etc • Multiple data sources • OBIEE Metadata Layer • Oracle BI Answers reports Oracle BI Publisher • Any source accessible via SQL Delivery Server • XML over HTTP Layout Interfaces XSL

Data Logic

Oracle BI Administration • MS Windows application for managing BI Server • Creates and manages Common Enterprise Information Model • Physical, logical and presentation layers • Manage other aspects of BI Server • Security • Scheduler (Delivers server) Oracle BI Administration • Jobs • Joins • Marketing Metadata Management Services • Sessions Multi-User Development Services • Cache Metadata Documentation Services • Utilities and Tools Server Management Services • Connect to BI Server via ODBC DSN

Core BI Server Functionality • • • •

Data Integration through Business Model View Calculation and analytical engine Support for RDBMS and OLAP data sources Intelligent Request Generation and Optimized Data Access • Security and Data Scoping • ODBC Interface out, native RDBMS and OLAP interfaces in • Clustering, replication, support for Windows and Unix/Linux hosts, 32-bit and 64-bit

Data Access • Reporting data may be sourced from multiple databases, applications • Oracle Database (including Oracle OLAP) • IBM DB2 • Microsoft SQL Server • Teradata • Microsoft Analysis Services (OLAP) • SAP BW 'info cubes' • SAP, Peoplesoft, Siebel, E-Business Suite • Oracle BI Server joins this source data together and presents a logical business model • Facts • Dimensions • Calculations

Customer

Common Enterprise Information Model • Metadata layer for OBIEE tools • Three layers of metadata • Physical layer, connections to sources • Logical layer, denormalized facts and dimensions • Presentation layer, “report-centric” folders • Maintained using BI Administrator • Model is the “center” of the OBIEE universe

Data Modeling using Oracle BI Server • Design goal for the BI Server metadata layer is to create a Star Schema • Fact tables • Dimension tables • Drill paths • Source data can either be pre-integrated (data mart, data warehouse) or integrated using the BI Server (“federated”) • Federated can be faster to deliver (report in place) • Assumes data sources share common key values • Data marts are simpler and faster to query • MDX (OLAP, Multi-dimensional) data is converted into RDBMS row-sets • Logical layer is main integration point • Presentation layer allows “report centric” folders

“Model First” Development • • • •

Business Model is centre of the OBIEE “universe” First element of design that is produced Integrated view of the organization’s data Option 1: Star with model first, then map on physical sources • “purest” approach, uses theoretical model as basis • Tricky to get joins and integration working • Lots of manual logical table and column definition steps • Option 2: Produce model on paper, then construct it using elements of physical model • Quicker and faster, just drag and drop • May end up with redundant columns • Still needs a design phase at the start

Sample Business Model

Data Modeling Workflow Steps 1. Define logical model (on paper or using BI Administrator) 2. Import physical data sources 3. Create physical primary and foreign keys 4. Either drag physical tables to create logical tables, or map

Step 1: Create Physical Layer • Define ODBC connection from client or server to database • ODBC only needed to define initial connection, import metadata • Native connection (OCI etc) takes over from then • Import tables, views, keys etc from data source • Define keys, foreign keys if needed • Update row counts • Define table aliases

Physical Join Types • If source data has foreign key joins, they will be imported • If not, they can be defined in the physical model • Two types of join • Foreign key joins (e.g. CUSTID = CUSTID) • 1:N joins only • Complex joins (TRAN_DATE between STARTDATEID and ENDDATEID) • also supports 1:1, O:1, 1:O etc • Important to create physical joins before importing in to logical layer • Logical joins rely on physical joins to define paths • Makes life a lot simpler • Create joins using Join Manager, or Diagrammer

Step 2: Create Logical Layer • Firstly, create new logical model • Then create logical tables and columns • Either create from scratch, then connect to physical • Or define by dragging and dropping physical tables • Start off with single source for each table, then add connnections • Define dimensions to create drill paths • Add calculations • Use Rename Wizard to clean up table and column names

Calculations • OBIEE has it’s own SQL syntax • Cross-platform • Same SQL across all sources • Calculations, analytics, time-series • Calculation Wizard for time-series calculations • AGO, TO_DATE • Oracle BI Answers features SQL pass-thru • Has to be enabled per user • Allows native functions to be used • Future: direct support for native SQL functions

Time-Series Calculations • OBIEE 10.1.3.2 supports limited time-series functions • AGO – value of a measure x months, quarters, years ago AGO(Videostore."Sales fact view".Sales, Videostore.TimesDim."Month", 1)

• TODATE – value of a measure to the end of a month, quarter, year AGO(Videostore."Sales fact view".Sales, Videostore.TimesDim."Quarter", 1)

• Requires at least one “time dimension” in the metadata layer • Requires at least one or more Chronological keys • Monotonically increasing value (DAY etc)

Create Calculations in the Optimal Location • Whilst the BI Server can create calculations, it may be more appropriate to do this in the underlying database • Oracle Database analytic functions (LAG, LEAD, RANK etc) • Oracle Database OLAP functions (time-series, forecast, allocation) • Use the available architecture efficiently • Perform calculations as close to the source data as possible • Use dedicated OLAP technology if available • Use BI Server to provide features not found in the sources • Just because the BI Server can do something, doesn’t mean it should do something

Logical Join Types • Logical layer supports both foreign key and complex joins • Foreign key joins not recommended • Restrict choices that BI server can make over join paths • Only there for backward compatibility • Complex joins are preferred • Indicates relationship between tables • Allows BI Server to determine best join path • Useful when more than one Logical Table Source

Logical Table Sources • Logical tables can have more than one physical source • Can create >1 Logical Table Source • BI Server creates two physical queries • Useful when mapping data from different levels • and when using “fragmentation” • Each Logical table source can have >1 table mapped • BI server creates one query per LTS, joining as per physical foreign key joins • Used when you need columns from other joined tables in your logical table (join only happens if additional columns requested)

Consider Where to Join Data • Again, consider where to integrate and join data • Whilst the BI Server can join data, it may be faster to load into a data mart • ETL tools can integrate data at time of load • Denormalized data structures can be faster than joining at report time • Indexing, Summary Management, Query Optimization • Creating a data warehouse still is the optimal solution • BIEE data source joining features are most appropriate when bringing new data in that’s not yet in the data warehouse

Step 3: Create Presentation Layer • Business models themselves are not visible to users • Users see presentation catalogs whose columns are defined in business models. • Presentation catalogs appear as “subject areas” in the web UI • Presentation catalogs are “databases” to client tools using ODBC • To create a presentation catalog, drag business model to left panel. Then customize – re-order, re-folder, re-name. • Can create multiple Presentation Catalogs from single Logical Model

Initial Reporting Scenario • Global Company has a data warehouse, containing order and customer data • Requirement is to take the existing warehouse (normalized) and turn it into a star schema (denormalized) • For now, OBIEE will be used to map the data

Demonstration Creating the initial Common Enterprise Information Model

Reporting Across Multiple Data Sources • • • •

OBIEE can combine and join data from multiple data sources Data is retrieved in separate SQL, MDX calls and joined using the BI Server Allows you to perform federated queries Watch out for performance though • End goal should be to eventually move reporting data to a data warehouse

Federated Query Scenario • Additional data exists in an Excel spreadsheet • Contains quotas, plus information on products and dates • Quotas are at the product category and month level • Existing data is at the product and day level • Requirement is to join data, but only show quotas the relevant level

Joining A Second Data Source : Step 1 • Import the new schema into the physical model, ensure FK and PK constraints are set

Joining A Second Data Source : Step 2 • Use the Join Manager, or Physical Model Diagrammer, to create joins between tables in the two schemas

Joining A Second Data Source : Step 3 • Add the new dimension tables as additional logical table sources to the exisiting logical dimension tables • Set the dimension hierarchy level at which they are applicable

Joining A Second Data Source : Step 4 • Add the additional measures to the fact table • Set the dimension levels at which the measures become relevant

Demonstration Adding a Second Data Source

Incorporating Real-Time Data • Initial data set consists of historical Customer Data Warehouse data • Data also exists for real-time Order Bookings system • Two logical models could be created, but ideal is to combine historical and real-time data in one model • Known as “Fragmentation”

Fragmentation • Logical tables have >1 Logical Table Source • Logical Table Source 1 = Historical Data • Logical Table Source 2 = Real-Time Data

Fragmentation Step 1 • Add new logical table source to Items logical table

Fragmentation Step 2 • Ensure mapped tables are added to new LTS

Fragmentation Step 3 • Add Fragmentation Content conditions

Fragmentation Step 4 • Repeat process for Dimensions • Do not add fragmentation condition if same data in both dimension versions • Add “distinct” clause instead

Demonstration Adding real-time data using Fragmentation

Summary • The Business Logic layer is central to Oracle’s BI&DW Architecture • Consists of physical, logical and presentation layers • Foundation is properly-joined physical table sources • Denormalized star schema built using complex logical joins • Calculations (including time-series calculations) not available in the source database can be added to logical model • Where possible though, use in-database analytics and OLAP • Dimensions defined to determine drill paths • Additional data sources can be combined in a single business model • Real-time data can be incorporated using Fragmentation

Analytics and DW Using Oracle 11g and Oracle BIEE Lesson 2 : Creating the Business Logic Layer with OBIEE BI Server

Analytics and DW Using Oracle 11g and Oracle BIEE Lesson 3 : Reporting and Dashboards using OBIEE Presentation Server

Oracle BI Presentation Server • • • • •

“BI Application Server” Single application that provides Oracle BI Answers, Dashboards Metadata stored in Web Catalog User Profiling, Security and Session Management SOAP Web Services, XML and URL Interface

Web Server (IIS, Tomcat, Websphere, iPlanet)

Oracle BI Presentation Services SOAP Web Services, XML and URL Interface Oracle Interactive Dashboards

Web Browser

SAW Bridge (J2EE/ISAPI)

Oracle Answers

User Profiling, Security and Session Mngmt Cache Services (Web) & Connection Mngmt

TCP/IP (SSL) Web Catalog Service XML Framework

XML, HTML, XLS, PDF, TXT over HTTP/HTTPS

Javascript for Usability & Interactivity

HTML, SOAP over HTTP/HTTPS

External Applications and Portals

The Future Oracle BI&DW Platform • Leverages the analytic capabilities of the Oracle database • In-database OLAP, data mining, partitioning, analytic queries • Scalable store of detail-level data for analysis • Business logic and metadata layer provided by Oracle BI Server (OBIEE) • Semantic model, calculations, time-series functions, hierarchies • Oracle BI Presentation Server (OBIEE) plus Hyperion EPM tools provide Presentation Layer • Three-tier BI&DW architecture leverages Oracle RDBMS and OBIEE/Hyperion toolset

The Presentation Layer • Data access layer, query tools, interfaces, MS Office integration • Oracle BI Answers, Dashboard, BI Publisher • BI Office integration • SOAP (SOA) interface • ODBC interface (BO, Cognos etc) • Hyperion tools • May access Essbase directly, by-passing Business Logic Layer

Oracle BI Answers • Query authoring tool for BI EE • Report against business model • Tables, Cross-tabs • Graphs, Gauges • View and column selectors • Narratives • Publish to Oracle BI Interactive Dashboards • Database-neutral • Connects through BI Server

Answers Views • Each report (“request”) contains a single query • Possible to union, minus etc with similar queries • Query has one or more views associated with it • Table, pivot-table • Chart, gauge, funnel chart • Title, narrative • View and column selectors • Views are combined in the compound view • Compound view generally displayed in dashboard

Answers Formatting Options • Columns can be formatted on column, or other column value • Colors, fonts, • Images and icons • Can replace or supplement figures • Images can be added to report titles • Comes with a library of stock images

Oracle BI Interactive Dashboards • Main method of interacting with reports • Fully interactive • Drill on data, graphs • Switch between views • Drill out to detail • Incorporate images, web content • Role-based security • Lightweight, DHTML

Dashboard Interactivity • • • • •

Prompts can be added to dashboard Drop-down lists, multi-select Passes parameters to reports Switch between different views Change columns used in reports

Demonstration Creating Reports and Dashboards using Oracle BI Presentation Server

Oracle BI Publisher • Publishes Answers reports as PDF, XLS, RTF etc • Also direct access to metadata layer • “Pixel-perfect” reports • Used for distributionquality reporting • Integrated into dashboard

Oracle BI Delivers • Scheduling and distribution • Uses iBots (“Intelligent Bots”) • Integration (through Java) with BPEL • iBot chaining, calling from BPEL, calling BPEL • Permits alerts, through execution of Answers reports • If report returns rows, alert triggered • Alert appears on dashboard, via email, SMS, cell phone • Can trigger narrative and conditionally-appearing reports on dashboard

Dashboard Alerts & Guided Analytics • Enables content and layout of dashboard to change based on information analyzed • Sections or links can be defined to only appear when there is “interesting” information in the data

Types of Guided Analytics 1.Adding a link to a dashboard, that only appears if a condition is met 2.Making reports appear on the dashboard, when a condition is met • Requires two things • A report to run, that determines whether the event has happened • A report, or link, that you want to appear

Step 1 : Create a “Trigger” Report • Create a report that checks for your condition • Has a branch fell below its sales threshold? • Has a product’s year on year sales fallen? • Use filters to check for the condition • If the report returns rows, the trigger will be met • Save to the Web Catalog

Step 2 : Add Guided Analytics Link to Dashboard • Drag and drop from Dashboard Objects palette • Add to dashboard, just under the report to which it is related

Step 3 : Edit the Guided Analytics Link Properties • • • •

Select the report that is run to determine whether the link is displayed Select whether any rows, or no rows, is the trigger Enter link text Select report, dashboard or URL to link to

Step 4 : View Link in Dashboard (if Triggered) • Link only appears if condition is met • Guides user to the next action they should take

Making Sections Display Conditionally • Individual sections can be displayed conditionally • Select “Guided Navigation” from the section properties • Pick the report to run

Demonstration Guided Analytics and Conditional Display

Summary • Oracle BI Answers and Dashboards give you many ways to display and visualize data • Many formatting and layout options are now possible • Best practices exist for laying out dashboards and presenting data • Reports and dashboards can be made interactive and navigable

Analytics and DW Using Oracle 11g and Oracle BIEE Lesson 3 : Reporting and Dashboards using OBIEE Presentation Server

Analytics and DW Using Oracle 11g and Oracle BIEE Lesson 4 : Analytics and OLAP Analysis using Oracle OLAP and Oracle Essbase

The Future Oracle BI&DW Platform • Leverages the analytic capabilities of the Oracle database • In-database OLAP, data mining, partitioning, analytic queries • Scalable store of detail-level data for analysis • Business logic and metadata layer provided by Oracle BI Server (OBIEE) • Semantic model, calculations, time-series functions, hierarchies • Oracle BI Presentation Server (OBIEE) plus Hyperion EPM tools provide Presentation Layer • Three-tier BI&DW architecture leverages Oracle RDBMS and OBIEE/Hyperion toolset

The Data Services Layer • Based around a core Oracle Data Warehouse • Or several federated data warehouses • Data Warehouse has three tiers • Staging • Atomic/Normalized • Performance/Dimensional • ETL process for loading DW • OWB (default ETL tool) • ODI (heterogenous sources /targets) • Data Services Layer also provides real-time and SOA feeds to OBIEE • For applications not yet in DW • For real-time reports • For transitory requirements

The Presentation Layer • Data access layer, query tools, interfaces, MS Office integration • Oracle BI Answers, Dashboard, BI Publisher • BI Office integration • SOAP (SOA) interface • ODBC interface (BO, Cognos etc) • Hyperion tools • May access Essbase directly, by-passing Business Logic Layer

Hyperion Financial Management Tools • Financial Management tools • Hyperion Planning, Hyperion Performance Scorecard • Hyperion Capital Expenses Planning, Hyperion Workforce Planning • Essbase OLAP Server • Hyperion System 9 infrastructure • ex-Brio reporting and dashboard tools

Oracle OLAP Option • Adds multi-dimensional data types and a multi-dimensional engine • Stores data in indexed arrays • Adds an OLAP API and an SQL access layer • (Can be) Faster than star schemas for loading and querying • Supports time-series and dimensional queries • Based on Express Server technology • Provides consistently-fast access to summarized data

New Features in Oracle OLAP 11g • • • •

• • • •

The latest release of the Oracle Database OLAP Option Continues the objective of embedding OLAP in the database engine SQL SELECT access Cubes stored in Analytic Workspaces • Runs faster in 11g, uses SecureFiles instead of LOBs Metadata in the Oracle Data Dictionary Automatic creation of views over AW objects • Optimizer now “AW view aware” CUBE_TABLE for easier direct SQL access CUBE access now detailed in execution plans

Essbase Overview • • • • • •

Multi-dimensional database engine Calculations, aggregations, forecasting, allocation Foundation for many analytical applications Originally developed by Arbor, merged with Hyperion Software 1998 Part of Oracle Fusion Middleware End-User focused, very popular with business users

Essbase Architecture • • • •

ODBC connectivity to all major RDBMSs, plus SAP B/W Multi-dimensional OLAP engine Java and XML/A mid-tier Essbase query tools or OBIEE+

Essbase Analytic Capabilities • Drill, pivot, select, compare, rank, sort, filter, group, stack, calculate, annotate, write-back.... • Reporting tools enabled by database “outline” • Maps to the business

Essbase Calculation Ability • Wide range of functionality • Simple aggregations to complex cross-dimensional allocations • Results stored or calculated upon retrieval • Relationship-aware • Application-wide or subset-specific • Conditional and procedural logic

Essbase Storage Options • Traditionally, Essbase uses the Block Storage Option • Useful for dense, sparse/dense cubes • Full range of calculations • Slower to aggregate • Recently introduced, the Aggregate Storage Option • Useful for sparse data sets • Generally read-only • Fast to aggregate • Similar storage model to MS AS, Oracle OLAP compressed composites

Query and Administration Languages • MaxL DDL for systems administration (SQL-like object definition language) create or replace outline on database global.global

• MDX for accessing data and metadata With set [CUSTOMER_DIM3] as '[CUSTOMER_DIM].Levels(2).members' set [PRODUCT_DIM3] as '[PRODUCT_DIM].Levels(2).members' set [TIME_DIM3] as '[TIME_DIM].Levels(2).members' set [Q] as 'crossjoin ({[CUSTOMER_DIM3]},crossjoin ({[PRODUCT_DIM3]}, {[TIME_DIM3]}))' select { [Accounts].[SALES] } on columns, NON EMPTY {[Q]} properties ANCESTOR_NAMES on rows from [global.Global]

Essbase Development & Support Tools • • • •

Analytic Administration Services Analytic Integration Services Analytic Provider Services Hyperion Shared Services

Administration Services • • • •

Three-tier application for administering Essbase Console for editing, maintaining Essbase databases Java, launched through Web browser Allows you to maintain the OLAP database “outline” • Definition of dimensions, hierarchies, measures

Integration Services • • • • •

IDE for defining and loading an Essbase database Derive an Essbase cube from a relational star schema Define dimensions, hierarchies, measures Populate cube Set up hybrid (HOLAP) storage • Generate calculation scripts

OLAP Models and Metamodels • Metadata used by Integration Services • OLAP Models define the relational star schema used to source data for an Essbase database • OLAP Metaoutline defines the dimensional model used in the Essbase database • Administration Services can derive the metaoutline from the OLAP Model • Outline can then be maintained in the Administration Services Console

Provider Services • Mid-tier that provides XML/A connectivity, Java API • Fail-over, clustering • Requirement for OBIEE and Smart View access

Smart View for Office • Microsoft Office Integration (Word, Excel, Powerpoint) • General analytic capability • Query designer, linked views • Inclusion of Hyperion Visual Explorer • Oracle Hyperion Planning Integration • UI for Planning off-line • BI+ repository • Run-time prompts • Multi-page support • Editing POV of imported objects

Essbase Within Oracle BI Framework • Adds multi-dimensional analytic server to OBIEE BI Foundation • Calculations, fast analysis, predictive modeling

Creating an Essbase Cube from Oracle Data 1. Create new OLAP Model using Integration Services 2. Connect to the source database 3. Create fact table and associated Account dimension 4. Create other dimensions, including a time dimension if required 5. Review dimensional model 6. Create hierarchies for the dimensions 7. Create the OLAP Metaoutline, define the measures 8. Create the database using Administration Services, choose storage option 9. Use Integration Services to load members and data 10.Preview OLAP cube in Administration Services

Creating in Essbase Cube : Step 1 • Start Integration Services console, select New OLAP Model • Select ODBC Data Source that provides a connection to the source schema

Creating in Essbase Cube : Step 1 • Start Integration Services console, select New OLAP Model • Select ODBC Data Source that provides a connection to the source schema

Creating an Essbase Cube : Step 2 • Start up the Integration Services Console, show Intelligent Help (optional) • Select “Create Fact Table” from menu, select table that contains measures • Check box to create Account dimension • Uncheck box to create Time dimension

Creating an Essbase Cube : Step 2 • Start up the Integration Services Console, show Intelligent Help (optional) • Select “Create Fact Table” from menu, select table that contains measures • Check box to create Account dimension • Uncheck box to create Time dimension

Creating an Essbase Cube : Step 3 • Select “Create Dimensions” from the application menu

Creating an Essbase Cube : Step 3 • Select “Create Dimensions” from the application menu

Creating an Essbase Cube : Step 4 • View initial model in Integration Services

Creating an Essbase Cube : Step 5 • Create Hierarchies for all of the dimensions • Create TIME_DIM as a Time Dimension • Validate and then Save model

Creating an Essbase Cube : Step 5 • Create Hierarchies for all of the dimensions • Create TIME_DIM as a Time Dimension • Validate and then Save model

Creating an Essbase Cube : Step 6 • Create new MetaOutline • Base MetaOutline on Model that you just created

Creating an Essbase Cube : Step 6 • Select “Autodetect” and create the dimensions from existing hierarchies

Creating an Essbase Cube : Step 6 • Select “Autodetect” and create the dimensions from existing hierarchies

Creating an Essbase Cube : Step 7 • Drag and drop Accounts dimension over to dimension list • Set dimensions to dense or sparse, as needed

Creating an Essbase Cube : Step 7 • Drag and drop Accounts dimension over to dimension list • Set dimensions to dense or sparse, as needed

Creating an Essbase Cube : Step 8 • Use Administration Services to create a new application • Select Block Storage or Aggregate Storage

Creating an Essbase Cube : Step 8 • Use Administration Services to create a new application • Select Block Storage or Aggregate Storage

Creating an Essbase Cube : Step 9 • Use Integration Services to carry out a member and data load

Creating an Essbase Cube : Step 9 • Use Integration Services to carry out a member and data load

Creating an Essbase Cube : Step 9 • Use Integration Services to carry out a member and data load

Creating an Essbase Cube : Step 10 • Use Administration Services to preview the cube • Note : if no values in cube, check that “use default calc script” was selected when performing the member and data load

Demonstration Creating an Essbase Cube using Oracle Data

Querying Essbase Cubes using Oracle BI EE • Essbase is a native data source for Oracle BI Enterprise Edition • Uses XMLA compatibility feature of OBIEE • Requires OBIEE 10.1.3.3.2, or 10.1.3.3.1 with registry edit • HKEY_CURRENT_USER\Software\Oracle Corporation\Oracle BI\Oracle BI Administration Tool\Hidden Features • Create new DWORD value called “Essbase”, set to “1” • Some restrictions in 10.1.3.3.1 release, treat as “for evaluation only”

OBIEE 10.1.3.3.1 Guidelines & Limitations • Measure hierarchies not supported, all measures shown on same level • Import process does not know that dimension member names may be unique, you will need to inform OBIEE of this in the logical dimension member properties • EVALUATE function can now work against MDX data sources • EVALUATE_AGGREGATE function will need to be used for Essbase to natively handle time-series functions (not function shipped by default) • Essbase UDAs suppressed by default, can be enabled using EVALUATE • Requires Essbase 9.3.1 stack • Essbase Server 9.3.1 • Essbase Provider Services 9.3.1

Registering an Essbase Cube in OBIEE Step 1 • Use Oracle BI Administrator to Import from XMLA • Requires Essbase Provider Services to be installed and running

Registering an Essbase Cube in OBIEE Step 2 • • • • •

Enter the connection details for the Essbase server URL = :13080/aps/XMLA Press “Update” to pre-populate the Data Source field Username = Essbase login ID Password = Essbase password • Not Shared Services login

Registering an Essbase Cube in OBIEE Step 3 • Select the cube that you want to import • Essbase cubes are hypercubes, contain measures of the same dimensionality • Cube brings across measures and dimensions

Registering an Essbase Cube in OBIEE Step 3 • Select the cube that you want to import • Essbase cubes are hypercubes, contain measures of the same dimensionality • Cube brings across measures and dimensions

Registering an Essbase Cube in OBIEE Step 4 • Cube appears as new database data source in OBIEE Physical layer • Note different icons for dimensions and measures • All “joins” are created automatically

Registering an Essbase Cube in OBIEE Step 5 • Drag physical cube into logical layer • All dimensions, joins, “tables” are automatically created • Note that level names do not get imported

Registering an Essbase Cube in OBIEE Step 6 • Rename levels to correct names

Registering an Essbase Cube in OBIEE Step 7 • Drag logical model into Presentation Layer • Delete all the Level 0 levels • Not required, if selected will cause request to error

Registering an Essbase Cube in OBIEE Step 8 • Presentation layer should now only contain the tidied-up column names, as if it was a relational schema • Now ready for reporting against

Registering an Essbase Cube in OBIEE Step 9 • Create request as normal, results are returned super-fast

Demonstration Importing and Reporting against Essbase using Oracle BI Enterprise Edition

Oracle Database 11g OLAP Option Overview • The latest release of the Oracle Database OLAP Option • In-database OLAP, tightly integrated with the Oracle database • Data stored in Analytic Workspaces • Runs faster in 11g, uses SecureFiles instead of LOBs • Queried using SQL, OLAP DML, OLAP API • Metadata in the Oracle Data Dictionary • Fully compatible with RAC / Grid Computing

New Features in Oracle OLAP 11g • All AWs run faster through optimized SQL access layer, use of SecureFiles • New 11g Cube Mode • CUBE_ORGANIZED materialized views • CUBE_TABLE SQL access method • AWs refreshed using DBMS_MVIEW.REFRESH • AWs qualify for query rewrite • New query expression syntax • Enhancements to AWM • Automatic creation of SQL views • Data security based on Hierarchies

Cube Organized Materialized Views • Transparently enhance the query performance of BI applications • Data is managed in an Oracle cube • Fast refresh • Fast query • Manage a single cube instead of 100’s of individual MVs • Applications query the base relational tables • Oracle automatically rewrites the queries to use the OLAP cube • Access to the summary data in the OLAP cube is fully transparent

Automatic Generation of SQL Views • • • • • • •

SQL views are automatically created when AW objects are defined Views over dimensions Views over hierarchies Views over cube Denormalized view over cube + dims Fully solved Maintained automatically

Cost-Based Aggregation • • • • •

Alternative to level-based pre-aggregation 0 = no pre-summarization 1 = build composite, no pre-summarization Typical figure is 20, 50 is quite high Higher figures tell AWM to precompute more of the aggregate space

Positioned As the Oracle Data Warehouse Enhancer • • • •

Transparent integration with SQL Swap-in replacement for Materialized Views Refreshed and loaded in the same way as Materialized Views Either accessed directly using SQL views, or indirectly through query rewrite

Creating an 11g Analytic Workspace Step 1 • Start up Analytic Workspace Manager 11g • Select “OLAP 11g” Cube Type

Creating an 11g Analytic Workspace Step 2 • Create dimensions as normal • Create dimension • Create levels • Create hierarchies • Create mappings

Creating an 11g Analytic Workspace Step 3 • Switch to the Materialized View tab on the dimension • Enable MV refresh • Select refresh mode • Fast does not work on initial release • Check compatibility checklist • Check MV details

Creating an 11g Analytic Workspace Step 4 • Build the dimension using the Maintain Dimension menu item • Check progress of build, and new CommandOutput log file • View dimension in Dimension Viewer

Creating an 11g Analytic Workspace Step 4 • Build the dimension using the Maintain Dimension menu item • Check progress of build, and new CommandOutput log file • View dimension in Dimension Viewer

Creating an 11g Analytic Workspace Step 5 • Analyze the dimension so that it is eligible for Query Rewrite • Updates stats used by the Cost-Based Optimizer

Creating an 11g Analytic Workspace Step 6 • Define the cube as normal • New option to aggregate using Cost-based setting • % is of the total cube, starting from the bottom hierarchy levels • Adds % on to total size of cube (if using compression)

Creating an 11g Analytic Workspace Step 7 • Use the cube Materialized View tab to set MV parameters • Fast refresh does not work (yet) • Enable Query Rewrite • Check that all conditions are OK • Check that the MV Details panel shows the MV as being created

Creating an 11g Analytic Workspace Step 8 • Use Partitioning Advisor to advise on cube partitioning • New in 11g - “Capstone Partition” can be pre-summarized

Creating an 11g Analytic Workspace Step 8 • Use Partitioning Advisor to advise on cube partitioning • New in 11g - “Capstone Partition” can be pre-summarized

Creating an 11g Analytic Workspace Step 8 • Use Partitioning Advisor to advise on cube partitioning • New in 11g - “Capstone Partition” can be pre-summarized

Creating an 11g Analytic Workspace Step 9 • Storage Advisor recommends storage options to use • Compressed vs. regular, dense dimensions vs. sparse • Analyzes source data to determine sparsity

Creating an 11g Analytic Workspace Step 9 • Storage Advisor recommends storage options to use • Compressed vs. regular, dense dimensions vs. sparse • Analyzes source data to determine sparsity

Creating an 11g Analytic Workspace Step 10 • Maintain Cube as normal • Check new Build Log and CommandOutput log file

Creating an 11g Analytic Workspace Step 11 • View cube results in AWM Cube Viewer • Check that everything aggregates properly, figures look correct

Creating an 11g Analytic Workspace Step 12 • Create calculations using new Create Calculated Measure dialog • New calculation types • New expression language • SQL-style syntax • Easier for non-OLAP developers to use • (Not OLAP DML or MDX)

Creating an 11g Analytic Workspace Step 12 • Create calculations using new Create Calculated Measure dialog • New calculation types • New expression language • SQL-style syntax • Easier for non-OLAP developers to use • (Not OLAP DML or MDX)

Creating an 11g Analytic Workspace Step 13 • (Optional) Create Cube Script to load cube • Steps for clearing dims and measures, loading, analyzing, OLAP DML etc • Note - not compatible with MV refresh

Creating an 11g Analytic Workspace Step 14 • View SQL Views over AW objects in SQL Developer • View Materialized Views created by AWM

Creating an 11g Analytic Workspace Step 15 • Use the MV Advisor to prepare source data for rewrite • Create MV logs • Create source tab constraints • Create relational dimensions

Creating an 11g Analytic Workspace Step 16 • Write SQL against the base-level tables • Rewritten against AW (initial release joins back to dims, in future this will go) explain plan for select sum( u.units), c.market_segment_dsc, t.year_dsc, p.family_id from units_fact u, customer_dim c, time_dim t, product_dim p where u.ship_to_id = c.ship_to_id and u.month_id = t.month_id and u.item_id = p.item_id group by c.market_segment_dsc, t.year_dsc, p.family_id; PLAN_TABLE_OUTPUT ----------------------------------------Plan hash value: 3517365018 ------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 19 | 2375 | 37 (6)| 00:00:01 | | 1 | HASH GROUP BY | | 19 | 2375 | 37 (6)| 00:00:01 | |* 2 | HASH JOIN | | 19 | 2375 | 36 (3)| 00:00:01 | |* 3 | HASH JOIN | | 19 | 2071 | 33 (4)| 00:00:01 | |* 4 | CUBE SCAN PARTIAL OUTER| CB$UNITS | 20 | 2000 | 29 (0)| 00:00:01 | | 5 | TABLE ACCESS FULL | TIME_DIM | 96 | 864 | 3 (0)| 00:00:01 | | 6 | TABLE ACCESS FULL | CUSTOMER_DIM | 61 | 976 | 3 (0)| 00:00:01 | -------------------------------------------------------------------------------------------

Demonstration Creating an Oracle Database 11g OLAP Option Cube, Enabling for Query Rewrite and SQL Access

Summary • • • • • • • • • • •

Essbase is the analytical OLAP engine provided by Hyperion Standalone analytical applications, engine behind Hyperion Planning etc Query using MDX, Smart View, XML/A applications Powerful query features, calculations, high uptake by business users Integrates with OBIEE via XML/A interface Oracle OLAP is the in-database analytical solution for the Oracle RDBMS Tight SQL integration Accessed through SQL views or through Cube Organized Materialized Views Both are highly capable OLAP servers Essbase aimed at standalone analytical apps, integration with OBIEE Oracle OLAP aimed at database-centric OLAP, also integrates well with OBIEE

Analytics and DW Using Oracle 11g and Oracle BIEE Lesson 4 : Analytics and OLAP Analysis using Oracle OLAP and Oracle Essbase

Analytics and DW Using Oracle 11g and Oracle BIEE Closing Slides

The Oracle 11g Data Warehousing Platform • The latest release of the Oracle database • Enterprise Edition contains a large number of data warehousing features • Partitioning • Bitmap indexes • Star transformations • Compression • Options for in-database OLAP and Data Mining • Oracle Warehouse Builder for database-centric ETL and Data Quality

The Oracle 10g Business Intelligence Platform • Based around Oracle Business Intelligence Enterprise Edition • Dashboards, Answers, Delivers, Oracle BI Server • Complete set of web-based query, reporting and analysis tools • Based on Siebel Analytics • Heterogeneous database access • Logical Business Model • Data integration capabilities • Add-ons for data integration, real-time decisions

Hyperion Financial Management Tools • Financial Management tools • Hyperion Planning, Hyperion Performance Scorecard • Hyperion Capital Expenses Planning, Hyperion Workforce Planning • Essbase OLAP Server • Hyperion System 9 infrastructure • ex-Brio reporting and dashboard tools

The Future Oracle BI&DW Platform • Leverages the analytic capabilities of the Oracle database • In-database OLAP, data mining, partitioning, analytic queries • Scalable store of detail-level data for analysis • Business logic and metadata layer provided by Oracle BI Server (OBIEE) • Semantic model, calculations, time-series functions, hierarchies • Oracle BI Presentation Server (OBIEE) plus Hyperion EPM tools provide Presentation Layer • Three-tier BI&DW architecture leverages Oracle RDBMS and OBIEE/Hyperion toolset

Analytics and DW Using Oracle 11g and OBIEE 10g • An in-depth look at the technologies behind Oracle’s future BI&DW platform • What’s new in Oracle Database 11g for data warehousing • How the Oracle BI Server provides the business logic for your BI&DW platform • Creating the Enterprise Semantic Model, how the technology works and what are its strengths and weaknesses • Where to place calculations, summaries and hierarchies in your architecture • How Oracle BI Presentation Server provides dashboards and reports (and what it’s missing in the current release) • Where the Hyperion tools fit in • What’s new in Oracle OLAP 11g, and an introduction to Oracle Essbase

And Finally... • • • • • •

Thank you all for attending I hope today has been of interest to you Please remember to complete the seminar evaluations Leave your contact details with me to forward on the slides Have a safe journey home Goodbye!