Oracle Real Applications Testing Overview by Kevin Patterson (Oracle)

Real Application Testing Kevin Patterson, Senior Sales Consultant Systems and Applications Management (SAM)

The following is intended to outline our general product direction. It is intended for information information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, release, and timing t iming of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Oracle Enterprise Manager  Top-Down Application Management Application Operations Management

Application Quality Management

Real User  Experience Monitoring Service Level Management

Load Testing and Accelerators for  Packaged Apps & Web Services Application Testing Suite

SOA Application Management

Functional Testing and Accelerators for  Packaged Apps & Web Services

Diagnostic and Tuning

Test Management

Provisioning, Configuration Management

Real Application Testing

Packaged Application Management

Change Tracking and Synchronization

Data Masking

Real Application Testing Load and SQL impact testing solution providing highest quality testing for the database tier  • Value • Rapid technology adoption Deploy

• Higher testing quality • Business Benefit

Test

• Lower cost

Change

• Lower risk • Features • Database Replay

Remediate

• SQL Performance Analyzer (SPA)

Business Agility through Superior  Testing

Testing Today Productio Prod uction n – 1,000s of of Real Online Online Users Users

Application Servers

Database Servers

PRODUCTION

Testing Today Test – 1-2 testers testers trying trying to be 1,000s 1,000s of users users

Application Servers

Database

Database

Servers

Servers

PRODUCTION

TEST

Database Replay Real workload for 1,000s of online users captured

Application Servers

Database Servers

Capture Workload

PRODUCTION

Database Replay Real workload for 1,000s of online users replayed

Application Servers

Database

Database

Servers

Servers

Capture

Replay Workload

PRODUCTION

TEST

Database Replay Test your system changes at production levels

Application

Application

Servers

Servers

Database

Database

Servers

Servers

9.2.0.8, 10.2.0.3+, 11g

Capture

PRODUCTION

Replay

11g ONLY

Workload

TEST

Real Application Testing with Database Replay • •

Replay production workload in test environment Identify, analyze and fix potential instabilities before making changes to production

•

Capt Ca ptu ure Wo Worrkl kloa oad d in Pro rod duc ucti tion on • •

•

Replay Workload in Test • • •

•

Capture full production workload with real load, timing & concurrency characteristics Move the captured workload to test system Make the desired changes in test system Replay workload with full production characteristics Honor commit ordering

Analyze & Report • • •

Errors Data divergence Performance divergence

Analysis & Reporting

Why DB Replay?

150 Days

From:

To:

Artificial workloads

Production workloads

Partial workflows

Complete workflows

Months of development

Days of development

Manual intensive

Automated

High risk

Low risk

10 Days

Supported Changes with Real Application Testing

Changes Unsupported

Application Servers

Changes Supported •Database Upgrades, Patches •Schema, Parameters •RAC nodes, Interconnect •OS Platforms, OS Upgrades •CPU, Memory •Storage •Etc.

Database Servers

Recording of  External Client Requests

Database Replay Workflow Production

Test

(9.2.0.8, 10gR2, 11g) Clients

…

(11g)

Replay Driver 

Mid-Tier 

…

Storage

Storage

Capture

Process

Replay

Analysis & Reporting



Application Quality Management Oracle Real Application Testing Database Replay Demonstration

Database Replay Restrictions • Database Replay does not support the following features in the current release •

Direct Path Load (SQL Loader)

•

Shared server (Oracle MTS)

•

Oracle Streams & Advanced Replication Streams

•

Non-PL/SQL based Advanced Queuing (AQ)

•

Flashback queries

•

OCI-based Object Navigation

•

Non SQL-Based Object Access

•

Distributed transactions, remote describe/commit operations (will be replayed as local transactions)

Real Application Testing with SQL Performance Analyzer (SPA) •

Test impact of change on SQL query performance

•

Capture SQL workload in production pro duction including statistics & bind variables

•

Re-execute SQL queries in test environment

•

Tune regressed SQL and seed SQL plans for production Client

Client

…

Client

Production Test Re-execute SQL Queries

Middl Mid dle e Ti Tier  er 

Capture SQL Oracle DB

9i, 10g, 11g Storage

…

…

Use SQL Tuning   Advisor to tune regression

10g, 11g

Why SQL Performance Analyzer (SPA)? • Busin Busines esses ses wan wantt syste systems ms that that are are perfor performa mant nt and and meet eet SLA’s • SQL performance regressions are #1 cause of poor  system performance • Solution for proactively detecting all  all SQL SQL regressions resulting from changes not available • DBA’s DBA’s use ineffective ineffective and and time-consu time-consuming ming manual manual scripts to identify problems

SPA identifies all changes in SQL performance before impacting users

SQL Performance Analyzer Workflow Production (9.2, 10.1, 10.2, 11g)

…

Clients

Test (10.2 or 11g) Mid-Tier 

Storage

Storage Capture SQL

Transport SQL

Execute SQL Pre-change

Execute SQL Post-change

Compare Perf 

SQL Performance Analyzer  Restrictions • SQL Performance Analyzer does not support the following features in the current release •

Shared server (Oracle MTS)

•

9i Parallel Query (SQL Trace Capture)



Application Quality Management Oracle Real Application Testing SQL Performance Analyzer  Demonstration

Real Application Testing: Tools of the Trade SQL Performance Analyzer 

Database Replay 

What is it?

• Predicts SQL performance deviations before end-users can be impacted

• Replays real database workload on test system

What Purpose?

• Assess impact of change on SQL response time

• Assess impact of change on workload throughput

How it works?

• Executes each SQL, stored in SQL Tuning Set, in isolation using production context and then compares before and after execution plans and run-time statistics

• Captures workloads and replays it with production characteristics including concurrency, synchronization & dependencies

When to use?

• Unit testing of SQL with the goal to identify the set of SQL statements with improved/regressed performance

• Comprehensive testing of all sub-systems of the database server using real production workload

SQL Performance Analyzer and Database Replay are complementary 

Why Consider Oracle 11g for Upgrades? • Oracle Database 9i Release 2 Premium Support ended Jul 30, 2007 • 1 yr free Extended Support offer ends Jul 30, 2008 • Help enable upgrades from 9.2.0.8 Today  August 2012 

July 2010 

R2

January 2009

Premier Support  R2

 August 2015 

July 2013

January 2012 

Sustaining Support 

Extended Support 

July 2007 

July 2010 

Jul 2008

t  2    0    0   2  

2    0    0    3  

2    0    0   4  

2    0    0    5  

2    0    0    6  

2    0    0   7  

2    0    0    8  

2    0    0    9  

2    0   1    0  

2    0   1   1  

2    0   1   2  

2    0   1    3  

2    0   1   4  

2    0   1    5  

2    0   1    6  

David Mitchell Senior Vice President, OVUM

“Oracle Real Application Testing reduces the time required to test changes by as much as 80%, lower  testing costs by as much as 70%, mitigate risks by  reducing the number of unexpected outages, and  improve the quality of service for their IT operations.” operations .”

Further Information • Oracle Real Application Testing Homepage on OTN http://www.oracle.com/technology/produ http://www.oracle.co m/technology/products/manageabili cts/manageability/database/index.html ty/database/index.html • Data Sheets • Technical and Customer Whitepapers • Analyst Reports • Tech Notes on support for Earlier Releases • Examples and Scripts for Command Line Interface Usage • Demo Viewlets

• Oracle Real Application Testing User Guide http://download.oracle.com/docs/cd/B2 http://download.oracl e.com/docs/cd/B28359_01/server.111/e1 8359_01/server.111/e12253.pdf  2253.pdf 

Further Information

(Continued)

• 11G Top Features for DBAs and Developers • http://www.oracle.com/technology/pub http://www.oracle.com/technology/pub/articles/oracle-data /articles/oracle-database-11gbase-11gtop-features/index.html

• 11G Oracle By Example Series: • http://www.oracle.com/technology/obe http://www.oracle.com/technology/obe/11gr1_db/index.htm /11gr1_db/index.htm • Database Replay • http://www.oracle.co http://www.oracle.com/technology/obe m/technology/obe/11gr1_db/manage/dbrepla /11gr1_db/manage/dbrepla y/dbreplay.htm • SQL Performance Analyzer  • http://www.oracle.co http://www.oracle.com/technology/obe m/technology/obe/11gr1_db/manage/spa/sp /11gr1_db/manage/spa/spa_g a_g uide.htm

• Metalink Notes for Patches and Steps to Implement: • 560977.1: Real Application Testing Now Available for Earlier  Releases • 562899.1: Oracle Real Application Testing: Test the SQL Performance Impact of an Oracle 9i to Oracle Database 10g Release 2 upgrade with SQL Performance Analyzer 

Database Replay for Earlier Releases • Help customers smoothly transition to Oracle Database 11g • “Workload Capture” portion of Database Replay is now available with Oracle 9i and Oracle Database 10g • “Workload Replay” only possible on Oracle Database 11g and above • For more details: • Metalink Metalink Note: 560977.1 560977.1 – “Real Application Testing for Earlier  Releases”  Upgrade From

Upgrade To

10gR2

11g

Release / Patches needed • 10.2.0.4 Patchset • 10.2.0.x + one-off patch

9iR2

11g

• 9.2.0.8 + one-off patch

Comments • Need to apply one-off patch for 10.2.0.2 and 10.2.0.3 • Limited performance data available for  captured captured workload – can use STATSPACK STATSPACK

SQL Performance Analyzer  for Earlier Releases • Help customers smoothly transition to Oracle Database 10g and 11g • SQL Test Execution feature of SPA backported backported to 10gR2 • For more details: • Metalink Metalink Note: Note: 560977.1 560977.1 – “ Real Application Testing for Earlier Releases” Releases ” • Metalink Metalink Note: Note: 562899.1 562899.1 – “ Testing Performance Impact of an Oracle 9i to Oracle Database 10g Release 2 Upgrade with SQL Performance Analyzer ” Upgrade Upgrade From To 10gR2

10gR2 or   11g

Release / Patches needed • 11.1.0.6 + one-off patch or 11.1.0.7

Comments • Uses STS to capture SQL stats

• 10gR2 + one-off patch 9i 10gR1

10gR2 or  11g

• 11.1.0.6 + one-off patch or 11.1.0.7 • 10gR2 + one-off patch

• Uses SQL*Trace to capture SQL stats