Architecture Oracle RDBMS
1
Introduction Dan Brint - DBA with ITEC
[email protected] [email protected]
Architect Oracle database for high performance and availability SUNY Confidential
2
Todays Agenda Traditional Oracle architecture Performance / Availability technologies Oracle in a partitioned world Real Application Clusters (RAC) Management 10g (if time permits)
3
Topic 1 Traditional Oracle architecture Performance / Availability technologies Oracle in a partitioned world Real Application Clusters (RAC) Management 10g (if time permits)
4
Introduction What follows is a simplification. Oracle education spends 8 hours covering this topic alone in DBA 1 This basic design hasn’t changed in 5 major releases of Oracle dating back to Oracle 6 released in 1988 Major enhancements with every release but the core architecture has not changed 5
Overview Remote and Local users accessing an Oracle Server
R e mote U se rs N od e L oc a l U se rs
Server components: Software
S Q L ne t L iste ne r
S e rve r P roc e ss
Insta nc e PG A
Server and Background processes
d b wr
a rc h
lgrw
SG A smon
Memory structures
R edo B uffe r
Database Files Data and Index files
S oftwa re
Control files
c kpt pmon
D a ta / Ind e xe s C ontrol F ile s
Redolog files
R edo L ogs
Archivelog files
Arc hive L ogs
Temp and Undo files
U nd o / T e mp
6
Components Remote Users
Lets examine each component:
Node Local Users
Server Process
Instance PGA
1. Users
SQLnet Listener
dbwr
arch
lgrw
SGA
Local Users
smon
Remote Users
Redo Buffer
Data / Indexes
Software
2. Server Processes 3. Software
ckpt pmon
Control Files
SQLnet Listener
Server Process
Redo Logs Archive Logs Undo / Temp
Software
4. Background Processes 5. Memory Structures 6. Database Files
Data / Indexes
7
dbwr
SGA
smon
ckpt pmon
Redo Buffer
PGA
Control Files
lgrw
arch
Redo Logs
Archive Logs
Undo / Temp
Users Remote Users
Remote
Local Users
SQLNET over TCP/IP (generally) Web server Client Desktops Other remote databases Local - users on the same server as the database. Bequeath (process to process) 8
Server Processes Server process
SQLnet Listener
Server Process
Created on behalf of the user at connect time Responsible for issuing queries and negotiating updates with the instance Server process never updates data directly - this is done via a background process
Dedicated (1:1) or pooled (1:n) SQLnet Listener Listens for sqlnet connections and initiates server processes 9
Software Software
Oracle RDBMS (code tree) Core RDBMS + options + SQLnet + Oracle Enterprise Manager Certified on the hardware & OS Different versions can coexist via ‘oracle homes’ (directories) Oracle Universal Installer Oracle Inventory Should be externally mirrored 10
Background Processes 1 of 2 dbwr
lgrw
arch
DBWR - database writer.
Responsible for updating database on behalf of users
LGWR - log writer.
Writes database changes to separate log files for recovery purposes
ARCH - archiver.
Writes ‘old’ redo log files to archive location before they can be overwritten
11
Background Processes 2 of 2 smon
ckpt pmon
SMON - system monitor.
Responsible for instance recovery. Coalesces free space. Frees temporary segments
PMON - process monitor.
Keeps track of all processes - rolling back user transactions if the process fails.
CKPT - checkpoint.
Responsible for flushing database changes to disk. 12
Memory Structures 1 of 2 SGA PGA
Redo Buffer
PGA - Process Global Area One per server process Maps user process to the SGA Exists only for the duration of the user session 13
Memory Structures 2 of 2 SGA
SGA - System Global Area
PGA
Redo Buffer
Shared Pool Database Buffer Cache Data Dictionary Cache Java Pool
REDO - Redo Log Buffer Buffers all changes to data prior to being sent to disk 14
Files Data / Indexes
Data and Index
Control Files
Redo Logs
1 of 4 Archive Logs
Undo / Temp
Data - holds your corporate data Index - contain metadata used to assist speedy access to data and enforce business rules (uniqueness) Stored separate from each other and often split up across many devices Use RAID storage 15
Files Data / Indexes
Control Files
Redo Logs
2 of 4 Archive Logs
Undo / Temp
Control files Contain name & location of data files make up the database. Tracks the System Change Number Mirrored by Oracle
16
Files Data / Indexes
Control Files
Redo Logs
3 of 4 Archive Logs
Undo / Temp
Online Redo log files Information to reapply database changes in case of failure. Limited number of these used in a circular fashion Mirrored by Oracle
Archive log files Archived copy of online redo logs as they ‘age’ Mirrored by Oracle or OS 17
Files Data / Indexes
Control Files
Redo Logs
4 of 4 Archive Logs
Undo contains information to undo a transaction in the event of rollback (i.e. the ‘old’ values) Temp is a scratch area for temporary operations - usually sorting 18
Undo / Temp
Files Summary Data / Indexes
Control Files
Redo Logs
Archive Logs
Undo / Temp
This is a simplification of what is possible. Refer to the OFA - Optimal Flexible Architecture for a more detailed explanation This topology applies for local disk or SAN UNLESS you have very high performing SAN. 19
Component Summary Remote Users
Software
Node
Server and Background processes Memory structures
Local Users
SQLnet Listener
Server Process
Instance PGA
Database Files Data and Index files
dbwr
arch
lgrw
SGA smon Redo Buffer
Control files Redo log files
Software
Archive log files
ckpt pmon
Data / Indexes Control Files
Temp and Undo files
Redo Logs
So we understand what these components are - how do they work together 20
Archive Logs Undo / Temp
Sample user session Present a simplified user session 1. User Scott connects to the PROD database 2. Selects employee salaries 3. Updates employee salary 4. Commits the change 5. Ends the session VERY simplified (no security, no data dictionary, no optimizer, fuzzy timeline) 21
User Scott Connects
• •
sqlplus scott/tiger@prod Client process (user) accesses SQLnet listener
Remote Users
Node SQLnet Listener
Instance PGA
•
SQLnet listener confirms that database is ‘open for business’ and creates server process server process allocates PGA
dbwr
arch
lgrw
SGA smon Redo Buffer
Software
• •
Server Process
ckpt pmon
Data / Indexes Control Files Redo Logs
‘connected’ message returned to the user
Archive Logs Undo / Temp
Issues Select
• • •
Remote Remote Users Users
select * from emp; Node
server process (via the PGA) checks SGA to see if the data is already in the buffer cache If not then data is retrieved from disk and copied to the SGA (database buffer cache)
SQLnet Listener
Instance PGA
•
Data is returned to user via the PGA and server process. Cached data is left in the SGA
dbwr
arch
lgrw
SGA smon Redo Buffer
Software
•
Server Server Process Process
ckpt pmon
Data / Indexes Control Files Redo Logs Archive Logs Undo / Temp
Issues Update Remote Users
• • • • • • •
update emp set salary = 300 where empid = 10; server process (via the PGA) checks SGA to see if the data is already in the buffer cache
Node SQLnet Listener
Instance PGA
In our situation chances are the data is still in the SGA Data updated in the database buffer cache and marked as ‘dirty’ ‘update emp’ placed into redo buffer Row updated message returned to user
Server Server Process Process
dbwr
arch
lgrw
SGA smon Redo Buffer
Software
ckpt pmon
Data / Indexes Control Files Redo Logs Archive Logs
Note that NOTHING is written to disk yet
Undo / Temp
Commits work
• • • • • • • •
commit;
Remote Users
Newest System Change Number (SCN) obtained from control files Data in the database buffer cache marked as ‘updated and ready for saving’ ‘commit’ placed into redo buffer LGWR writes redo buffer contents to redo log files & removes from redo buffer
Node SQLnet Listener
Instance PGA
Commit complete message returned to user Later ... DBWR updates emp table in the data file and updates the header of the data file with the latest SCN
dbwr dbwr
arch
lgrw
SGA smon Redo Buffer
Software
Control file updated with latest SCN
Server Process
ckpt pmon
Data / Indexes Control Control Files Files Redo Logs Archive Logs Undo / Temp
Ends Session Remote Users
• • • • • •
exit;
Node SQLnet Listener
Unsaved changes are rolled back (we don’t have any) Server process deallocates the PGA
Server Process
Instance PGA
dbwr
arch arch
lgrw
SGA smon Redo Buffer
ckpt pmon
Server process terminates Software
Remote user terminates After some period of time - redo logs are archived by ARCH process
Data / Indexes Control Files Redo Logs Archive Logs Undo / Temp
Topic 1 Summary Discussed traditional Oracle architecture Defined and explained the major components of an Oracle Instance Walked through a simple user session 27
Topic 2 Traditional Oracle architecture Performance / Availability technologies Oracle in a partitioned world Real Application Clusters (RAC) Management 10g (if time permits)
28
Performance Availability Technologies that Oracle provides to deliver High Performance and High Availability (to a standard configuration) Parallel Server Replication Dataguard 29
Parallel Server(old) Remote Users
Remote Users Node
Local Users
SQLnet Listener
Node
Server Process
Local Users
Instance PGA
dbwr
lgrw
PGA
arch
dbwr
lgrw
arch
SGA smon
Software
Server Process
Instance
SGA
Redo Buffer
SQLnet Listener
ckpt
smon
pmon
Redo Buffer
Data / Indexes
Control Files
Redo Logs
Archive Logs
Undo / Temp
30
ckpt pmon
Software
Parallel Server Parallel server was still limited by the hardware on which it resided - in the beginning mostly on VMS only Did not provide fail-over protection SGAs totally separate with no load balancing Not terribly popular due to : cost and complexity 31
Performance Fortunately every release of the RDBMS features SUBSTANTIAL improvements in the RDBMS optimizer and buffering/caching algorithms (sometimes 10 - 100x). Multiple database writers, log writers Larger Caches Took advantage of multiple CPU’s by making many sql operations ‘parallelizable’. 32
Replication Remote Users
A second instance on a separate server is used to replicate all changes to the original database. This is referred to as the Standby database.
Node Local Users
SQLnet Listener
Server Process
Instance PGA
dbwr
arch
lgrw
SGA smon Redo Buffer
Software
ckpt pmon
Data / Indexes Control Files
Done via database triggers defined in the database which fire for every row updated. Can be failed over to fairly rapidly but requires intervention
Redo Logs Archive Logs Undo / Temp
Node Local Users
33
Server Process
Instance PGA
dbwr
arch
lgrw
SGA smon Redo Buffer
Software
Traditionally an active-inactive setup
SQLnet Listener
ckpt pmon
Data / Indexes Control Files Redo Logs Archive Logs Undo / Temp
Replication Remote Users
Remote Users Node
Local Users
Two way replication enables you to change this into an activeactive configuration
Somewhat complex setup Situations where one of the nodes fails can also be complicated 34
Server Process
Instance PGA
dbwr
arch
lgrw
SQLnet Listener
Server Process
PGA
dbwr
arch
lgrw
SGA smon
Redo Buffer
Node Local Users
Instance
SGA
Software
This can be extended to n nodes to achieve n:n replication
SQLnet Listener
ckpt pmon
Data / Indexes
smon Redo Buffer
Software
ckpt pmon
Data / Indexes
Control Files
Control Files
Redo Logs
Redo Logs
Archive Logs
Archive Logs
Undo / Temp
Undo / Temp
Data Guard Remote Users
Replacement for replication, Data Guard is Oracle’s definitive Disaster Recovery solution
Node Local Users
Server Process
Instance PGA
dbwr
arch
lgrw
SGA smon Redo Buffer
Database is cloned to a standby database and archive logs are constantly applied to the standby.
SQLnet Listener
Software
ckpt pmon
Data / Indexes Control Files Redo Logs Archive Logs Undo / Temp
Node
Results in low load on the source database
Local Users
SQLnet Listener
Server Process
Instance PGA
dbwr
arch
lgrw
SGA
Traditionally an active-inactive setup
smon Redo Buffer
Software
ckpt pmon
Data / Indexes Control Files Redo Logs Archive Logs
35
Undo / Temp
Data Guard Remote Users
You can however use the 2nd server in read-only mode Useful as a report server or datawarehouse extraction server Useful as an upgrade facility as well Great for long distances because the archive logs are applied asynchronously 36
Remote Users
Node Local Users
SQLnet Listener
Server Process
dbwr
arch
lgrw
SGA
Software
Server Process
PGA
dbwr
arch
lgrw
SGA smon
Redo Buffer
SQLnet Listener
Instance
Instance PGA
Node Local Users
ckpt pmon
Data / Indexes
smon Redo Buffer
Software
ckpt pmon
Data / Indexes
Control Files
Control Files
Redo Logs
Redo Logs
Archive Logs
Archive Logs
Undo / Temp
Undo / Temp
Performance Availability Summary Remote Users
Node Local Users
Parallel server evolved into parallel processing with multiple processes working to spread the load. Larger / more efficient caches
SQLnet SQLnet Listener SQLnet Listener SQLnet Listener Listener
Server Process Remote Users
Instance PGA
dbwr dbwr dbwr
lgrw lgrw lgrw
arch arch arch
Node Local Users
SGA ckpt pmon
PGA
arch
lgrw
smon Redo Buffer
ckpt pmon
Data / Indexes Control Files
Software Data / Indexes Control Files Redo Logs Archive Logs Undo / Temp
37
dbwr
SGA
Software
Replication evolved into Data Guard
Server Process
Instance
smon Redo Buffer
SQLnet Listener
Redo Logs Archive Logs Undo / Temp
Topic 3 Overview of traditional Oracle architecture Performance / Availability technologies Oracle in a partitioned world Real Application Clusters (RAC) Management 10g (if time permits) 38
Partitioned Server
So how does all this fit into a partitioned server environment?
39
Oracle on Partitioned Server pt1 Remote Users Node Local Users
No Oracle software or architecture changes required
SQLnet Listener
Server Process
Instance PGA
dbwr
arch
lgrw
SGA smon Redo Buffer
Software
ckpt pmon
Data / Indexes Control Files Redo Logs Archive Logs Undo / Temp
40
Oracle on Partitioned Server 1 of 3
CPU 1
Simplifying the image...
Production Instance
CPU 2
“Right size” the partition Separate Production & Development / Testing partitions
CPU 3
Development Instance
41
CPU 4
Oracle on Partitioned Server 2 of 3 Balance Production and Development resources based upon need Especially useful at registration or month end If using Dataguard a micropartition for the standby database. A Test partition that only exists for testing OS or database upgrades
CPU 1
CPU 2 Production Instance CPU 3
Development Instance DataGuard (standby) Instance
42
CPU 4
Oracle on Partitioned Server 3 of 3 Oracle Management Server (OMS) A repository used to manage the Oracle instance(s) with the Oracle Enterprise Manager (OEM - a gui tool) can be placed in a separate partition ensuring performance of the management server at critical times If your application server / web server is on the same server it too can be “right sized” in a separate partition. Network traffic between the RDBMS and Application Server remains internal to the server 43
Partitioned Summary Since the partitioned servers all basically provide the same or similar features this can be implemented on all three of the platforms discussed previously in the manner in which those platforms support partitioning. Are there more options? 44
Topic 4 Overview of traditional Oracle architecture Performance / Availability technologies Oracle in a partitioned world Real Application Clusters (RAC) Management 10g (if time permits) 45
Oracle RAC Real Application Clusters
46
RAC
Real Application Clusters - introduced as 9i RAC June 2001 Spread the work of the RDBMS engine across multiple servers Scale out - not up. Acquire CPU as you need it Added benefit of high availability / fault tolerance 47
RAC Philosophy As opposed to achieving high performance and high availability via a platform of sophisticated software layered on top of sophisticated hardware - build one based upon commodity servers utilizing an efficient OS (Linux) with a focus on core functionality. If the servers are easily replaced and the cluster can function without one then don’t invest too much in preventing it from failing in the first place (Oracle Information Architecture Overview whitepaper 2004)
48
RAC Architecture A simplified illustration High Speed Interconnect Delivering Cache Fusion
Node A SQLnet Listener
Node B
Server Process
SQLnet Listener
Instance A PGA Redo Buffer
Node C
Server Process
SQLnet Listener
Instance B dbwr
smon
Instance C
PGA
arch
lgrw
dbwr
Redo Buffer
ckpt pmon
SGA
PGA
arch
lgrw
smon
Redo Buffer
ckpt pmon
SGA
Software
Control Files
dbwr
arch
lgrw
smon
ckpt pmon
SGA
Software
Data / Indexes
Server Process
Redo Logs
Archive Logs
49
Software
Undo / Temp
RAC Architecture A natural evolution of Parallel Server multiple nodes working together with a high speed interconnect that allows the database instances to fuse their SGAs together. This ‘cache fusion’ enables each instance to be aware of what all other instances are doing and enables improved performance, commit consistency and fail over.
High Speed Interconnect Delivering Cache Fusion
Node A SQLnet Listener
Node B
Server Process
SQLnet Listener
Instance A PGA Redo Buffer
SQLnet Listener
lgrw
smon
dbwr
Redo Buffer
ckpt pmon
PGA
arch
lgrw
smon
Redo Buffer
ckpt pmon
SGA
Data / Indexes
Redo Logs
dbwr
arch
lgrw
smon
ckpt pmon
SGA
Software
Control Files
Server Process
Instance C
PGA
arch
Software
50
Server Process
Instance B dbwr
SGA
Each node serves up an instance complete with memory structures and background processes
Node C
Archive Logs
Software
Undo / Temp
RAC Architecture Remote Users
Remote Users
SQLnet listeners on each node loadbalance user connections Utilizes cache fusion to share SGA for the purpose of coordination of updates and application failover
pt2
Remote Users
Remote Users
Prod Database
High Speed Interconnect Delivering Cache Fusion
Node A SQLnet Listener
Node B
Server Process
SQLnet Listener
Instance A PGA Redo Buffer
Node C
Server Process
SQLnet Listener
Instance B dbwr
smon
Instance C
PGA
arch
lgrw
dbwr
Redo Buffer
ckpt pmon
SGA
PGA
arch
lgrw
smon
Redo Buffer
ckpt pmon
SGA
Software
Data / Indexes
51
Redo Logs
dbwr
arch
lgrw
smon
ckpt pmon
SGA
Software
Control Files
Server Process
Archive Logs
Software
Undo / Temp
RAC Instances Users access what appears to be a single database but is in fact: Multiple instances (processes + memory) serving up a SINGLE Data Base Multiple instances can (and almost always do) exist on separate hardware Client side OR server side load balancer (listener) chooses instance for a user at connect time 52
Cache Fusion
Cache Fusion Data retrieved by one instance is available to all (less disk IO) via shared Database Buffer Caches (part of SGA) SQL in the shared pool (another part of the SGA) is also shared across instances as well which means that a query issued on one instance can be continued by another. 53
RAC Servers Multiple nodes serve up the instances (usually 1 to 1) These need a separate high speed interconnect dedicated to transporting the cache fusion (usually gigabit ethernet) Nodes also need access to shared storage for database files All nodes run same OS (for now) but hardware can vary Dual architect according to needs / $ node A
node B
Data / Indexes
node C
Control Files
node D
Redo Logs
54
Archive Logs
node E
Undo / Temp
node F
Node/Instance Failure Users (SQLNET CLIENTS)
PROD Database
PROD1
PROD2
PROD3
PROD4
PROD5
PROD6
node D
node E
node F
Cache Fusion - shared SGA
node A
node B
Data / Indexes
node C
Control Files
Redo Logs
55
Archive Logs
Undo / Temp
Fail-over Since all nodes are aware of each other, sessions (user connections) can fail over Fail-over behavior depends on the kind of TAF (transparent application failover) policy you have set for the service (later) Basically a query and session can fail PROD3 PROD4 over but server side variables and Cache Fusion - shared SGA package states do not (yet...) Can use NONE as a TAF policy. This results in no fail-over but improved node C performance. 56
node D
PROD5
PROD6
node E
node F
Multiple DBs Multiple databases, multiple instances on multiple nodes all on one Cluster PROD has most resources and most availability TEST has no fail over node
PPRD
PROD
node A
TEST
node B
node C
node D
57
node E
node F
Multiple DBs The workload can be re balanced by starting and stopping instances on each node For max flexibility have instances for each database on every node PPRD
PROD
node A
TEST
node B
node C
node D
58
node E
node F
Services
1 of 3
A SQLnet connect time load balancing mechanism by which users are directed to groups of instances (remember PROD in this case is 5 instances)
Dynamic (via OEM) and /or time based via cron. Each service has a its own failover policy Finance Payroll Enrollment
HR
PROD
node A
TEST
node B
node C
node D 59
node E
node F
Services
2 of 3
If one service MUST have the maximum resources you can put all the others on one node with no fail over. Should that one node fail enable the service on one of the remaining nodes and you’re back in business with minimal disruption in service Finance Payroll Enrollment
HR
PROD
node A
TEST
node B
node C
node D 60
node E
node F
Services
3 of 3
User or application specifies which service to use at connect time then SQLnet listener chooses a node that is offering that service Services are not granular Load can be balanced based upon: Individual node work load # of sessions Simple round robin pmon communicates with the listener to provide node load and # of sessions so the listener can load balance You can even have simple client side load balancer 61
Resource Plans 1 of 2 Resource Plans let you allocate resource utilization further Users are associated with a resource group and these groups are allocated % of resources CPU, # concurrent sessions, shared pool ... 9 am
12 pm
5 pm
Finance 50
Finance 25
Finance 25
Payroll 10
Payroll 30
62
Batch 30
Human Resources 30
Human Resources 10
Payroll 10
Human Resources 10
Batch 55
Batch 15
Resource Plans 2 of 2 Resource Plans can be defined in a multi-level fashion (max 8) Level 1 40% left over
Level 2 This 40% is divided thusly:
Finance 40
Engineering 35
Level 3 The remaining 20% is subdivided:
OLAP 50
63
Payroll 10
HR 10
Facilities 10
Other 40
Adhoc 35
Reorg 10
Other 20
Other 40
Rolling Upgrades One of the benefits of RAC is that many patches can be done in a rolling fashion. Nodes are removed from the cluster, patched up and then re-introduced into the cluster one at a time Result is 0 downtime Not applicable to all patches. Determined by Oracle when the patch is created Basically if the structure or contents of the underlying database is being modified then the patch is probably not going to be ‘rollable’ 64
Grow the cluster The end of the year is at hand and you budgeted for an extra node so how to add it? Metalink note 270512.1 - use the existing nodes to add a new one in 5 steps Install OS and configure as prereq’s state
(it may
have been wise to save a disk image from your initial install)
CRS_HOME - addNode.sh from existing node to install CRS software (uses OUI) ORACLE_HOME - addNode.sh from existing node to install RDBMS software (uses OUI) 65
Grow the cluster NETCA (Network Configuration Assistant) - to reconfigure the SQLnet listeners to recognize the new node DBCA (DataBase Configuration Assistant) - to create the new instance via ‘instance management’ Done - the new node is now part of the RAC serving up an instance and services ‘Bare metal install’ in 10g R2 will require ‘an IP and nothing else’. Clones the OS, OCFS, CRS and RDBMS from an existing node via the web interface. 66
Shrink the cluster What if you need to free up a node for some other purpose (can’t imagine why???) Metalink Note 269320.1 - 4 basic steps Use DBCA to remove the instance from the node Use scripts on the node being freed up to logically remove the node from the cluster Use OUI to remove the RDBMS and CRS software Manually delete init. scripts & configure OS ??? if 10g R2 gives you an easier way to do this 67
RAC Components OS Hardware & Networking Software Storage
68
RAC Components OS Hardware & Networking Software Storage
69
RAC - OS Can run RAC on: Windows 2000, 2003 VMS HP Tru54 HP-UX Sun Solaris IBM AIX Linux (Suse and Redhat and ...) Mac OSX 70
RAC - Hardware x86 Itanium AMD PA-Risc Sparc IBM Power4, Power5 and s390 71
Windows, VMS OS
Hardware
RAC
Windows 2000
x86
9.2 and 10g
Windows 2003
AMD64 (32 bit mode only)
9.2 and 10g
Windows 2003
Itanium
9.2 and 10g
Windows 2003
x86
9.2 and 10g
VMS 7.3
Alpha EV56+
9.2
VMS 8.x
?
nothing yet
72
Linux OS
Hardware
RAC
RedHat AS/ES 3
AMD64
9.2 and 10g
RedHat AS/ES 3
EMT64
10g
Redhat AS/ES 2.1 or 3
Itanium
9.2 and10g
Redhat AS/ES 2.1 or 3
x86
9.2 and 10g
Redhat AS 3
IBM zSeries
10g(projected)
SUSE SLES 8 or 9
AMD64
9.2 and 10g
SUSE SLES 8 or 9
EMT64
10g
SUSE SLES 8 or 9
Itanium
9.2 and 10g
SUSE SLES 8 or 9
x86
9.2 and 10g
SUSE SLES 7 or 8
IBM s390
9.2
SUSE SLES 8
IBM zSeries
9.2 and 10g(projected)
73
Unix OS
Hardware
RAC
HP Tru64 5.1a
Tru64 Cluster
9.2
HP Tru64 5.1b
Tru64 Cluster
9.2 and 10g
HP-UX 11i v1.6
Itanium
9.2
HP-UX 11i v2
Itanium
9.2 and 10g
HP-UX 11.0
PA-Risc
9.2
HP-UX 11i v1 and v2
PA-Risc
9.2 and 10g
Solaris 8 and 9
Sparc
9.2 and 10g
Solaris 9
x86
10g
AIX 5l 5.2 and 5.3
pSeries, iSeries
9.2 and 10g
AIX 5l 5.1
pSeries, iSeries
9.2
74
Compatibility Matrix Doublecheck metalink for certified hardware i.e. the “RAC Technologies Compatibility Matrix” Contact ITEC
75
Application Support Check that the version of Oracle you want to run in the configuration you want to run is supported by your application. For Banner check with the SICAS center. SCT FAQ 11533 76
Application Support If your application does not connect via sqlnet and/or is local you can run it unchanged on a single node in the cluster or ‘replicate it’ across nodes (manually) Of course it will not magically become ‘cluster aware’ 77
RAC Components OS Hardware & Networking Software Storage
78
Hardware Basically 2 (or more) nodes each with dual ethernet ports and a mechanism to attach to shared storage (usually fibre channel) Based on Linux install - suggest each node have at least a 2ghz processor and 2gb of RAM Can do a one node RAC install and add nodes later Can also migrate a non RAC implementation to RAC Oracle Technology Network (OTN) even has RAC VM’s that you can download and use (beta) 79
Networking Need private interconnect for cache fusion (gigabit ethernet) Need public ethernet controller as well Need 3 IP’s per node: 1 for the interconnect 1 for the outside world 1 Virtual IP - used to enable rapid failover 80
RAC Components OS Hardware & Networking Software Storage
81
Software Components
There are 3 software components to a RAC implementation Clustered File System (CFS) Cluster Ready Services (CRS) Database software (RDBMS)
App RDBMS CRS CFS O/S Hardware
82
CFS A Clustered File System is responsible for serving up the shared datafiles. Oracle includes OCFS on Linux (standard RPM) and some other platforms - includes management tools Or you can use other vendors clustering file system if it has been certified. 83
App RDBMS CRS CFS O/S Hardware
CRS Cluster Ready Services is supplied by Oracle and acts as the RAC management piece. It knows what nodes are involved and provides the mechanism for cache fusion and handles node up / down events Includes management tools 84
App RDBMS CRS CFS O/S Hardware
RDBMS Oracle RDBMS (code tree) Same as we discussed earlier in traditional setup
App RDBMS
Core RDBMS + RDBMS options
CRS
SQLnet server software
CFS
Oracle Enterprise Manager
O/S
There is no special RAC media - it’s included on the standard CDs , DVDs
85
Hardware
RAC Components OS Hardware & Networking Software Storage
86
Storage Many possible combinations of storage options but basically: There are 4 options for file systems on which to put the software and database files: non shared
non-shared
CFS
RAW
RAW
CFS - Clustered File System
ASM
ASM - Automatic Storage Management 87
non shared non shared
CFS
RAW
ASM
Non shared drives CAN be used to store the oracle software but NOT datafiles or control files or redologs. This is useful if you want to take advantage of rolling upgrades. Nodes are taken out of the cluster, their software patched up and then placed back into the mix. 88
CFS non shared
CFS
RAW
Data redo and control files Archive logs Optionally can place Oracle software on CFS with trade offs: A single code tree to maintain No rolling upgrades 89
ASM
RAW non shared
CFS
RAW
As always RAW devices can be used for datafiles Cannot be used for software Are difficult to maintain as you cannot use standard OS utilities 90
ASM
ASM non shared
CFS
RAW
Automatic Storage Management (ASM) is a new tool provided by Oracle Performance of RAW with convenience of standard file systems. Separate install with its own ‘instance’ 91
ASM
ASM non shared
CFS
RAW
ASM
Simplifies the management of disks AND replaces / supplements traditional volume managers n - way mirroring(software) Version 1.0 Performance is an unknown at this time
92
RAC Components OS Hardware & Networking Software Storage
So now we know about the components - how to install (very brief) 93
Install in 7 steps Prereq checks / config on all nodes Test interconnect and remote ssh on all nodes
O/S O/S O/S
Install CFS on all nodes
CFS CFS CFS
Install CRS (using OUI) into its own home on one node - will replicate via rsh
CRS
Install RDBMS (using OUI) on one node will replicate via rsh Create database using DBCA from one node Configure backup policy using OEM 94
RDBMS
DBCA Database Configuration Assistant (DBCA) - a GUI for creating databases DBCA is RAC aware Specify the nodes you want instances on Create the services you want and specify nodes that offer each service 95
Architecture Specifics A brief discussion of RAC implemented on: Standalone or 1u/ 2u rac mounted servers Blades in a bladecenter How to take advantage of partitioned servers for RAC 96
RAC in a rack Avoid temptation to mix vendors too much Multiple ethernet and fiber cards may be of concern minimum 2 ethernet and 1 fiber per server
Gigabit switch Toss out old servers to make room for new (or not)? 97
Rac on Bladecenter Very similar to 1u/2u Usually get integrated management software Usually the high speed interconnect is done via the backplane May be problematic building a RAC that spans bladecenters due to limited # of ethernet ports depending on the vendor 98
RAC on Partitioned Server Use only 2 partitions for maximum availability (and rolling upgrades) while retaining the servers intrinsic flexibility (moving cpu’s, IO and memory) Perhaps a 3rd small (micro) partition with ONLY the database control and backup running off it (ie. a DBA only partition)
99
Ultimate RAC? Node A SQLnet Listener
Node B
Server Process
SQLnet Listener
Instance PGA Redo Buffer
Server Process
lgrw
smon
dbwr
Redo Buffer
ckpt pmon
RAC with Dataguard implemented on RAC
lgrw
smon
Redo Buffer
ckpt pmon
Control Files
Redo Logs
Archive Logs
Undo / Temp
Node C
Server Process
SQLnet Listener
dbwr
Redo Buffer
ckpt
SGA
PGA
arch
lgrw
smon
Redo Buffer
ckpt pmon
SGA
Software
Redo Logs
dbwr
arch
lgrw
smon
ckpt pmon
SGA
Software
Control Files
Server Process
Instance
PGA
pmon
Data / Indexes
ckpt pmon
Software
Instance arch
smon
smon
Node B SQLnet Listener
Instance lgrw
arch
lgrw
Software
Server Process
dbwr
dbwr
SGA
Node A SQLnet Listener
Redo Buffer
PGA
arch
SGA
Data / Indexes
Server Process
Instance
PGA
arch
Software
PGA
SQLnet Listener
Instance dbwr
SGA
Rolling upgrades on both your main database and the standby
Node C
Archive Logs
Software
Undo / Temp
Summary
1 of 3
Discussed typical Oracle architecture in a traditional single server configuration Looked at Oracle in a partitioned server environment with improved flexibility yielding rapid dynamic workload management 101
Summary
2 of 3
RAC technology CAN provide high performance and high availability using commodity servers RAC is very flexible and can grow and shrink to meet demand Arguably not as quickly as a partitioned server as it takes hours instead of minutes. 102
Followup Presentations Possible presentations at STC or fall Wizard: “Getting On the RAC” - a “warts and all” installation walkthrough “Life on the RAC” - day to day DBA duties in a RAC environment 103
Collateral Presentation and assorted documents are available: http://www2.itec.suny.edu/wizard/2005/spring/oracle Oracle has a ‘RAC HOME’ on their site located at: http://www.oracle.com/database/rac_home.html Also check out: http://www.oracle.com/technology/products/database/clustering/index.html Try RAC on Linux on VMware: http://www.oracle.com/technology/tech/linux/vmware/cookbook/index.html Technet (sign up for a free account): http://www.oracle.com/technology email:
[email protected] or
[email protected]
104
105
