Step-By-step Installation Guide RAC 12cR1 on Linux

STEP BY STEP INSTALLATION OF ORACLE RAC 12cR1 (12.1.0.2) ON LINUX X86-64

Oracle Global Customer Support - RAC / Scalability Copyright © 1993, 2014, Oracle and/or its affiliates. All rights reserved

1

Contents 1.

Introduction ....................................................................................................................................... 3 1.1

1.1.1

Server Hardware Checklist for Oracle Grid Infrastructure ................................................. 3

1.1.2

Environment Configuration for Oracle Grid Infrastructure and Oracle RAC ..................... 3

1.1.3

Network Configuration Tasks for Oracle Grid Infrastructure and Oracle RAC ................... 4

1.1.4

Oracle Grid Infrastructure Storage Configuration Checks ................................................. 5

1. 2

Configuring Servers for Oracle Grid Infrastructure and Oracle RAC .......................................... 6

1.2.1

Checking Server Hardware and Memory Configuration .................................................... 6

1.2.2

Server Storage Minimum Requirements........................................................................... 7

1.2.3

64-bit System Memory Requirements ............................................................................... 7

1.3

2.

Oracle Grid Infrastructure Installation Server Hardware Checklist ........................................... 3

Operating System Requirements for x86-64 Linux Platforms ................................................... 7

1.3.1

Supported Oracle Linux 7 and Red Hat Enterprise Linux 7 Distributions for x86-64 ......... 8

1.3.2

Supported Oracle Linux 6 and Red Hat Enterprise Linux 6 Distributions for x86-64 ......... 9

1.3.3

Supported Oracle Linux 5 and Red Hat Enterprise Linux 5 Distributions for x86-64 ....... 10

Prepare the cluster nodes for Oracle RAC ....................................................................................... 12 2.1. User Accounts ............................................................................................................................... 12 2.2

Networking............................................................................................................................... 12

2.3. Synchronizing the Time on ALL Nodes .......................................................................................... 14 2.4

Installing the Oracle Preinstallation RPM with ULN support ................................................... 14

2.5

Configuring Kernel Parameters ................................................................................................ 15

2.6

Set shell limits for the oracle user............................................................................................ 16

2.7

Create the Oracle Inventory Directory..................................................................................... 17

2.7. Creating the Oracle Grid Infrastructure Home Directory ............................................................. 17 2.8. Creating the Oracle Base Directory ............................................................................................... 17 2.9. Creating the Oracle RDBMS Home Directory ................................................................................ 17 2.10. Stage the Oracle Software .......................................................................................................... 17 3. Prepare the shared storage for Oracle RAC ......................................................................................... 18 3.1. Shared Storage .............................................................................................................................. 18 3.1.1. Partition the Shared Disks ...................................................................................................... 18 3.1.2. Installing and Configuring ASMLib ......................................................................................... 19 3.1.3. Using ASMLib to Mark the Shared Disks as Candidate Disks ................................................. 20 3.2

Setting Disk I/O Scheduler on Linux ......................................................................................... 21

4.

Oracle Grid Infrastructure Install ..................................................................................................... 22

5.

RDBMS Software Install ................................................................................................................... 45

6.

ASMCA to create Diskgroups ........................................................................................................... 57

7.

Run DBCA to create the database.................................................................................................... 60

8.

Applying Latest PSUs to GRID & RDBMS Homes .............................................................................. 74

2

1.

Introduction

1.1

Oracle Grid Infrastructure Installation Server Hardware Checklist

1.1.1

Server Hardware Checklist for Oracle Grid Infrastructure

Server hardware: server make, model, core architecture, and host bus adaptors (HBA) are supported to run with Oracle RAC. Network Switches: •

Public network switch, at least 1 GbE, connected to a public gateway.

•

Private network switch, at least 1 GbE, with 10 GbE recommended, dedicated for use only with other cluster member nodes. The interface must support the user datagram protocol (UDP) using high-speed network adapters and switches that support TCP/IP. Alternatively, use InfiniBand for the interconnect.

Runlevel: Servers should be either in runlevel 3 or runlevel 5. Random Access Memory (RAM): At least 4 GB of RAM for Oracle Grid Infrastructure for a Cluster installations, including installations where you plan to install Oracle RAC. Temporary disk space allocation: At least 1 GB allocated to/tmp. Storage hardware: Either Storage Area Network (SAN) or Network-Attached Storage (NAS). Local Storage Space for Oracle Software •

At least 8 GB of space for the Oracle Grid Infrastructure for a cluster home (Grid home). Oracle recommends that you allocate 100 GB to allow additional space for patches.

•

At least 12 GB of space for the Oracle base of the Oracle Grid Infrastructure installation owner (Grid user). The Oracle base includes Oracle Clusterware and Oracle ASM log files.

•

For Linux x86-64 platforms, if you intend to install Oracle Database, then allocate 6.4 GB of disk space for the Oracle home (the location for the Oracle Database software binaries).

1.1.2

Environment Configuration for Oracle Grid Infrastructure and Oracle RAC

Create Groups and Users. A user created to own only Oracle Grid Infrastructure software installations is called the grid user. A user created to own either all Oracle installations, or only Oracle database installations, is called the oracle user. Create mount point paths for the software binaries. Oracle recommends that you follow the guidelines for an Optimal Flexible Architecture configuration. Review Oracle Inventory (oraInventory) and OINSTALL Group Requirements. The Oracle Inventory directory is the central inventory of Oracle software installed on your system. Users who have the Oracle Inventory group as their primary group are granted the OINSTALL privilege to write to the central inventory. Ensure that the Grid home (the Oracle home path you select for Oracle Grid Infrastructure) uses only ASCII characters

3

Unset Oracle software environment variables. If you have set ORA_CRS_HOME as an environment variable, then unset it before starting an installation or upgrade. Do not use ORA_CRS_HOME as a user environment variable. If you have had an existing installation on your system, and you are using the same user account to install this installation, then unset the following environment variables: ORA_CRS_HOME;ORACLE_HOME; ORA_NLS10; TNS_ADMIN. 1.1.3

Network Configuration Tasks for Oracle Grid Infrastructure and Oracle RAC

Public Network Hardware: •

Public network switch (redundant switches recommended) connected to a public gateway and to the public interface ports for each cluster member node.

•

Ethernet interface card (redundant network cards recommended, bonded as one Ethernet port name).

•

The switches and network interfaces must be at least 1 GbE.

•

The network protocol is TCP/IP.

Private Network Hardware for the Interconnect •

Private dedicated network switches (redundant switches recommended), connected to the private interface ports for each cluster member node. NOTE: If you have more than one private network interface card for each server, then Oracle Clusterware automatically associates these interfaces for the private network using Grid Interprocess Communication (GIPC) and Grid Infrastructure Redundant Interconnect, also known as Cluster High Availability IP (HAIP).

•

The switches and network interface adapters must be at least 1 GbE, with 10 GbE recommended. Alternatively, use InfiniBand for the interconnect.

•

The interconnect must support the user datagram protocol (UDP).

Oracle Flex ASM Network Hardware •

Oracle Flex ASM can use either the same private networks as Oracle Clusterware, or use its own dedicated private networks. Each network can be classified PUBLIC or PRIVATE+ASM or PRIVATE or ASM. ASM networks use the TCP protocol.

Cluster Names and Addresses: Determine and configure the following names and addresses for the cluster •

Cluster name: Decide a name for the cluster, and be prepared to enter it during installation. The cluster name should have the following characteristics: • •

•

Globally unique across all hosts, even across different DNS domains. At least one character long and less than or equal to 15 characters long.

Grid Naming Service Virtual IP Address (GNS VIP): If you plan to use GNS, then configure a GNS name and fixed address on the DNS for the GNS VIP, and configure a subdomain on your DNS delegated to the GNS VIP for resolution of cluster addresses. GNS domain delegation is mandatory with dynamic public networks (DHCP, autoconfiguration). 4

•

Single Client Access Name (SCAN) and addresses •

Using Grid Naming Service Resolution: Do not configure SCAN names and addresses in your DNS. SCANs are managed by GNS.

•

Using Manual Configuration and DNS resolution: Configure a SCAN name to resolve to three addresses on the domain name service (DNS).

Standard or Hub Node Public, Private and Virtual IP names and Addresses: •

Public node name and address, configured on the DNS and in /etc/hosts (for example, node1.example.com, address 192.0.2.10). The public node name should be the primary host name of each node, which is the name displayed by the hostname command.

•

Private node address, configured on the private interface for each node. The private subnet that the private interfaces use must connect all the nodes you intend to have as cluster members. Oracle recommends that the network you select for the private network uses an address range defined as private by RFC 1918.

•

Public node virtual IP name and address (for example,node1-vip.example.com, address 192.0.2.11).

1.1.4 Oracle Grid Infrastructure Storage Configuration Checks During installation, you are asked to provide paths for the following Oracle Clusterware files. These path locations must be writable by the Oracle Grid Infrastructure installation owner (Grid user). These locations must be shared across all nodes of the cluster, either on Oracle ASM (preferred), or on a cluster file system, because the files created during installation must be available to all cluster member nodes. •

Voting files are files that Oracle Clusterware uses to verify cluster node membership and status. The location for voting files must be owned by the user performing the installation (oracle or grid), and must have permissions set to 640.

•

Oracle Cluster Registry files (OCR) contain cluster and database configuration information for Oracle Clusterware. Before installation, the location for OCR files must be owned by the user performing the installation (grid or oracle). That installation user must have oinstall as its primary group. During installation, the installer creates the OCR files and changes ownership of the path and OCR files to root.

5

1. 2

Configuring Servers for Oracle Grid Infrastructure and Oracle RAC

1.2.1 Checking Server Hardware and Memory Configuration Run the following commands to gather your current system information: 1. To determine the physical RAM size, enter the following command: # grep MemTotal /proc/meminfo If the size of the physical RAM installed in the system is less than the required size, then you must install more memory before continuing. 2. To determine the size of the configured swap space, enter the following command: # grep SwapTotal /proc/meminfo If necessary, see your operating system documentation for information about how to configure additional swap space. 3. To determine the amount of space available in the /tmp directory, enter the following command: # df -h /tmp 4. To determine the amount of free RAM and disk swap space on the system, enter the following command: # free 5. To determine if the system architecture can run the software, enter the following command: # uname -m Verify that the processor architecture matches the Oracle software release to install. For example, you should see the following for a x86-64 bit system: x86_64 If you do not see the expected output, then you cannot install the software on this system. 6. Verify that shared memory (/dev/shm) is mounted properly with sufficient size using the following command: # df -h /dev/shm

6

1.2.2

Server Storage Minimum Requirements Each system must meet the following minimum storage requirements: •

1 GB of space in the /tmp directory.

If the free space available in the /tmp directory is less than what is required, then complete one of the following steps: o o

Delete unnecessary files from the /tmp directory to make available the space required. Extend the file system that contains the /tmp directory. If necessary, contact your system administrator for information about extending file systems.

•

At least 8.0 GB of space for the Oracle Grid Infrastructure for a cluster home (Grid home). Oracle recommends that you allocate 100 GB to allow additional space for patches.

•

Upto 10 GB of additional space in the Oracle base directory of the Grid Infrastructure owner for diagnostic collections generated by Trace File Analyzer (TFA) Collector.

•

At least 3.5 GB of space for the Oracle base of the Oracle Grid Infrastructure installation owner (Grid user). The Oracle base includes Oracle Clusterware and Oracle ASM log files.

•

For Oracle Solaris platforms, if you intend to install Oracle Database, then allocate 5.2 GB of disk space for the Oracle home (the location for the Oracle Database software binaries).

1.2.3 64-bit System Memory Requirements Each system must meet the following memory requirements: • •

1.3

At least 4 GB of RAM for Oracle Grid Infrastructure for cluster installations, including installations where you plan to install Oracle RAC. Swap space equivalent to the multiple of the available RAM, as indicated in the following table: Available RAM

Swap Space Required

Between 4 GB and 16 GB

Equal to RAM

More than 16 GB

16 GB of RAM

Operating System Requirements for x86-64 Linux Platforms

The Linux distributions and packages listed in this section are supported for this release on x86-64. No other Linux distributions are supported. Identify operating system requirements for Oracle Grid Infrastructure, and identify additional operating system requirements for Oracle Database and Oracle RAC installations. •

Supported Oracle Linux 7 and Red Hat Enterprise Linux 7 Distributions for x86-64

•

Supported Oracle Linux 6 and Red Hat Enterprise Linux 6 Distributions for x86-64 7

•

Supported Oracle Linux 5 and Red Hat Enterprise Linux 5 Distributions for x86-64

•

Supported SUSE Distributions for x86-64

1.3.1

Supported Oracle Linux 7 and Red Hat Enterprise Linux 7 Distributions for x86-64

Use the following information to check supported Oracle Linux 7 and Red Hat Linux 7 distributions: Oracle Linux 7 Supported distributions: •

Oracle Linux 7 with the Unbreakable Enterprise kernel: 3.8.13-33.el7uek.x86_64 or later

•

Oracle Linux 7 with the Red Hat Compatible kernel: 3.10.0-54.0.1.el7.x86_64 or later

Red Hat Enterprise Linux 7 Supported distribution: •

Red Hat Enterprise Linux 7: 3.10.0-54.0.1.el7.x86_64 or later

Packages for Oracle Linux 7 and Red Hat Enterprise Linux 7 binutils-2.23.52.0.1-12.el7.x86_64 compat-libcap1-1.10-3.el7.x86_64 gcc-4.8.2-3.el7.x86_64 gcc-c++-4.8.2-3.el7.x86_64 glibc-2.17-36.el7.i686 glibc-2.17-36.el7.x86_64 glibc-devel-2.17-36.el7.i686 glibc-devel-2.17-36.el7.x86_64 ksh libaio-0.3.109-9.el7.i686 libaio-0.3.109-9.el7.x86_64 libaio-devel-0.3.109-9.el7.i686 libaio-devel-0.3.109-9.el7.x86_64 libgcc-4.8.2-3.el7.i686 libgcc-4.8.2-3.el7.x86_64 libstdc++-4.8.2-3.el7.i686 libstdc++-4.8.2-3.el7.x86_64 libstdc++-devel-4.8.2-3.el7.i686 libstdc++-devel-4.8.2-3.el7.x86_64 libXi-1.7.2-1.el7.i686 libXi-1.7.2-1.el7.x86_64 libXtst-1.2.2-1.el7.i686 libXtst-1.2.2-1.el7.x86_64 make-3.82-19.el7.x86_64 sysstat-10.1.5-1.el7.x86_64 8

1.3.2

Supported Oracle Linux 6 and Red Hat Enterprise Linux 6 Distributions for x86-64

Use the following information to check supported Oracle Linux 7 and Red Hat Linux 7 distributions: Oracle Linux 6 Supported distributions: •

Oracle Linux 6 with the Unbreakable Enterprise kernel: 2.6.39-200.24.1.el6uek.x86_64 or later

•

Oracle Linux 6 with the Red Hat Compatible kernel: 2.6.32-71.el6.x86_64 or later

Red Hat Enterprise Linux 6 Supported distribution: •

Red Hat Enterprise Linux 6: 2.6.32-71.el6.x86_64 or later

Packages for Oracle Linux 6 and Red Hat Enterprise Linux 6 binutils-2.20.51.0.2-5.11.el6 (x86_64) compat-libcap1-1.10-1 (x86_64) compat-libstdc++-33-3.2.3-69.el6 (x86_64) compat-libstdc++-33-3.2.3-69.el6 (i686) gcc-4.4.4-13.el6 (x86_64) gcc-c++-4.4.4-13.el6 (x86_64) glibc-2.12-1.7.el6 (i686) glibc-2.12-1.7.el6 (x86_64) glibc-devel-2.12-1.7.el6 (x86_64) glibc-devel-2.12-1.7.el6 (i686) ksh libgcc-4.4.4-13.el6 (i686) libgcc-4.4.4-13.el6 (x86_64) libstdc++-4.4.4-13.el6 (x86_64) libstdc++-4.4.4-13.el6 (i686) libstdc++-devel-4.4.4-13.el6 (x86_64) libstdc++-devel-4.4.4-13.el6 (i686) libaio-0.3.107-10.el6 (x86_64) libaio-0.3.107-10.el6 (i686) libaio-devel-0.3.107-10.el6 (x86_64) libaio-devel-0.3.107-10.el6 (i686) libXext-1.1 (x86_64) libXext-1.1 (i686) libXtst-1.0.99.2 (x86_64) libXtst-1.0.99.2 (i686) libX11-1.3 (x86_64) libX11-1.3 (i686) libXau-1.0.5 (x86_64) libXau-1.0.5 (i686) 9

libxcb-1.5 (x86_64) libxcb-1.5 (i686) libXi-1.3 (x86_64) libXi-1.3 (i686) make-3.81-19.el6 sysstat-9.0.4-11.el6 (x86_64) 1.3.3

Supported Oracle Linux 5 and Red Hat Enterprise Linux 5 Distributions for x86-64

Use the following information to check supported Oracle Linux 7 and Red Hat Linux 7 distributions: Oracle Linux 5 Supported distributions: •

Oracle Linux 5 Update 6 with the Unbreakable Enterprise kernel: 2.6.32-100.0.19 or later

•

Oracle Linux 5 Update 6 with the Red Hat compatible kernel: 2.6.18-238.0.0.0.1.el5 or later

Red Hat Enterprise Linux 5 Supported distribution: •

Red Hat Enterprise Linux 6: 2.6.18-238.0.0.0.1.el5 or later

Packages for Oracle Linux 5 and Red Hat Enterprise Linux 5 binutils-2.17.50.0.6 compat-libstdc++-33-3.2.3 compat-libstdc++-33-3.2.3 (32 bit) coreutils-5.97-23.el5_4.1 gcc-4.1.2 gcc-c++-4.1.2 glibc-2.5-58 glibc-2.5-58 (32 bit) glibc-devel-2.5-58 glibc-devel-2.5-58 (32 bit) ksh libaio-0.3.106 libaio-0.3.106 (32 bit) libaio-devel-0.3.106 libaio-devel-0.3.106 (32 bit) libgcc-4.1.2 libgcc-4.1.2 (32 bit) libstdc++-4.1.2 libstdc++-4.1.2 (32 bit) libstdc++-devel 4.1.2 libXext-1.0.1 libXext-1.0.1 (32 bit) 10

libXtst-1.0.1 libXtst-1.0.1 (32 bit) libX11-1.0.3 libX11-1.0.3 (32 bit) libXau-1.0.1 libXau-1.0.1 (32 bit) libXi-1.0.1 libXi-1.0.1 (32 bit) make-3.81 sysstat-7.0.2 The following command can be run on the system to list the currently installed packages: rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' binutils \ compat-libstdc++-33 \ elfutils-libelf \ elfutils-libelf-devel \ gcc \ gcc-c++ \ glibc \ glibc-common \ glibc-devel \ glibc-headers \ ksh \ libaio \ libaio-devel \ libgcc \ libstdc++ \ libstdc++-devel \ make \ sysstat \ unixODBC \ unixODBC-devel Any missing RPM from the list above should be added using the "--aid" of "/bin/rpm" option to ensure all dependent packages are resolved and installed as well. NOTE: Be sure to check on all nodes that the Linux Firewall and SE Linux is disabled.

11

2.

Prepare the cluster nodes for Oracle RAC

2.1. User Accounts NOTE: We recommend different users for the installation of the Grid Infrastructure (GI) and the Oracle RDBMS home. The GI will be installed in a separate Oracle base, owned by user 'grid.' After the grid install the GI home will be owned by root, and inaccessible to unauthorized users. 1.

Create OS groups using the command below Enter commands as the root user:

#/usr/sbin/groupadd oinstall #/usr/sbin/groupadd dba #/usr/sbin/groupadd asmadmin #/usr/sbin/groupadd asmdba #/usr/sbin/groupadd asmoper 2.

Create the users that will own the Oracle software using the commands:

#/usr/sbin/useradd -g oinstall -G asmadmin,asmdba,asmoper -d /home/grid -m grid #/usr/sbin/useradd -g oinstall -G dba,asmdba -d /home/oracle -m oracle 3. Set the password for the oracle account using the following command. Replace password with your own password. #passwd oracle Changing password for user oracle. New UNIX password: password retype new UNIX password: password passwd: all authentication tokens updated successfully. #passwd grid Changing password for user oracle. New UNIX password: password retype new UNIX password: password passwd: all authentication tokens updated successfully Repeat Step 1 through Step 3 on each node in your cluster. OUI can setup passwordless SSH for you, if you want to configure this yourself, refer to Note. 300548.1

2.2

Networking

NOTE: This section is intended to be used for installations NOT using GNS. Determine your cluster name. The cluster name should satisfy the following conditions: •

The cluster name is globally unique throughout your host domain.

•

The cluster name is at least 1 character long and less than 15 characters long.

•

The cluster name must consist of the same character set used for host names: single-byte alphanumeric characters (a to z, A to Z, and 0 to 9) and hyphens (-).

12

Determine the public host name for each node in the cluster. For the public host name, use the primary hostname of each node. In other words, use the name displayed by the hostname command for example: racnode1. Determine the public virtual hostname for each node in the cluster. The virtual host name is a public node name that is used to reroute client requests sent to the node if the node is down. Oracle recommends that you provide a name in the format -vip, for example: racnode1vip. The virtual hostname must meet the following requirements: -The virtual IP address and the network name must not be currently in use. •

The virtual IP address must be on the same subnet as your public IP address.

•

The virtual host name for each node should be registered with your DNS.

Determine the private hostname for each node in the cluster. This private hostname does not need to be resolvable through DNS and should be entered in the /etc/hosts file. A common naming convention for the private hostname is -pvt. • • • • •

The private IP should NOT be accessible to servers not participating in the local cluster. The private network should be on standalone dedicated switch(es). The private network should NOT be part of a larger overall network topology. The private network should be deployed on Gigabit Ethernet or better. It is recommended that redundant NICs are configured with the Linux bonding driver. Active/passive is the preferred bonding method due to its simplistic configuration.

Define a SCAN DNS name for the cluster that resolves to three IP addresses (round-robin). SCAN VIPs must NOT be in the /etc/hosts file, it must be resolved by DNS. Even if you are using a DNS, Oracle recommends that you add lines to the /etc/hosts file on each node, specifying the public IP, VIP and private addresses. Configure the /etc/hosts file so that it is similar to the following example: NOTE: The SCAN IPs MUST NOT be in the /etc/hosts file. This will result in only 1 SCAN IP for the entire cluster. [oracle@cehaovmsp145 ~]$ cat /etc/hosts # Created by DB/RAC OVM at Tue Aug 25 16:59:39 EDT 2015 127.0.0.1 localhost localhost.localdomain localhost4 ::1 localhost localhost.localdomain localhost6 10.64.146.69 cehaovmsp145.us.oracle.com cehaovmsp145 10.64.131.119 cehaovmsp145-i.us.oracle.com cehaovmsp145-i 10.64.146.70 cehaovmsp145-v.us.oracle.com cehaovmsp145-v 10.64.146.92 cehaovmsp146.us.oracle.com cehaovmsp146 10.64.131.120 cehaovmsp146-i.us.oracle.com cehaovmsp146-i 10.64.146.93 cehaovmsp146-v.us.oracle.com cehaovmsp146-v # For referene: DNS IP is 192.135.82.132; SCAN Name is cehaovmsp1-scan23

13

If you configured the IP addresses in a DNS server, then, as the root user, change the hosts search order in /etc/nsswitch.conf on all nodes as shown here: Old: hosts: files nis dns New: hosts: dns files nis After modifying the nsswitch.conf file, restart the nscd daemon on each node using the following command: # /sbin/service nscd restart After you have completed the installation process, configure clients to use the SCAN to access the cluster. Using the previous example, the clients would use docrac-scan to connect to the cluster. The fully qualified SCAN for the cluster defaults to cluster_name-scan.GNS_subdomain_name, for example docrac-scan.example.com. The short SCAN for the cluster is docrac-scan. You can use any name for the SCAN, as long as it is unique within your network and conforms to the RFC 952 standard.

2.3. Synchronizing the Time on ALL Nodes Ensure that the date and time settings on all nodes are set as closely as possible to the same date and time. Time may be kept in sync with NTP with the -x option or by using Oracle Cluster Time Synchronization Service (ctssd). Instructions on configuring NTP with the -x option can be found in My Oracle Support Ext Note : 551704.1.

2.4

Installing the Oracle Preinstallation RPM with ULN support

Use the following procedure to subscribe to Unbreakable Linux Network (ULN) Oracle Linux channels, and to add the Oracle Linux channel that distributes the Oracle Preinstallation RPM: 1. Register your server with Unbreakable Linux Network (ULN). By default, you are registered for the Oracle Linux Latest channel for your operating system and hardware. 2.

Log in to Unbreakable Linux Network:

https://linux.oracle.com Opens a new window 3. Click the Systems tab, and in the System Profiles list, select a registered server. The System Details window opens and displays the subscriptions for the server. 4.

Click Manage Subscriptions. The System Summary window opens.

14

5. From the Available Channels list, select the Linux installation media copy and update patch channels corresponding to your Oracle Linux distribution. For example, if your distribution is Oracle Linux 5 Update 6 for x86_64, then select the following: Oracle Linux 5 Update 6 installation media copy (x86_64) Oracle Linux 5 Update 6 Patch (x86_64) 6.

Click Subscribe.

7. Start a terminal session and enter the following command as root, depending on your platform. For example: Oracle Linux 6: # yum install oracle-rdbms-server-12cR1-preinstall Oracle Linux 5: # yum install oracle-validated You should see output indicating that you have subscribed to the Oracle Linux channel, and that packages are being installed. For example: el5_u6_i386_base el5_u6_x86_64_patch Oracle Linux automatically creates a standard (not role-allocated) Oracle installation owner and groups, and sets up other kernel configuration settings as required for Oracle installations. Repeat steps 1 through 7 on all other servers in your cluster.

2.5

Configuring Kernel Parameters

Note:- This section can be ignored if you have setup the rpm using the previous steps As the root user add the following kernel parameter settings to /etc/sysctl.conf. If any of the parameters are already in the /etc/sysctl.conf file, the higher of the 2 values should be used. kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 fs.file-max = 6553600 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048576 NOTE: The latest information on kernel parameter settings for Linux can be found in My Oracle Support ExtNote:169706.1. Run the following as the root user to allow the new kernel parameters to be put in place: 15

#/sbin/sysctl –p Repeat the above steps on all cluster nodes. NOTE: OUI checks the current settings for various kernel parameters to ensure they meet the minimum requirements for deploying Oracle RAC.

2.6

Set shell limits for the oracle user

Note:- This section can be ignored if you have setup the rpm using the previous steps (2.4) To improve the performance of the software on Linux systems, you must increase the shell limits for the oracle user 1.

Add the following lines to the /etc/security/limits.conf file:

grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 1024 grid hard nofile 65536 oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536 2.

Add or edit the following line in the /etc/pam.d/login file, if it does not already exist:

session required pam_limits.so 3. Make the following changes to the default shell startup file, add the following lines to the /etc/profile file: if [ $USER = "oracle" ] || [ $USER = "grid" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 else ulimit -u 16384 -n 65536 fi umask 022 fi For the C shell (csh or tcsh), add the following lines to the /etc/csh.login file: if ( $USER = "oracle" || $USER = "grid" ) then limit maxproc 16384 limit descriptors 65536 endif Repeat this procedure on all other nodes in the cluster.

16

2.7

Create the Oracle Inventory Directory

To create the Oracle Inventory directory, enter the following commands as the root user: # mkdir -p /u01/app/oraInventory # chown -R grid:oinstall /u01/app/oraInventory # chmod -R 775 /u01/app/oraInventory

2.7. Creating the Oracle Grid Infrastructure Home Directory To create the Grid Infrastructure home directory, enter the following commands as the root user: # mkdir -p /u01/app/12.1.0/grid # chown -R grid:oinstall //u01/app/12.1.0/grid # chmod -R 775 /u01/app/12.1.0/grid

2.8. Creating the Oracle Base Directory To create the Oracle Base directory, enter the following commands as the root user: # mkdir -p /u01/app/oracle # mkdir /u01/app/oracle/cfgtoollogs --needed to ensure that dbca is able to run after the rdbms installation. # chown -R oracle:oinstall /u01/app/oracle # chmod -R 775 /u01/app/oracle

2.9. Creating the Oracle RDBMS Home Directory To create the Oracle RDBMS Home directory, enter the following commands as the root user: # mkdir -p /u01/app/oracle/product/12.1.0/db_1 # chown -R oracle:oinstall /u01/app/oracle/product/12.1.0/db_1 # chmod -R 775 /u01/app/oracle/product/12.1.0/db_1

2.10. Stage the Oracle Software It is recommended that you stage the required software onto a local drive on Node 1 of your cluster. Starting with the first patch set for Oracle Database 12c Release 1 (12.1.0.2), Oracle Database patch sets are full installations of the Oracle Database software. In past releases, Oracle Database patch sets consisted of sets of files that replaced files in an existing Oracle home. Oracle Database 12c Release 1, patch sets are full (out-of-place) installations that replace existing installations. This simplifies the installation since you may simply install the latest patch set (version). You are no longer required to install the base release, and then apply the patch set.

17

3. Prepare the shared storage for Oracle RAC This section describes how to prepare the shared storage for Oracle RAC Each node in a cluster requires external shared disks for storing the Oracle Clusterware (Oracle Cluster Registry and voting disk) files, and Oracle Database files. To ensure high availability of Oracle Clusterware files on Oracle ASM, you need to have at least 2 GB of disk space for Oracle Clusterware files in three separate failure groups, with at least three physical disks. Each disk must have at least 1 GB capacity to ensure that there is sufficient space to create Oracle Clusterware files. Use the following guidelines when identifying appropriate disk devices: All of the devices in an Automatic Storage Management diskgroup should be the same size and have the same performance characteristics. A diskgroup should not contain more than one partition on a single physical disk device. Using logical volumes as a device in an Automatic Storage Management diskgroup is not supported with Oracle RAC. The user account with which you perform the installation (typically, 'oracle') must have write permissions to create the files in the path that you specify.

3.1. Shared Storage For this example installation we will be using ASM for Clusterware and Database storage on top of SAN technology. The following Table shows the storage layout for this implementation: 3.1.1. Partition the Shared Disks Once the LUNs have been presented from the SAN to ALL servers in the cluster, partition the LUNs from one node only, run fdisk to create a single whole-disk partition with exactly 1 MB offset on each LUN to be used as ASM Disk. Tip: From the fdisk prompt, type "u" to switch the display unit from cylinder to sector. Then create a single primary partition starting on sector 2048 (1MB offset assuming sectors of 512 bytes per unit). See below:fdisk /dev/sda Command (m for help): u Changing display/entry units to sectors Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First sector (61-1048575, default 61): 2048 Last sector or +size or +sizeM or +sizeK (2048-1048575, default 1048575): Using default value 1048575 Command (m for help): w The partition table has been altered! 18

Calling ioctl() to re-read partition table. Syncing disks. 2. Load the updated block device partition tables by running the following on ALL servers participating in the cluster: #/sbin/partprobe 3.1.2. Installing and Configuring ASMLib The ASMLib is highly recommended for those systems that will be using ASM for shared storage within the cluster due to the performance and manageability benefits that it provides. Perform the following steps to install and configure ASMLib on the cluster nodes: NOTE: ASMLib automatically provides LUN persistence, so when using ASMLib there is no need to manually configure LUN persistence for the ASM devices on the system. Download the following packages from the ASMLib OTN page, if you are an Enterprise Linux customer you can obtain the software through the Unbreakable Linux network. NOTE: The ASMLib kernel driver MUST match the kernel revision number, the kernel revision number of your system can be identified by running the "uname -r" command. Also, be sure to download the set of RPMs which pertain to your platform architecture, in our case this is x86_64. oracleasm-support-2.1.3-1.el5x86_64.rpm oracleasmlib-2.0.4-1.el5.x86_64.rpm oracleasm-2.6.18-92.1.17.0.2.el5-2.0.5-1.el5.x86_64.rpm Install the RPMs by running the following as the root user: # rpm -ivh oracleasm-support-2.1.3-1.el5x86_64.rpm \ oracleasmlib-2.0.4-1.el5.x86_64.rpm \ oracleasm-2.6.18-92.1.17.0.2.el5-2.0.5-1.el5.x86_64.rpm 3. Configure ASMLib by running the following as the root user: NOTE: If using user and group separation for the installation (as documented here), the ASMLib driver interface owner is 'grid' and the group to own the driver interface is 'asmadmin'. These groups were created in section 2.1. If a more simplistic installation using only the Oracle user is performed, the owner will be 'oracle' and the group owner will be 'dba'. #/etc/init.d/oracleasm configure Configuring the Oracle ASM library driver. This will configure the on-boot properties of the Oracle ASM library driver. The following questions will determine whether the driver is loaded on boot and what permissions it will have. The current values will be shown in brackets ('[]'). Hitting without typing an answer will keep that current value. Ctrl-C will abort. Default user to own the driver interface []: grid Default group to own the driver interface []: asmadmin Start Oracle ASM library driver on boot (y/n) [n]: y Scan for Oracle ASM disks on boot (y/n) [y]: y Writing Oracle ASM library driver configuration: done Initializing the Oracle ASMLib driver: [ OK ] Scanning the system for Oracle ASMLib disks: [ OK ] 19

4. Repeat steps 2 - 4 on ALL cluster nodes. 3.1.3. Using ASMLib to Mark the Shared Disks as Candidate Disks To create ASM disks using ASMLib: 1. As the root user, use oracleasm to create ASM disks using the following syntax: # /usr/sbin/oracleasm createdisk disk_name device_partition_name In this command, disk_name is the name you choose for the ASM disk. The name you choose must contain only ASCII capital letters, numbers, or underscores, and the disk name must start with a letter, for example,DISK1 or VOL1, or RAC_FILE1. The name of the disk partition to mark as an ASM disk is the device_partition_name. For example: [root@cehaovmsp145 ~]# /usr/sbin/oracleasm createdisk OCR_VOTE01 /dev/xvdc1 Writing disk header: done Instantiating disk: done [root@cehaovmsp145 ~]# /usr/sbin/oracleasm createdisk OCR_VOTE02 /dev/xvdd1 Writing disk header: done Instantiating disk: done [root@cehaovmsp145 ~]# /usr/sbin/oracleasm createdisk DG01 /dev/xvde1 Writing disk header: done Instantiating disk: done [root@cehaovmsp145 ~]# /usr/sbin/oracleasm createdisk DG02 /dev/xvdf1 Writing disk header: done Instantiating disk: done [root@cehaovmsp145 ~]# /usr/sbin/oracleasm createdisk DG03 /dev/xvdg1 Writing disk header: done Instantiating disk: done If you need to unmark a disk that was used in a createdisk command, you can use the following syntax as the root user: # /usr/sbin/oracleasm deletedisk disk_name 2. Repeat step 1 for each disk that will be used by Oracle ASM. 3. After you have created all the ASM disks for your cluster, use the listdisks command to verify their availability: [root@cehaovmsp145 ~]# /usr/sbin/oracleasm scandisks Reloading disk partitions: done Cleaning any stale ASM disks... Scanning system for ASM disks... [root@cehaovmsp145 ~]# /usr/sbin/oracleasm listdisks DG01 DG02 DG03 OCR_VOTE01 OCR_VOTE02 [root@cehaovmsp145 ~]#

20

4. On all the other nodes in the cluster, use the scandisks command as the root user to pickup the newly created ASM disks. You do not need to create the ASM disks on each node, only on one node in the cluster. [root@cehaovmsp146 ~]# /usr/sbin/oracleasm scandisks Reloading disk partitions: done Cleaning any stale ASM disks... Scanning system for ASM disks... Instantiating disk "OCR_VOTE01" Instantiating disk "OCR_VOTE02" Instantiating disk "DG01" Instantiating disk "DG02" Instantiating disk "DG03" 5. After scanning for ASM disks, display the available ASM disks on each node to verify their availability: [root@cehaovmsp146 ~]# /usr/sbin/oracleasm listdisks DG01 DG02 DG03 OCR_VOTE01 OCR_VOTE02 [root@cehaovmsp146 ~]#

3.2

Setting Disk I/O Scheduler on Linux

Disk I/O schedulers reorder, delay, or merge requests for disk I/O to achieve better throughput and lower latency. Linux has multiple disk I/O schedulers available, including Deadline, Noop, Anticipatory, and Completely Fair Queuing (CFQ). For best performance for Oracle ASM, Oracle recommends that you use the Deadline I/O Scheduler. Enter the following command to ensure that the Deadline disk I/O scheduler is configured for use: # echo deadline > /sys/block/${ASM_DISK}/queue/scheduler

21

4.

Oracle Grid Infrastructure Install

Basic Grid Infrastructure Install (without GNS and IPMI) As the grid user (Grid Infrastructure software owner) start the installer by running "runInstaller" from the staged installation media. NOTE: Be sure the installer is run as the intended software owner, the only supported method to change the software owner is to reinstall. #xhost + #su - grid cd into the folder where you staged the Grid Infrastructure software ./runInstaller

Action: Select radio button 'Install and Configure Grid Infrastructure for a Cluster' and click ' Next> '

22

Action: Select radio button 'Advanced Installation' and click ' Next> '

23

Action: Accept 'English' as language' and click ' Next> '

24

Action: Specify your cluster name and the SCAN name you want to use and click ' Next> ' Note: Make sure 'Configure GNS' is NOT selected.

25

Action: Use the Edit and Add buttons to specify the node names and virtual IP addresses you configured previously in your /etc/hosts file. Use the 'SSH Connectivity' button to configure/test the passwordless SSH connectivity between your nodes.

ACTION: Type in the OS password for the user 'grid' and press 'Setup'

26

After click ' OK '

Action: Click on 'Interface Type' next to the Interfaces you want to use for your cluster and select the correct values for 'Public' and 'Private' and '. When finished click ' Next> '

27

Action: Select radio button 'Automatic Storage Management (ASM) and click ' Next> '

28

Action: Select the 'DiskGroup Name' specify the 'Redundancy' and tick the disks you want to use, when done click ' Next> ' NOTE: The number of voting disks that will be created depend on the redundancy level you specify: EXTERNAL will create 1 voting disk, NORMAL will create 3 voting disks, HIGH will create 5 voting disks. NOTE: If you see an empty screen for your candidate disks it is likely that ASMLib has not been properly configured. If you are sure that ASMLib has been properly configured click on 'Change Discovery Path' and provide the correct destination.

29

Action: Specify and conform the password you want to use and click ' Next> '

30

Action: Select NOT to use IPMI and click ' Next> '

31

Action: Select if you wish to Register with EM Cloud control and click ' Next> '

32

Action: Assign the correct OS groups for OS authentication and click ' Next> '

33

Action: Specify the locations for your ORACLE_BASE and for the Software location and click ' Next> '

34

Action: Specify the locations for your Inventory directory and click ' Next> '

35

Action: Specify the required credential if you wish to automatically run configuration scripts and click 'Next> '

36

Action: Check that status of all checks is Succeeded and click ' Next> ' Note: If you have failed checks marked as 'Fixable' click 'Fix & Check again'. This will bring up the window Action: Execute the runfixup.sh script as described on the screen as root user

37

Action: Wait for the OUI to complete its tasks, After it completes the copying of binaries to all the nodes of the cluster, it will bring up a pop up window. At this point you may need to run oraInstRoot.sh on all cluster nodes (if this is the first installation of an Oracle product on this system). root.sh script output on Node 1 [root@cehaovmsp145 ~]# /u01/app/oraInventory/orainstRoot.sh Changing permissions of /u01/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall. The execution of the script is complete. [root@cehaovmsp145 ~]# /u01/app/12.1.0/grid/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/12.1.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite.

Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by

38

Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/12.1.0/grid/crs/install/crsconfig_params 2015/09/12 20:44:44 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector. 2015/09/12 20:45:19 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector. 2015/09/12 20:45:21 CLSRSC-363: User ignored prerequisites during installation OLR initialization - successful root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user cert 2015/09/12 20:46:13 CLSRSC-330: Adding Clusterware entries to file '/etc/inittab' CRS-4133: CRS-4123: CRS-4133: CRS-4123: CRS-2672: CRS-2672: CRS-2676: CRS-2676: CRS-2672: CRS-2676: CRS-2672: CRS-2672: CRS-2676: CRS-2676: CRS-2672: CRS-2672: CRS-2676: CRS-2676:

Oracle High Availability Services has been stopped. Oracle High Availability Services has been started. Oracle High Availability Services has been stopped. Oracle High Availability Services has been started. Attempting to start 'ora.evmd' on 'cehaovmsp145' Attempting to start 'ora.mdnsd' on 'cehaovmsp145' Start of 'ora.mdnsd' on 'cehaovmsp145' succeeded Start of 'ora.evmd' on 'cehaovmsp145' succeeded Attempting to start 'ora.gpnpd' on 'cehaovmsp145' Start of 'ora.gpnpd' on 'cehaovmsp145' succeeded Attempting to start 'ora.cssdmonitor' on 'cehaovmsp145' Attempting to start 'ora.gipcd' on 'cehaovmsp145' Start of 'ora.cssdmonitor' on 'cehaovmsp145' succeeded Start of 'ora.gipcd' on 'cehaovmsp145' succeeded Attempting to start 'ora.cssd' on 'cehaovmsp145' Attempting to start 'ora.diskmon' on 'cehaovmsp145' Start of 'ora.diskmon' on 'cehaovmsp145' succeeded Start of 'ora.cssd' on 'cehaovmsp145' succeeded

ASM created and started successfully. Disk Group OCRVD created successfully. CRS-2672: Attempting to start 'ora.crf' on 'cehaovmsp145' CRS-2672: Attempting to start 'ora.storage' on 'cehaovmsp145' CRS-2676: Start of 'ora.storage' on 'cehaovmsp145' succeeded CRS-2676: Start of 'ora.crf' on 'cehaovmsp145' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'cehaovmsp145' CRS-2676: Start of 'ora.crsd' on 'cehaovmsp145' succeeded CRS-4256: Updating the profile Successful addition of voting disk ae201939c23b4f12bf57fceabf2ad60f. Successfully replaced voting disk group with +OCRVD. CRS-4256: Updating the profile CRS-4266: Voting file(s) successfully replaced ## STATE File Universal Id File Name Disk group -- ----------------------------- --------1. ONLINE ae201939c23b4f12bf57fceabf2ad60f (/dev/oracleasm/disks/OCR_VOTE01) [OCRVD] Located 1 voting disk(s). CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'cehaovmsp145' CRS-2673: Attempting to stop 'ora.crsd' on 'cehaovmsp145' CRS-2677: Stop of 'ora.crsd' on 'cehaovmsp145' succeeded

39

CRS-2673: Attempting to stop 'ora.ctssd' on 'cehaovmsp145' CRS-2673: Attempting to stop 'ora.evmd' on 'cehaovmsp145' CRS-2673: Attempting to stop 'ora.storage' on 'cehaovmsp145' CRS-2673: Attempting to stop 'ora.mdnsd' on 'cehaovmsp145' CRS-2673: Attempting to stop 'ora.gpnpd' on 'cehaovmsp145' CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'cehaovmsp145' CRS-2677: Stop of 'ora.storage' on 'cehaovmsp145' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'cehaovmsp145' CRS-2677: Stop of 'ora.drivers.acfs' on 'cehaovmsp145' succeeded CRS-2677: Stop of 'ora.ctssd' on 'cehaovmsp145' succeeded CRS-2677: Stop of 'ora.evmd' on 'cehaovmsp145' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'cehaovmsp145' succeeded CRS-2677: Stop of 'ora.gpnpd' on 'cehaovmsp145' succeeded CRS-2677: Stop of 'ora.asm' on 'cehaovmsp145' succeeded CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'cehaovmsp145' CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'cehaovmsp145' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'cehaovmsp145' CRS-2677: Stop of 'ora.cssd' on 'cehaovmsp145' succeeded CRS-2673: Attempting to stop 'ora.crf' on 'cehaovmsp145' CRS-2677: Stop of 'ora.crf' on 'cehaovmsp145' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'cehaovmsp145' CRS-2677: Stop of 'ora.gipcd' on 'cehaovmsp145' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'cehaovmsp145' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Starting Oracle High Availability Services-managed resources CRS-2672: Attempting to start 'ora.mdnsd' on 'cehaovmsp145' CRS-2672: Attempting to start 'ora.evmd' on 'cehaovmsp145' CRS-2676: Start of 'ora.mdnsd' on 'cehaovmsp145' succeeded CRS-2676: Start of 'ora.evmd' on 'cehaovmsp145' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'cehaovmsp145' CRS-2676: Start of 'ora.gpnpd' on 'cehaovmsp145' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'cehaovmsp145' CRS-2676: Start of 'ora.gipcd' on 'cehaovmsp145' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'cehaovmsp145' CRS-2676: Start of 'ora.cssdmonitor' on 'cehaovmsp145' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'cehaovmsp145' CRS-2672: Attempting to start 'ora.diskmon' on 'cehaovmsp145' CRS-2676: Start of 'ora.diskmon' on 'cehaovmsp145' succeeded CRS-2676: Start of 'ora.cssd' on 'cehaovmsp145' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'cehaovmsp145' CRS-2672: Attempting to start 'ora.ctssd' on 'cehaovmsp145' CRS-2676: Start of 'ora.ctssd' on 'cehaovmsp145' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'cehaovmsp145' succeeded CRS-2672: Attempting to start 'ora.asm' on 'cehaovmsp145' CRS-2676: Start of 'ora.asm' on 'cehaovmsp145' succeeded CRS-2672: Attempting to start 'ora.storage' on 'cehaovmsp145' CRS-2676: Start of 'ora.storage' on 'cehaovmsp145' succeeded CRS-2672: Attempting to start 'ora.crf' on 'cehaovmsp145' CRS-2676: Start of 'ora.crf' on 'cehaovmsp145' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'cehaovmsp145' CRS-2676: Start of 'ora.crsd' on 'cehaovmsp145' succeeded CRS-6023: Starting Oracle Cluster Ready Services-managed resources CRS-6017: Processing resource auto-start for servers: cehaovmsp145 CRS-6016: Resource auto-start has completed for server cehaovmsp145 CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources CRS-4123: Oracle High Availability Services has been started. 2015/09/12 20:51:49 CLSRSC-343: Successfully started Oracle Clusterware stack CRS-2672: CRS-2676: CRS-2672: CRS-2676:

Attempting to start 'ora.asm' on 'cehaovmsp145' Start of 'ora.asm' on 'cehaovmsp145' succeeded Attempting to start 'ora.OCRVD.dg' on 'cehaovmsp145' Start of 'ora.OCRVD.dg' on 'cehaovmsp145' succeeded

2015/09/12 20:53:00 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

root.sh script output on Node 2 [root@cehaovmsp146 ~]# /u01/app/oraInventory/orainstRoot.sh Changing permissions of /u01/app/oraInventory. Adding read,write permissions for group. Removing read,write,execute permissions for world. Changing groupname of /u01/app/oraInventory to oinstall. The execution of the script is complete.

40

[root@cehaovmsp146 ~]# /u01/app/12.1.0/grid/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/12.1.0/grid Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed. Using configuration parameter file: /u01/app/12.1.0/grid/crs/install/crsconfig_params 2015/09/12 21:02:53 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector. 2015/09/12 21:03:23 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector. 2015/09/12 21:03:25 CLSRSC-363: User ignored prerequisites during installation OLR initialization - successful 2015/09/12 21:05:12 CLSRSC-330: Adding Clusterware entries to file '/etc/inittab' CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Oracle High Availability Services has been started. CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'cehaovmsp146' CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'cehaovmsp146' CRS-2677: Stop of 'ora.drivers.acfs' on 'cehaovmsp146' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'cehaovmsp146' has completed CRS-4133: Oracle High Availability Services has been stopped. CRS-4123: Starting Oracle High Availability Services-managed resources CRS-2672: Attempting to start 'ora.mdnsd' on 'cehaovmsp146' CRS-2672: Attempting to start 'ora.evmd' on 'cehaovmsp146' CRS-2676: Start of 'ora.mdnsd' on 'cehaovmsp146' succeeded CRS-2676: Start of 'ora.evmd' on 'cehaovmsp146' succeeded CRS-2672: Attempting to start 'ora.gpnpd' on 'cehaovmsp146' CRS-2676: Start of 'ora.gpnpd' on 'cehaovmsp146' succeeded CRS-2672: Attempting to start 'ora.gipcd' on 'cehaovmsp146' CRS-2676: Start of 'ora.gipcd' on 'cehaovmsp146' succeeded CRS-2672: Attempting to start 'ora.cssdmonitor' on 'cehaovmsp146' CRS-2676: Start of 'ora.cssdmonitor' on 'cehaovmsp146' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'cehaovmsp146' CRS-2672: Attempting to start 'ora.diskmon' on 'cehaovmsp146' CRS-2676: Start of 'ora.diskmon' on 'cehaovmsp146' succeeded CRS-2676: Start of 'ora.cssd' on 'cehaovmsp146' succeeded CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'cehaovmsp146' CRS-2672: Attempting to start 'ora.ctssd' on 'cehaovmsp146' CRS-2676: Start of 'ora.ctssd' on 'cehaovmsp146' succeeded CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'cehaovmsp146' succeeded CRS-2672: Attempting to start 'ora.asm' on 'cehaovmsp146' CRS-2676: Start of 'ora.asm' on 'cehaovmsp146' succeeded CRS-2672: Attempting to start 'ora.storage' on 'cehaovmsp146' CRS-2676: Start of 'ora.storage' on 'cehaovmsp146' succeeded CRS-2672: Attempting to start 'ora.crf' on 'cehaovmsp146' CRS-2676: Start of 'ora.crf' on 'cehaovmsp146' succeeded CRS-2672: Attempting to start 'ora.crsd' on 'cehaovmsp146' CRS-2676: Start of 'ora.crsd' on 'cehaovmsp146' succeeded CRS-6017: Processing resource auto-start for servers: cehaovmsp146 CRS-2672: Attempting to start 'ora.net1.network' on 'cehaovmsp146' CRS-2676: Start of 'ora.net1.network' on 'cehaovmsp146' succeeded CRS-2672: Attempting to start 'ora.ons' on 'cehaovmsp146' CRS-2676: Start of 'ora.ons' on 'cehaovmsp146' succeeded CRS-6016: Resource auto-start has completed for server cehaovmsp146 CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources CRS-4123: Oracle High Availability Services has been started. 2015/09/12 21:08:43 CLSRSC-343: Successfully started Oracle Clusterware stack 2015/09/12 21:09:00 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

41

Action: Wait for the OUI to finish the cluster configuration.

42

Action: You should see the confirmation that installation of the Grid Infrastructure was successful. Click 'Close' to finish the install. [root@cehaovmsp145 ~]# crsctl stat res -t -------------------------------------------------------------------------------Name Target State Server State details -------------------------------------------------------------------------------Local Resources -------------------------------------------------------------------------------ora.LISTENER.lsnr ONLINE ONLINE cehaovmsp145 STABLE ONLINE ONLINE cehaovmsp146 STABLE ora.OCRVD.dg ONLINE ONLINE cehaovmsp145 STABLE ONLINE ONLINE cehaovmsp146 STABLE ora.asm ONLINE ONLINE cehaovmsp145 STABLE ONLINE ONLINE cehaovmsp146 Started,STABLE ora.net1.network ONLINE ONLINE cehaovmsp145 STABLE ONLINE ONLINE cehaovmsp146 STABLE ora.ons ONLINE ONLINE cehaovmsp145 STABLE ONLINE ONLINE cehaovmsp146 STABLE

43

-------------------------------------------------------------------------------Cluster Resources -------------------------------------------------------------------------------ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE cehaovmsp145 STABLE ora.MGMTLSNR 1 ONLINE ONLINE cehaovmsp145 169.254.41.177 10.64 .131.119,STABLE ora.cehaovmsp145.vip 1 ONLINE ONLINE cehaovmsp145 STABLE ora.cehaovmsp146.vip 1 ONLINE ONLINE cehaovmsp146 STABLE ora.cvu 1 ONLINE ONLINE cehaovmsp145 STABLE ora.mgmtdb 1 ONLINE ONLINE cehaovmsp145 Open,STABLE ora.oc4j 1 ONLINE ONLINE cehaovmsp145 STABLE ora.scan1.vip 1 ONLINE ONLINE cehaovmsp145 STABLE -------------------------------------------------------------------------------[root@cehaovmsp145 ~]# ocrcheck Status of Oracle Cluster Registry is as follows : Version : 4 Total space (kbytes) : 409568 Used space (kbytes) : 1456 Available space (kbytes) : 408112 ID : 1201040793 Device/File Name : +OCRVD Device/File integrity check succeeded Device/File not configured Device/File not configured Device/File not configured Device/File not configured Cluster registry integrity check succeeded Logical corruption check succeeded [root@cehaovmsp145 ~]# crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----------------------------- --------1. ONLINE ae201939c23b4f12bf57fceabf2ad60f (/dev/oracleasm/disks/OCR_VOTE01) [OCRVD] Located 1 voting disk(s). [root@cehaovmsp145 ~]# crsctl check cluster -all ************************************************************** cehaovmsp145: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** cehaovmsp146: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online **************************************************************

44

5.

RDBMS Software Install

As the oracle user (rdbms software owner) start the installer by running "runInstaller" from the staged installation media. NOTE: Be sure the installer is run as the intended software owner, the only supported method to change the software owner is to reinstall. Change into the directory where you staged the RDBMS software ./runInstaller

Action: Provide your e-mail address, tick the check box and provide your Oracle Support Password if you want to receive Security Updates from Oracle Support and click ' Next> '

45

Action: Select the option 'Install Database software only' and click ' Next> '

46

Action: Select the option Oracle Real Application Clusters database Installation' and click ' Next> '

47

Action: Select all nodes. Use the 'SSH Connectivity' button to configure/test the passwordless SSH connectivity between your nodes ' Type in the OS password for the oracle user and click 'Setup'

48

Action: To confirm English as selected language click ' Next> '

49

Action: Make sure radio button 'Enterprise Edition' is ticked, click ' Next> '

50

Action: Specify path to your Oracle Base and below to the location where you want to store the software (Oracle home). Click ' Next> '

51

Action: Use the drop down menu to select the names of the Database Administrators and Database Operators group and click ' Next> '

52

Action: Check that the status of all checks is 'Succeeded' and click ' Next> ' Note: If you are sure the unsuccessful checks can be ignored tick the box 'Ignore All' before you click ' Next> '

53

Action: Perform last check that the information on the screen is correct before you click ' Finish '

54

Action: Log in to a terminal window as root user and run the root.sh script on the first node. When finished do the same for all other nodes in your cluster as well. When finished click 'OK' NOTE: root.sh should be run on one node at a time. [root@cehaovmsp145 ~]# /u01/app/oracle/product/12.1.0/db_1/root.sh Performing root user operation.

The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME=

/u01/app/oracle/product/12.1.0/db_1

Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed.

[root@cehaovmsp146 ~]# /u01/app/oracle/product/12.1.0/db_1/root.sh Performing root user operation. The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME=

/u01/app/oracle/product/12.1.0/db_1

Enter the full pathname of the local bin directory: [/usr/local/bin]: The contents of "dbhome" have not changed. No need to overwrite. The contents of "oraenv" have not changed. No need to overwrite. The contents of "coraenv" have not changed. No need to overwrite. Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root script. Now product-specific root actions will be performed.

55

Action: Click ' Close ' to finish the installation of the RDBMS Software.

56

6.

ASMCA to create Diskgroups

As the grid user start the ASM Configuration Assistant (ASMCA) $ cd /u01/app/12.1.0/grid/bin ./asmca

Action: Click 'Create' to create a new diskgroup

57

Type in a name for the diskgroup, select the redundancy you want to provide and mark the tick box for the disks you want to assign to the new diskgroup.

58

Action: Click 'OK' Note: It is Oracle's Best Practice to have an OCR mirror stored in a second diskgroup. To follow these recommendations add an OCR mirror. Mind that you can only have one OCR in a diskgroup. Action: 1. To add OCR mirror to an Oracle ASM diskgroup, ensure that the Oracle Clusterware stack is running and run the following command as root from the $GRID_HOME /bin directory: 2. # ocrconfig -add +DATA 3. # ocrcheck

59

7.

Run DBCA to create the database

As the oracle user start the Database Configuration Assistant (DBCA) cd /u01/app/oracle/product/12.1.0/db_1/bin ./dbca

Action: Choose option 'Create a Database' and click 'Next'

60

Action: Choose option 'Advanced Mode' and click 'Next >'

61

Action: Select 'Oracle Real Application Clusters (RAC) database' and ‘Admin Managed’ click 'Next'

62

Action: Type in the name you want to use for your database and select “Create As Container Database” click 'Next >'

63

Action: Select all nodes before you click 'Next>'

64

Action: Select the options you want to use to manage your database and click 'Next'

65

Action: Type in the passwords you want to use and click 'Next'

66

Action: Select the diskgroup you created for the database files, optionally select FRA & Enable Archiving if you wish to configure. Click ‘Next >’

67

Action: Select Appropriate option, if you wish to configure Database Vault & Label Security click 'Next >'

68

Action: Review and change the settings for memory allocation, character sets etc. according to your needs and click 'Next >'

69

Action: Make sure the tick box 'Create Database' is ticked and Optionally you mayselct “Generate Database Creation Scripts” and click 'Next >'

70

Action: Review the validation Results, if you are convinced you can go with “Ignore All” option, Click ‘Next >’

71

Action: Review the database configuration details again and click 'Finish >'

72

Action: The database is now created, you can either change or unlock your passwords or just click Exit to finish the database creation. 73

8.

Applying Latest PSUs to GRID & RDBMS Homes

Please download the latest patch from MOS Doc ID 756671.1. When the document was written latest PSU release was:Patch 20996835: GRID INFRASTRUCTURE PATCH SET UPDATE 12.1.0.2.4 (JUL2015) The above patch contains both GI & RDBMS Patch and can be applied on a rolling manner. Please note:1. Make sure that there are no active sqlplus sessions opened through putty.(ps -ef | grep sqlplus) 2. Make sure that emagent / em dbcontrol is stopped manually before you execute the patch steps (ps -ef | grep em) Steps for applying the patch ======================= 1. Download the latest opatch (6880880) utility and replace the existing OPatch with the new one (it will not allow you to replace, so you have to move with root user and change the ownership to grid:oinstall) https://updates.oracle.com/download/6880880.html 2. Create the OCM response file for silent installation of "opatch auto" features as a root user. $ $GRID_HOME/OPatch/ocm/bin/emocmrsp -no_banner -output /tmp/ocmconf.rsp Provide your email address to be informed of security issues, install and initiate Oracle Configuration Manager. Easier for you if you use your My Oracle Support Email address/User Name. Visit http://www.oracle.com/support/policies.html for details. Email address/User Name: