This chapter describes the steps to install Sun Cluster Support for Oracle Real Application Clusters on your Sun Cluster nodes. This chapter contains the following procedures.
How to Create a Node-Specific Directory for a Shared File System
How to Install Sun Cluster Support for Oracle Real Application Clusters Packages
How to Create the Database Administrator Group and the Oracle User Account
How to Create a VxVM Shared-Disk Group for the Oracle Real Application Clusters Database
The following table summarizes the installation tasks and provides cross-references to detailed instructions for performing the tasks.
Perform these tasks in the order in which they are listed in the table.
Table 1–1 Tasks for Installing Sun Cluster Support for Oracle Real Application Clusters
Task |
Instructions |
---|---|
Understand preinstallation considerations and special requirements | |
Install storage management software |
Installing Storage Management Software With Sun Cluster Support for Oracle Real Application Clusters |
Create node-specific files and directories that the Sun Cluster Support for Oracle Real Application Clusters software requires |
Creating Node-Specific Files and Directories for a Shared File System |
Install data service packages |
Installing Sun Cluster Support for Oracle Real Application Clusters Packages |
Prepare the Sun Cluster nodes | |
Install the UNIX Distributed Lock Manager | |
Create a multi-owner disk set in Solaris Volume Manager for Sun Cluster for the Oracle Real Application Clusters database | |
Create a VxVM shared-disk group for the Oracle Real Application Clusters database |
Creating a VxVM Shared-Disk Group for the Oracle Real Application Clusters Database |
Oracle Real Application Clusters is a scalable application that can run on more than one node concurrently. Sun Cluster Support for Oracle Real Application Clusters is a set of packages that, when installed, enables Oracle Real Application Clusters to run on Sun Cluster nodes. This data service also enables Oracle Real Application Clusters to be managed by using Sun Cluster commands.
In earlier versions of Oracle, this scalable application is referred to as “Oracle Parallel Server”. In this book, references to “Oracle Real Application Clusters” also apply to Oracle Parallel Server unless this book explicitly states otherwise.
This data service provides fault monitoring only to enable the status of Oracle Real Application Clusters resources to be monitored by Sun Cluster utilities. This data service does not provide automatic fault recovery because the Oracle Real Application Clusters software provides similar functionality.
Before you begin the installation, note the hardware and software requirements in the subsections that follow.
Sun Cluster Support for Oracle Real Application Clusters requires a functioning cluster with the initial cluster framework already installed. See Sun Cluster Software Installation Guide for Solaris OS for details about initial installation of cluster software.
You must configure Oracle Real Application Clusters to use the shared-disk architecture of the Sun Cluster software. In this configuration, a single database is shared among multiple instances of the Oracle Real Application Clusters software that access the database concurrently. The UNIX Distributed Lock Manager (Oracle UDLM) controls access to shared resources between cluster nodes.
To satisfy these requirements, use one storage management scheme from the following list:
Solaris Volume Manager for Sun Cluster
Solaris Volume Manager for Sun Cluster is supported only with Oracle Real Application Clusters. Solaris Volume Manager for Sun Cluster with Oracle Parallel Server is not supported.
VERITAS Volume Manager (VxVM) with the cluster feature
Hardware redundant array of independent disks (RAID) support
Sun StorEdgeTM QFS shared file system with hardware RAID support
Verify that you have obtained and installed the appropriate licenses for your software. If you install your licenses incorrectly or incompletely, the nodes might fail to boot correctly.
For example, if you are using VxVM with the cluster feature, verify that you have installed a valid license for the Volume Manager cluster feature by running one of the following commands:
Check with a Sun Enterprise Services representative for the current supported topologies for Sun Cluster Support for Oracle Real Application Clusters, cluster interconnect, storage management scheme, and hardware configurations.
Ensure that you have installed all of the applicable software patches for the Solaris Operating System, Sun Cluster, Oracle, and your volume manager. If you need to install any Sun Cluster Support for Oracle Real Application Clusters patches, you must apply these patches after you install the data service packages.
You can install the Oracle binary files and Oracle configuration files on one of the following locations.
The local disks of each cluster node
A shared file system from the following list:
The Sun StorEdge QFS shared file system
The cluster file system
Placing the Oracle binary files and Oracle configuration files on the individual cluster nodes enables you to upgrade the Oracle application later without shutting down the data service.
The disadvantage is that you then have several copies of the Oracle application binary files and Oracle configuration files to maintain and administer.
To simplify the maintenance of your Oracle installation, you can install the Oracle binary files and Oracle configuration files on a shared file system. The following shared file systems are supported:
The Sun StorEdge QFS shared file system
The cluster file system
If you use the cluster file system, decide which volume manager to use:
Solaris Volume Manager
VxVM without the cluster feature
If you put the Oracle binary files and Oracle configuration files on a shared file system, you have only one copy to maintain and manage. However, you must shut down the data service in the entire cluster to upgrade the Oracle application. If a small amount of downtime for upgrades is acceptable, place a single copy of the Oracle binary files and Oracle configuration files on a shared file system.
You can store all of the files that are associated with Oracle Real Application Clusters on the Sun StorEdge QFS shared file system.
Distribute these files among several file systems as follows:
Create one file system in the cluster to store these files:
Oracle binary files
Oracle configuration files (for example, init.ora, tnsnames.ora, listener.ora, and sqlnet.ora)
Alert files (for example, alert_sid.log)
Trace files (*.trc)
Create one file system for each database to store these files for all Oracle Real Application Clusters instances of the database:
Data files
Control files
Online redo log files
Archived redo log files
For information about how to create a Sun StorEdge QFS shared file system, see the following documentation for Sun StorEdge QFS:
Sun StorEdge QFS and Sun StorEdge SAM-FS Software Installation and Configuration Guide
Sun StorEdge QFS and Sun StorEdge SAM-FS File System Administration Guide
You can store only these files that are associated with Oracle Real Application Clusters on the cluster file system:
Oracle binary files
Oracle configuration files (for example, init.ora, tnsnames.ora, listener.ora, and sqlnet.ora)
Archived redo log files
Alert files (for example, alert_sid.log)
Trace files (*.trc)
You must not store data files, control files, or online redo log files on the cluster file system.
The input/output (I/O) performance during the writing of archived redo log files is affected by the location of the device group for archived redo log files. For optimum performance, ensure that the primary of the device group for archived redo log files is located on the same node as the Oracle Real Application Clusters database instance. This device group contains the file system that holds archived redo log files of the database instance.
For information about how to create cluster file systems, see:
Use the questions in the subsections that follow to plan the installation and configuration of Sun Cluster Support for Oracle Real Application Clusters. Write the answers to these questions in the space that is provided on the data service worksheets in “Configuration Worksheets” in Sun Cluster 3.1 Data Service Planning and Administration Guide.
Which resource groups will you use for the Oracle Real Application Clusters (RAC) server resources?
You require one resource group for each Oracle Real Application Clusters database instance. Each resource group contains the Oracle RAC server resource for the database instance.
Use the answer to this question when you perform the procedure in Registering and Configuring Oracle RAC Server Resources.
Which resource groups will you use for the Oracle listener resources?
Use the answer to this question when you perform the procedure in Registering and Configuring Oracle Listener Resources.
The resource groups depend on your configuration of Oracle listeners with Real Application Clusters database instances. For general information about possible configurations of listeners for Real Application Clusters instances, see your Oracle documentation. Example configurations are described in the subsections that follow.
One listener serves only one Real Application Clusters instance. The listener listens on the fixed Internet Protocol (IP) address of the node. The listener cannot fail over.
In this situation, configure the listener resource as follows:
Configure the listener resource and the RAC server resource in the same resource group.
Ensure that this resource group is mastered on only one node.
One listener serves several Real Application Clusters instances on the same node. The listener uses Oracle's transparent application failover (TAF) and load balancing to distribute client connections across all Real Application Clusters instances. The listener cannot fail over.
In this situation, configure the listener resource as follows:
Configure the listener resource in its own resource group.
Ensure that the listener's resource group is mastered on only one node.
Create a dependency between the listener's resource group and RAC servers' resource groups.
One listener that can fail over serves several Real Application Clusters instances on the same node. When the listener fails over to another node, the listener serves several Real Application Clusters instances on the other node.
The listener uses Oracle's TAF and load balancing to distribute client connections across all Real Application Clusters instances. To ensure fast error detection and short failover times, the listener listens on an address that is represented by a LogicalHostname resource.
In this situation, configure the listener resource as follows:
Configure the listener resource and the LogicalHostname resource in the same resource group.
Ensure that this resource group is mastered on the nodes on which Oracle Real Application Clusters is running.
For more information, see LogicalHostname Resources for Oracle Listener Resources.
One listener serves all Real Application Clusters instances on all nodes. The listener listens on an address that is represented by a LogicalHostname resource. This configuration ensures that the address is plumbed very quickly on another node after a node fails.
You can use this configuration if you configure Real Application Clusters instances to use a multithreaded server (MTS). In such a configuration, the REMOTE_LISTENERS parameter in the init.ora file specifies that each dispatcher registers with the listener on a logical IP address.
All clients connect through the one listener. The listener redirects each client connection to the least busy dispatcher. The least busy dispatcher might be on a different node from the listener.
If the listener fails, the listener's fault monitor restarts the listener. If the node where the listener is running fails, the listener is restarted on a different node. In both situations the dispatchers reregister after the listener is restarted.
If you are using one listener for the entire cluster, configure the following resources in the same resource group:
The listener resource
The LogicalHostname resource
For more information, see LogicalHostname Resources for Oracle Listener Resources.
Which LogicalHostname resources will Oracle listener resources use?
Use the answer to this question when you perform the procedure in Registering and Configuring Oracle Listener Resources.
If a cluster node that is running an instance of Oracle Real Application Clusters fails, an operation that a client application attempted might be required to time out before the operation is attempted again on another instance. If the Transmission Control Protocol/Internet Protocol (TCP/IP) network timeout is high, the client application might require a significant length of time to detect the failure. Typically, client applications require between three and nine minutes to detect such failures.
In such situations, client applications can connect to listener resources that are listening on an address that is represented by the Sun Cluster LogicalHostname resource. Configure the LogicalHostname resource and the listener resource in a separate resource group. Ensure that this resource group is mastered on the nodes on which Oracle Real Application Clusters is running. If a node fails, the resource group that contains the LogicalHostname resource and the listener resource fails over to another surviving node on which Oracle Real Application Clusters is running. The failover of the LogicalHostname resource enables new connections to be directed to the other instance of Oracle Real Application Clusters.
If you are using the Sun StorEdge QFS shared file system, answer the following questions:
Which resources will you create to represent the metadata server for the Sun StorEdge QFS shared file system?
One resource is required for each Sun StorEdge QFS metadata server.
Which resource groups will you use for these resources?
For more information, see the following documentation for Sun StorEdge QFS:
Sun StorEdge QFS and Sun StorEdge SAM-FS Software Installation and Configuration Guide
Sun StorEdge QFS and Sun StorEdge SAM-FS File System Administration Guide
Use the answers to these questions when you perform the procedure in Registering and Configuring Oracle RAC Server Resources.
Where will the system configuration files reside?
For the advantages and disadvantages of using the local file system instead of the cluster file system, see Location of Oracle Binary Files and Oracle Configuration Files.
This section lists special requirements for Sun Cluster Support for Oracle Real Application Clusters.
Before you decide which architecture to use for the Oracle UDLM and the Oracle relational database management system (RDBMS), note the following points.
The architecture of both Oracle components must match. For example, if you have 64-bit architecture for your Oracle UDLM, you must have 64-bit architecture for your RDBMS.
If you have 32-bit architecture for your Oracle components, you can boot the node on which the components reside in either 32-bit mode or 64-bit mode. However, if you have 64-bit architecture for your Oracle components, you must boot the node on which the components reside in 64-bit mode.
You must use the same architecture when you boot all of the nodes. For example, if you boot one node to use 32-bit architecture, you must boot all of the nodes to use 32-bit architecture.
The following list shows the locations of the data service log files.
Current log: /var/cluster/ucmm/ucmm_reconf.log
Previous logs: /var/cluster/ucmm/ucmm_reconf.log.0 (0,1,...) – This location is dependent on the Oracle UDLM package.
Oracle UDLM logs: /var/cluster/ucmm/dlm_nodename/logs – If you cannot find the Oracle log files at this location, contact Oracle support.
Oracle UDLM core files: /var/cluster/ucmm/dlm_nodename/cores – If you cannot find the Oracle log files at this location, contact Oracle support.
Logs for Oracle RAC server resource: /var/opt/SUNWscor/oracle_server
During installation of this data service, reboot the nodes only after you have installed and configured the Oracle UDLM software, and satisfied the prerequisites for performing this task. Otherwise, the nodes panic.
For information about how to recover from a node panic during installation, see Node Panic During Initialization of Sun Cluster Support for Oracle Real Application Clusters.
For information about the installation, administration, and operation of the Oracle Real Application Clusters Guard option, see the Oracle documentation. If you plan to use this product option with Sun Cluster 3.1, note the points in the subsections that follow before you install Sun Cluster 3.1.
If you use the Oracle Real Application Clusters Guard option with Sun Cluster 3.1, the following restrictions apply to hostnames that you use in your cluster:
Hostnames cannot contain special characters.
You cannot change the hostnames after you install Sun Cluster 3.1.
For more information about these restrictions and any other requirements, see the Oracle documentation.
If you use the Oracle Real Application Clusters Guard option with Sun Cluster 3.1, do not use Sun Cluster commands to perform the following operations:
Manipulating the state of resources that Oracle Real Application Clusters Guard installs. Using Sun Cluster commands for this purpose might cause failures.
Querying the state of the resources that Oracle Real Application Clusters Guard installs. This state might not reflect the actual state. To check the state of the Oracle Real Application Clusters Guard, use the commands that Oracle supplies.
Install the software for the storage management schemes that you are using for the Oracle Real Application Clusters database and the Oracle software.
For information about storage management schemes for the database, see Storage Management Requirements for the Oracle Real Application Clusters Database.
For information about storage management schemes for the Oracle software, see Location of Oracle Binary Files and Oracle Configuration Files.
To use the Solaris Volume Manager for Sun Cluster software with Sun Cluster Support for Oracle Real Application Clusters, perform the following tasks.
Ensure that you are using Solaris 9 9/04 or compatible versions.
Solaris Volume Manager for Sun Cluster is installed during the installation of the Solaris Operating System.
Configure the Solaris Volume Manager for Sun Cluster software on the cluster nodes.
For more information, see “Installing and Configuring Solstice DiskSuite/Solaris Volume Manager Software” in Sun Cluster Software Installation Guide for Solaris OS.
Go to Installing Sun Cluster Support for Oracle Real Application Clusters Packages to install the Sun Cluster Support for Oracle Real Application Clusters software packages.
To use the VxVM software with Sun Cluster Support for Oracle Real Application Clusters, perform the following tasks.
If you are using VxVM with the cluster feature, obtain a license for the Volume Manager cluster feature in addition to the basic VxVM license.
See your VxVM documentation for more information about VxVM licensing requirements.
Failure to correctly install the license for the Volume Manager cluster feature might cause a panic when you install Oracle Real Application Clusters support. Before you install the Oracle Real Application Clusters packages, run the vxlicense -p or vxlicrep command to ensure that you have installed a valid license for the Volume Manager cluster feature.
Install and configure the VxVM software on the cluster nodes.
See “Installing and Configuring VERITAS Volume Manager” in Sun Cluster Software Installation Guide for Solaris OS and the VxVM documentation for more information.
Go to Installing Sun Cluster Support for Oracle Real Application Clusters Packages to install the Sun Cluster Support for Oracle Real Application Clusters software packages.
You can use Sun Cluster Support for Oracle Real Application Clusters with hardware RAID support.
For example, you can use Sun StorEdgeTM A3500/A3500FC disk arrays with hardware RAID support and without VxVM software. To use this combination, configure raw device identities (/dev/did/rdsk*) on top of the disk arrays' logical unit numbers (LUNs). To set up the raw devices for Oracle Real Application Clusters on a cluster that uses StorEdge A3500/A3500FC disk arrays with hardware RAID, perform the following steps.
Create LUNs on the disk arrays.
See the Sun Cluster hardware documentation for information about how to create LUNs.
After you create the LUNs, run the format(1M) command to partition the disk arrays' LUNs into as many slices as you need.
The following example lists output from the format command.
# format 0. c0t2d0 <SUN18G cyl 7506 alt 2 hd 19 sec 248> /sbus@3,0/SUNW,fas@3,8800000/sd@2,0 1. c0t3d0 <SUN18G cyl 7506 alt 2 hd 19 sec 248> /sbus@3,0/SUNW,fas@3,8800000/sd@3,0 2. c1t5d0 <Symbios-StorEDGEA3000-0301 cyl 21541 alt 2 hd 64 sec 64> /pseudo/rdnexus@1/rdriver@5,0 3. c1t5d1 <Symbios-StorEDGEA3000-0301 cyl 21541 alt 2 hd 64 sec 64> /pseudo/rdnexus@1/rdriver@5,1 4. c2t5d0 <Symbios-StorEDGEA3000-0301 cyl 21541 alt 2 hd 64 sec 64> /pseudo/rdnexus@2/rdriver@5,0 5. c2t5d1 <Symbios-StorEDGEA3000-0301 cyl 21541 alt 2 hd 64 sec 64> /pseudo/rdnexus@2/rdriver@5,1 6. c3t4d2 <Symbios-StorEDGEA3000-0301 cyl 21541 alt 2 hd 64 sec 64> /pseudo/rdnexus@3/rdriver@4,2 |
To prevent a loss of disk partition information, do not start the partition at cylinder 0 for any disk slice that is used for raw data. The disk partition table is stored in cylinder 0 of the disk.
Run the scdidadm(1M) command to find the raw device identity (DID) that corresponds to the LUNs that you created in Step 1.
The following example lists output from the scdidadm -L command.
# scdidadm -L 1 phys-schost-1:/dev/rdsk/c0t2d0 /dev/did/rdsk/d1 1 phys-schost-2:/dev/rdsk/c0t2d0 /dev/did/rdsk/d1 2 phys-schost-1:/dev/rdsk/c0t3d0 /dev/did/rdsk/d2 2 phys-schost-2:/dev/rdsk/c0t3d0 /dev/did/rdsk/d2 3 phys-schost-2:/dev/rdsk/c4t4d0 /dev/did/rdsk/d3 3 phys-schost-1:/dev/rdsk/c1t5d0 /dev/did/rdsk/d3 4 phys-schost-2:/dev/rdsk/c3t5d0 /dev/did/rdsk/d4 4 phys-schost-1:/dev/rdsk/c2t5d0 /dev/did/rdsk/d4 5 phys-schost-2:/dev/rdsk/c4t4d1 /dev/did/rdsk/d5 5 phys-schost-1:/dev/rdsk/c1t5d1 /dev/did/rdsk/d5 6 phys-schost-2:/dev/rdsk/c3t5d1 /dev/did/rdsk/d6 6 phys-schost-1:/dev/rdsk/c2t5d1 /dev/did/rdsk/d6 |
Use the DID that the scdidadm output identifies to set up the raw devices.
For example, the scdidadm output might identify that the raw DID that corresponds to the disk arrays' LUNs is d4. In this instance, use the /dev/did/rdsk/d4sN raw device, where N is the slice number.
Go to Installing Sun Cluster Support for Oracle Real Application Clusters Packages to install the Sun Cluster Support for Oracle Real Application Clusters software packages.
You must use Sun StorEdge QFS shared file system with hardware RAID support.
For detailed instructions for installing, configuring, and using Sun StorEdge QFS shared file system, see Sun StorEdge QFS and Sun StorEdge SAM-FS Software Installation and Configuration Guide and Sun StorEdge QFS and Sun StorEdge SAM-FS File System Administration Guide.
Ensure that the Sun StorEdge QFS software is installed.
Ensure that each Sun StorEdge QFS shared file system is correctly configured for use with Sun Cluster Support for Oracle Real Application Clusters.
Ensure that each Sun StorEdge QFS shared file system is mounted with the correct options for use with Sun Cluster Support for Oracle Real Application Clusters.
For the file system that contains binary files, configuration files, alert files, and trace files, use the default mount options.
For the file systems that contain data files, control files, online redo log files, and archived redo log files, set the mount options as follows:
In the /etc/opt/SUNWsamfs/samfs.cmd file or the /etc/vfstab file, set the following options:
stripe=width sync_meta=1 mh_write qwrite forcedirectio nstreams=1024 rdlease=300Set this value for optimum performance. wrlease=300Set this value for optimum performance. aplease=300Set this value for optimum performance.
Ensure that settings in the /etc/vfstab file do not conflict with settings in the /etc/opt/SUNWsamfs/samfs.cmd file. Settings in the /etc/vfstab file override settings in the /etc/opt/SUNWsamfs/samfs.cmd file.
Register and configure the data service for the Sun StorEdge QFS metadata server.
The next step depends on whether you are using the Sun StorEdge QFS file system for Oracle binary files and Oracle configuration files.
If you are using the Sun StorEdge QFS file system for Oracle binary files and Oracle configuration files, go to Creating Node-Specific Files and Directories for a Shared File System.
Otherwise, go to Installing Sun Cluster Support for Oracle Real Application Clusters Packages.
Create and mount the cluster file system.
See “Configuring the Cluster” in Sun Cluster Software Installation Guide for Solaris OS for information about how to create and mount the cluster file system.
If you are using the UNIX file system (UFS), ensure that you specify the correct mount options for various types of Oracle files.
For the correct options, see the table that follows. You set these options when you add an entry to the /etc/vfstab file for the mount point.
File Type |
Options |
---|---|
global, logging, forcedirectio |
|
Oracle application binary files, configuration files, alert files, and trace files |
global, logging |
Go to Creating Node-Specific Files and Directories for a Shared File System to create node-specific files and directories that the Sun Cluster Support for Oracle Real Application Clusters software requires.
To simplify the maintenance of your Oracle installation, you can install the Oracle binary files and Oracle configuration files on a shared file system. The following shared file systems are supported:
The Sun StorEdge QFS shared file system
The cluster file system
When Oracle software is installed on a shared file system, all the files
in the directory that the ORACLE_HOME
environment variable specifies are accessible by all cluster nodes. However,
some Oracle files and directories must maintain node-specific information.
If you install Oracle software on a shared file system, you must create local copies of files and directories that must maintain node-specific information. To ensure that these files and directories are accessible by all cluster nodes, use a symbolic link whose target is a file or a directory on a file system that is local to a node. Such a file system is not part of the shared file system.
To use a symbolic link for this purpose, you must allocate an area on a local file system. To enable Oracle applications to create symbolic links to files in this area, the applications must be able to access files in this area. Because the symbolic links reside on the shared file system, all references to the links from all nodes are the same. Therefore, all nodes must have the same namespace for the area on the local file system.
Perform this procedure for each directory that is to maintain node-specific information. The following directories are typically required to maintain node-specific information:
For information about other directories that might be required to maintain node-specific information, see your Oracle documentation.
On each cluster node, create the local directory that is to maintain node-specific information.
Ensure that the local directory structure that you create matches the global directory structure that contains the node-specific information. For example, the global directory /global/oracle/network/agent might contain node-specific information that you require to be stored locally under the /local directory. In this situation, you would create a directory that is named /local/oracle/network/agent.
# mkdir -p local-dir |
Specifies that all nonexistent parent directories are created first
Specifies the full path name of the directory that you are creating
On each cluster node, make a local copy of the global directory that is to maintain node-specific information.
Ensure that the local copy of the node-specific information is contained in the local directory that you created in Step 1.
# cp -pr global-dir local-dir-parent |
Specifies that the owner, group, permissions modes, modification time, access time, and access control lists are preserved.
Specifies that the directory and all its files, including any subdirectories and their files, are copied.
Specifies the full path of the global directory that you are copying.
This directory resides on the shared file system under the directory that
the ORACLE_HOME
environment variable
specifies.
Specifies the directory on the local node that is to contain the local copy. This directory is the parent directory of the directory that you created in Step 1.
Replace the global directory that you copied in Step 2 with a symbolic link to the local copy of the global directory.
From any cluster node, remove the global directory that you copied in Step 2.
# rm -r global-dir |
Specifies that the directory and all its files, including any subdirectories and their files, are removed.
Specifies the file name and full path of the global directory that you are removing. This directory is the global directory that you copied in Step 2.
From any cluster node, create a symbolic link from the local copy of the directory to the global directory that you removed in Step a.
# ln -s local-dir global-dir |
This example shows the sequence of operations that is required to create node-specific directories on a two-node cluster. This cluster is configured as follows:
The ORACLE_HOME
environment variable specifies the /global/oracle directory.
The local file system on each node is located under the /local directory.
The following operations are performed on each node:
To create the required directories on the local file system, the following commands are run:
# mkdir -p /local/oracle/network/agent |
# mkdir -p /local/oracle/network/log |
# mkdir -p /local/oracle/network/trace |
# mkdir -p /local/oracle/srvm/log |
# mkdir -p /local/oracle/apache |
To make local copies of the global directories that are to maintain node-specific information, the following commands are run:
# cp -pr $ORACLE_HOME/network/agent /local/oracle/network/. |
# cp -pr $ORACLE_HOME/network/log /local/oracle/network/. |
# cp -pr $ORACLE_HOME/network/trace /local/oracle/network/. |
# cp -pr $ORACLE_HOME/srvm/log /local/oracle/srvm/. |
# cp -pr $ORACLE_HOME/apache /local/oracle/. |
The following operations are performed on only one node:
To remove the global directories, the following commands are run:
# rm -r $ORACLE_HOME/network/agent |
# rm -r $ORACLE_HOME/network/log |
# rm -r $ORACLE_HOME/network/trace |
# rm -r $ORACLE_HOME/srvm/log |
# rm -r $ORACLE_HOME/apache |
To create symbolic links from the local directories to their corresponding global directories, the following commands are run:
# ln -s /local/oracle/network/agent $ORACLE_HOME/network/agent |
# ln -s /local/oracle/network/log $ORACLE_HOME/network/log |
# ln -s /local/oracle/network/trace $ORACLE_HOME/network/trace |
# ln -s /local/oracle/srvm/log $ORACLE_HOME/srvm/log |
# ln -s /local/oracle/apache $ORACLE_HOME/apache |
Perform this procedure for each file that is to maintain node-specific information. The following files are typically required to maintain node-specific information:
For information about other files that might be required to maintain node-specific information, see your Oracle documentation.
On each cluster node, create the local directory that will contain the file that is to maintain node-specific information.
# mkdir -p local-dir |
Specifies that all nonexistent parent directories are created first
Specifies the full path name of the directory that you are creating
On each cluster node, make a local copy of the global file that is to maintain node-specific information.
# cp -p global-file local-dir |
Specifies that the owner, group, permissions modes, modification time, access time, and access control lists are preserved.
Specifies the file name and full path of the global file that you are
copying. This file was installed on the shared file system under the directory
that the ORACLE_HOME
environment
variable specifies.
Specifies the directory that is to contain the local copy of the file. This directory is the directory that you created in Step 1.
Replace the global file that you copied in Step 2 with a symbolic link to the local copy of the file.
From any cluster node, remove the global file that you copied in Step 2.
# rm global-file |
Specifies the file name and full path of the global file that you are removing. This file is the global file that you copied in Step 2.
From any cluster node, create a symbolic link from the local copy of the file to the global file that you removed in Step a.
# ln -s local-file global-file |
This example shows the sequence of operations that is required to create node-specific files on a two-node cluster. This cluster is configured as follows:
The ORACLE_HOME
environment variable specifies the /global/oracle directory.
The local file system on each node is located under the /local directory.
The following operations are performed on each node:
To create the local directory that will contain the files that are to maintain node-specific information, the following command is run:
# mkdir -p /local/oracle/network/admin |
To make a local copy of the global files that are to maintain node-specific information, the following commands are run:
# cp -p $ORACLE_HOME/network/admin/snmp_ro.ora \ /local/oracle/network/admin/. |
# cp -p $ORACLE_HOME/network/admin/snmp_rw.ora \ /local/oracle/network/admin/. |
The following operations are performed on only one node:
To remove the global files, the following commands are run:
# rm $ORACLE_HOME/network/admin/snmp_ro.ora |
# rm $ORACLE_HOME/network/admin/snmp_rw.ora |
To create symbolic links from the local copies of the files to their corresponding global files, the following commands are run:
# ln -s /local/oracle/network/admin/snmp_ro.ora \ $ORACLE_HOME/network/admin/snmp_rw.ora |
# ln -s /local/oracle/network/admin/snmp_rw.ora \ $ORACLE_HOME/network/admin/snmp_rw.ora |
Go to Installing Sun Cluster Support for Oracle Real Application Clusters Packages to install the Sun Cluster Support for Oracle Real Application Clusters software packages.
If you did not install the Sun Cluster Support for Oracle Real Application Clusters packages during your initial Sun Cluster installation, perform this procedure to install the packages. Perform this procedure on all of the cluster nodes that can run Sun Cluster Support for Oracle Real Application Clusters. To complete this procedure, you need the Sun Java Enterprise System Accessory CD Volume 3.
The Sun Cluster Support for Oracle Real Application Clusters packages are as follows:
Packages for the RAC framework resource group
Packages for the Oracle RAC server resource and Oracle RAC listener resource
Install the Sun Cluster Support for Oracle Real Application Clusters packages by using the pkgadd utility.
Because of the preparation that is required before installation, the scinstall(1M) utility does not support automatic installation of the packages for the RAC framework resource group.
Load the Sun Java Enterprise System Accessory CD Volume 3 into the CD-ROM drive.
Become superuser.
Change the current working directory to the directory that contains the packages for the RAC framework resource group.
This directory depends on the version of the Solaris Operating System that you are using
If you are using Solaris 8, run the following command:
# cd /cdrom/cdrom0/components/SunCluster_Oracle_RAC/Solaris_8/Packages |
If you are using Solaris 9, run the following command:
# cd /cdrom/cdrom0/components/SunCluster_Oracle_RAC/Solaris_9/Packages |
On each cluster node that can run Sun Cluster Support for Oracle Real Application Clusters, transfer the contents of the required software packages from the CD-ROM to the node.
The required software packages depend on the storage management scheme that you are using for the Oracle Real Application Clusters database.
If you are using Solaris Volume Manager for Sun Cluster, run the following command:
# pkgadd -d . SUNWscucm SUNWudlm SUNWudlmr SUNWscmd |
If you are using VxVM with the cluster feature, run the following command:
# pkgadd -d . SUNWscucm SUNWudlm SUNWudlmr SUNWcvmr SUNWcvm |
If you are using hardware RAID support, run the following command:
# pkgadd -d . SUNWscucm SUNWudlm SUNWudlmr SUNWschwr |
If you are using Sun StorEdge QFS shared file system with hardware RAID support, run the following command:
# pkgadd -d . SUNWscucm SUNWudlm SUNWudlmr SUNWschwr |
Change the current working directory to the directory that contains the packages for the Oracle RAC server resource and Oracle RAC listener resource.
This directory depends on the version of the Solaris Operating System that you are using.
If you are using Solaris 8, run the following command:
# cd /cdrom/cdrom0/components/SunCluster_HA_Oracle_3.1/Solaris_8/Packages |
If you are using Solaris 9, run the following command:
# cd /cdrom/cdrom0/components/SunCluster_HA_Oracle_3.1/Solaris_9/Packages |
On each cluster node that can run Sun Cluster Support for Oracle Real Application Clusters, transfer the contents of the required software packages from the CD-ROM to the node.
The required software packages depend on the locale that you require.
Go to Preparing the Sun Cluster Nodes to prepare the Sun Cluster nodes.
Preparing the Sun Cluster nodes modifies the configuration of the operating system to enable Oracle Real Application Clusters to run on Sun Cluster nodes. Preparing the Sun Cluster nodes and disks involves the following tasks:
Bypassing the NIS name service
Creating the database administrator group and the Oracle user account
Perform these tasks on all nodes where Sun Cluster Support for Oracle Real Application Clusters can run. If you do not perform these tasks on all nodes, the Oracle installation is incomplete. An incomplete Oracle installation causes Sun Cluster Support for Oracle Real Application Clusters to fail during startup.
Bypassing the NIS name service protects the Sun Cluster Support for Oracle Real Application Clusters data service against a failure of a cluster node's public network. A failure of a cluster node's public network might cause the NIS name service to become unavailable. If Sun Cluster Support for Oracle Real Application Clusters refers to the NIS name service, unavailability of the name service might cause the Sun Cluster Support for Oracle Real Application Clusters data service to fail.
Bypassing the NIS name service ensures that the Sun Cluster Support for Oracle Real Application Clusters data service does not refer to the NIS name service when the data service sets the user identifier (ID). The Sun Cluster Support for Oracle Real Application Clusters data service sets the user ID when the data service starts or stops the database.
Become superuser on all nodes where Sun Cluster Support for Oracle Real Application Clusters can run.
On each node, include the following entries in the /etc/nsswitch.conf file.
passwd: files nis [TRYAGAIN=0] publickey: files nis [TRYAGAIN=0] project: files nis [TRYAGAIN=0] group: files |
For more information about the /etc/nsswitch.conf file, see the nsswitch.conf(4) man page.
Go to How to Create the Database Administrator Group and the Oracle User Account.
Perform the following steps as superuser on each cluster node.
On each node, create an entry for the database administrator group in the /etc/group file, and add potential users to the group.
This group normally is named dba. Verify that root and oracle are members of the dba group, and add entries as necessary for other database administrator (DBA) users. Verify that the group IDs are the same on all of the nodes that run Sun Cluster Support for Oracle Real Application Clusters. For example, add the following entry to the /etc/group file.
dba:*:520:root,oracle |
You can create the name service entries in a network name service, such as the Network Information Service (NIS) or NIS+, so that the information is available to the data service clients. You can also create entries in the local /etc files to eliminate dependency on the network name service.
On each node, create an entry for the Oracle user ID (the group and password) in the /etc/passwd file, and run the pwconv(1M) command to create an entry in the /etc/shadow file.
This Oracle user ID is normally oracle. For example, add the following entry to the /etc/passwd file.
# useradd -u 120 -g dba -d /oracle-home oracle |
Ensure that the user IDs are the same on all of the nodes that run Sun Cluster Support for Oracle Real Application Clusters.
After you set up the cluster environment for Oracle Real Application Clusters, go to How to Install the Oracle UDLM to install the Oracle UDLM software on each cluster node.
To enable the Oracle UDLM software to run correctly, you must ensure that sufficient shared memory is available on all of the cluster nodes. For detailed instructions for installing the Oracle UDLM, see the Oracle Real Application Clusters CD-ROM.
Before you install the Oracle UDLM, ensure that you have created entries for the database administrator group and the Oracle user ID. See How to Create the Database Administrator Group and the Oracle User Account for details.
You must install the Oracle UDLM software on the local disk of each node.
Become superuser on a cluster node.
Install the Oracle UDLM software.
See the appropriate Oracle Real Application Clusters installation documentation for instructions.
Ensure that you did not receive any error messages when you installed the Oracle UDLM packages. If an error occurred during package installation, correct the problem before you install the Oracle UDLM software.
Update the /etc/system file with the shared memory configuration information.
You must configure these parameters on the basis of the resources that are available in the cluster. Decide the appropriate values, but ensure that the Oracle UDLM can create a shared memory segment that conforms to its configuration requirements.
The following example shows entries to configure in the /etc/system file.
*SHARED MEMORY/ORACLE set shmsys:shminfo_shmmax=268435456 set semsys:seminfo_semmap=1024 set semsys:seminfo_semmni=2048 set semsys:seminfo_semmns=2048 set semsys:seminfo_semmsl=2048 set semsys:seminfo_semmnu=2048 set semsys:seminfo_semume=200 set shmsys:shminfo_shmmin=200 set shmsys:shminfo_shmmni=200 set shmsys:shminfo_shmseg=200 |
Shut down and reboot each node on which the Oracle UDLM software is installed.
Before you reboot, you must ensure that you have correctly installed and configured the Oracle UDLM software. Also verify that you have correctly installed your volume manager packages. If you use VxVM, check that you have installed the software and that the license for the VxVM cluster feature is valid. Otherwise, a panic will occur.
For detailed instructions, see “Shutting Down and Booting a Single Cluster Node” in Sun Cluster System Administration Guide for Solaris OS.
After you have installed the Oracle UDLM software on each cluster node, the next step depends on your storage management scheme as shown in the following table.
Storage Management Scheme |
Next Step |
---|---|
Solaris Volume Manager for Sun Cluster | |
VxVM with the cluster feature |
Creating a VxVM Shared-Disk Group for the Oracle Real Application Clusters Database |
Other |
Registering and Configuring the RAC Framework Resource Group |
Perform this task only if you are using Solaris Volume Manager for Sun Cluster.
If you are using Solaris Volume Manager for Sun Cluster, Solaris Volume Manager requires a multi-owner disk set for the Oracle Real Application Clusters database to use. For information about Solaris Volume Manager for Sun Cluster multi–owner disk sets, see “Disk Set Concepts for Solaris Volume Manager for Sun Cluster” in Solaris Volume Manager Administration Guide.
Before you create a multi-owner disk set in Solaris Volume Manager for Sun Cluster for the Oracle Real Application Clusters database, note the following points.
Ensure that the required Sun Cluster Support for Oracle Real Application Clusters software packages are installed on each node. For more information, see Installing Sun Cluster Support for Oracle Real Application Clusters Packages.
Do not create any file systems in the multi-owner disk set because only the raw data file uses this disk set.
Disk devices that you add to the multi-owner disk set must be directly attached to all of the cluster nodes.
Create a multi-owner disk set.
Use the metaset(1M) command for this purpose.
# metaset -s setname -M -a -h nodelist |
Specifies the name of the disk set that you are creating.
Specifies that the disk set that you are creating is a multi-owner disk set.
Specifies that the nodes that the -h option specifies are to be added to the disk set.
Specifies a space-separated list of nodes that are to be added to the disk set. The Sun Cluster Support for Oracle Real Application Clusters software packages must be installed on each node in the list.
Add global devices to the disk set that you created in Step 1.
# metaset -s setname -a devicelist |
Specifies that you are modifying the disk set that you created in Step 1.
Specifies that the devices that devicelist specifies are to be added to the disk set.
Specifies a space-separated list of full device ID path names for the global devices that are to be added to the disk set. To enable consistent access to each device from any node in the cluster, ensure that each device ID path name is of the form /dev/did/dsk/dN, where N is the device number.
For the disk set that you created in Step 1, create the volumes that the Oracle Real Application Clusters database will use.
If you are creating many volumes for Oracle data files, you can simplify this step by using soft partitions. For more information, see “Soft Partitions (Overview)” in Solaris Volume Manager Administration Guide and “Soft Partitions (Tasks)” in Solaris Volume Manager Administration Guide.
Create each volume by concatenating slices on global devices that you added in Step 2. Use the metainit(1M) command for this purpose.
# metainit -s setname volume-abbrev numstripes width slicelist |
Specifies that you are creating a volume for the disk set that you created in Step 1.
Specifies the abbreviated name of the volume that you are creating. An abbreviated volume name has the format dV, where V is the volume number.
Specifies the number of slices in each stripe. If you set width to greater than 1, the slices are striped.
Specifies a space-separated list of slices that the volume contains. Each slice must reside on a global device that you added in Step 2.
Verify that each node is correctly added to the multi-owner disk set.
Use the metastat(1M) command for this purpose.
# metastat -s setname |
Specifies that you are verifying the disk set that you created in Step 1
This command displays a table that contains the following information for each node that is correctly added to the disk set:
The Host column contains the node name.
The Owner column contains the text multi-owner.
The Member column contains the text Yes.
Verify that the multi-owner disk set is correctly configured.
# scconf -pvv | grep setname |
Specifies that configuration information only for the disk set that you created in Step 1 is displayed
This command displays the device group information for the disk set. For a multi-owner disk set, the device group type is Multi-owner_SVM.
Verify the online status of the multi-owner disk set.
# scstat -D |
This command displays the node names of nodes in the multi-owner disk set that are online.
On each node that can own the disk set, change the ownership of each volume that you created in Step 3 as follows:
Owner: oracle
Group: dba
Ensure that you change ownership only of volumes that the Oracle Real Application Clusters database will use.
# chown oracle:dba volume-list |
Specifies a space-separated list of the logical names of the volumes that you created for the disk set. The format of these names depends on the type of device where the volume resides, as follows:
For block devices: /dev/md/setname/dsk/dV
For raw devices: /dev/md/setname/rdsk/dV
The replaceable items in these names are as follows:
Specifies the name of the multi-owner disk set that you created in Step 1
Specifies the volume number of a volume that you created in Step 3
Ensure that this list specifies each volume that you created in Step 3.
On each node that can own the disk set, grant the oracle user read access and write access to each volume for which you changed the ownership in Step 7.
Ensure that you change access permissions only of volumes that the Oracle Real Application Clusters database will use.
# chmod u+rw volume-list |
Specifies a space-separated list of the logical names of the volumes to which you are granting the oracle user read access and write access. Ensure that this list contains the volumes that you specified in Step 7.
This example shows the sequence of operations that is required to create a multi-owner disk set in Solaris Volume Manager for Sun Cluster. This example assumes that the volumes reside on raw devices.
To create the multi-owner disk set, the following command is run:
# metaset -s racdbset -M -a -h rachost1 rachost2 rachost3 rachost4 |
The multi-owner disk set is named racdbset. The nodes rachost1, rachost2, rachost3, and rachost4 are added to this disk set.
To add the global device /dev/did/dsk/d0 to the disk set, the following command is run:
# metaset -s racdbset -a /dev/did/dsk/d0 |
To create a volume for the disk set, the following command is run:
# metainit -s racdbset d0 1 1 /dev/did/dsk/d0s0 |
The volume is named d0. This volume is created by a one-on-one concatenation of the slice /dev/did/dsk/d0s0. The slice is not striped.
To verify that each node is correctly added to the multi-owner disk set, the following command is run:
# metastat -s racdbset Multi-owner Set name = racdbset, Set number = 1, Master = rachost2 Host Owner Member rachost1 multi-owner Yes rachost2 multi-owner Yes rachost3 multi-owner Yes rachost4 multi-owner Yes Drive Dbase d6 Yes d10 Yes |
To verify that the multi-owner disk set is correctly configured, the following command is run:
# scconf -pvv | grep racdbset Device group name: racdbset (racdbset) Device group type: Multi-owner_SVM (racdbset) Device group failback enabled: no (racdbset) Device group node list: rachost1, rachost2, rachost3, rachost4 (racdbset) Device group ordered node list: no (racdbset) Device group desired number of secondaries: 0 (racdbset) Device group diskset name: racdbset |
To verify the online status of the multi-owner disk set, the following command is run:
# scstat -D -- Device Group Servers -- Device Group Primary Secondary ------------ ------- --------- -- Device Group Status -- Device Group Status ------------ ------ -- Multi-owner Device Groups -- Device Group Online Status ------------ ------------- Multi-owner device group: racdbset rachost1,rachost2,rachost3,rachost4 |
To change the ownership of the volume in the disk set to owner oracle in group dba, the following command is run:
# chown oracle:dba /dev/md/racdbset/rdsk/d0 |
This command is run on each node that can own the disk set.
To grant the oracle user read access to the volume in the disk set, the following command is run:
# chmod u+rw /dev/md/racdbset/rdsk/d0 |
This command is run on each node that can own the disk set.
After you have created a multi-owner disk set for the Oracle Real Application Clusters database, go to Registering and Configuring the RAC Framework Resource Group to register and configure Sun Cluster Support for Oracle Real Application Clusters.
Perform this task only if you are using VxVM with the cluster feature.
If you are using VxVM with the cluster feature, VxVM requires a shared-disk group for the Oracle Real Application Clusters database to use.
Before you create a VxVM shared-disk group for the Oracle Real Application Clusters database, note the following points.
Ensure that the required Sun Cluster Support for Oracle Real Application Clusters software packages are installed on each node. For more information, see Installing Sun Cluster Support for Oracle Real Application Clusters Packages.
Do not register the shared-disk group as a cluster device group with the cluster.
Do not create any file systems in the shared-disk group because only the raw data file uses this disk group.
Disks that you add to the shared-disk group must be directly attached to all of the cluster nodes.
Ensure that your VxVM license is current. If your license expires, the node panics.
Use VERITAS commands that are provided for creating a VxVM shared-disk group.
For information about VxVM shared-disk groups, see your VxVM documentation.
After you have created a shared-disk group for the Oracle Real Application Clusters database, go to Registering and Configuring the RAC Framework Resource Group to register and configure Sun Cluster Support for Oracle Real Application Clusters.