Chapter 2 Installing and Configuring Sun Cluster HA for Oracle

This chapter provides instructions for setting up and administering the Sun Cluster HA for Oracle data service on your Sun Cluster nodes.

This chapter contains the following procedures:

• "How to Prepare the Nodes"

• "How to Install the Oracle Software"

• "How to Verify the Oracle Installation"

• "How to Configure Oracle Database Access"

• "How to Create an Oracle Database"

• "How to Set Up Oracle Database Permissions"

• "How to Install Sun Cluster HA for Oracle Packages"

• "How to Register and Configure Sun Cluster HA for Oracle"

• "How to Configure SUNW.HAStorage Resource Type"

• "How to Verify the Sun Cluster HA for Oracle Installation"

• "How to Configure Sun Cluster HA for Oracle Extension Properties"

You must configure Sun Cluster HA for Oracle as a failover service. For general information about data services, resource groups, resources, and other related topics, see Chapter 1, Planning for Sun Cluster Data Services and the Sun Cluster 3.0 Concepts document.

Installing and Configuring Sun Cluster HA for Oracle

Table 2-1 lists the sections that describe the installation and configuration tasks.

Preparing to Install Sun Cluster HA for Oracle

To prepare Sun Cluster nodes for Sun Cluster HA for Oracle installation, you must select an install location for the Oracle application files (Oracle binaries, configuration files, and parameter files) and for the database-related files (control file, redo logs, and data files).

Table 2-2 shows the possible install location combinations--either on the cluster file system, raw global devices, or the local disk of the physical host. For the advantages and disadvantages of placing the Oracle binaries on the local versus the global file system, see "Determining the Location of the Application Binaries".

Installing the Oracle Server Software

Use the procedures in this section to do the following:

• Prepare the Sun Cluster nodes.

• Install the Oracle application software.

• Verify the Oracle installation.

Before setting up Sun Cluster HA for Oracle, you must have configured the Sun Cluster software on each node by using the procedures described in the Sun Cluster 3.0 Installation Guide.

How to Prepare the Nodes

This procedure describes how to prepare the cluster nodes for installation of the Oracle application software.

Perform all the steps described in this section on all Sun Cluster nodes.

Consult the Oracle documentation before performing this procedure.

The following steps prepare Sun Cluster nodes and install the Oracle software:

1. Become superuser on all the nodes in the cluster.

2. Set up the /etc/nsswitch.conf files as follows so that the data service starts and stops correctly in case of switchovers or failovers.

   On each node that can master the logical host running Sun Cluster HA for Oracle, the /etc/nsswitch.conf file must have one of the following entries for group.

   group: group: files group: files [NOTFOUND=return] nis group: files [NOTFOUND=return] nisplus

   Sun Cluster HA for Oracle uses the su user command when starting and stopping the database node. The above settings ensure that the su(1M) command does not refer to NIS/NIS+ when the network information name service is not available because of a failure of the public network on the cluster node.

3. Set up the cluster file system for Sun Cluster HA for Oracle.

   If you are using raw devices to contain the databases, you must configure the global devices for raw device access. For information on configuring global devices, see the Sun Cluster 3.0 Installation Guide.

   When using Solstice^TM DiskSuite, configure Oracle to use UFS logging or raw mirrored meta devices. For more information on setting up raw mirrored meta devices, see the Solstice DiskSuite documentation.

4. Prepare the Oracle home directory for Oracle installation.

   Choose a location for the $ORACLE_HOME directory on either a local or multihost disk.

   ---

   Note -

   If you choose to install the Oracle binaries on a local disk of the physical hosts, use a separate disk, if possible. Doing so prevents Oracle binaries from being overwritten if the operating environment is reinstalled.

   ---

5. On each node, create an entry for the database administrator group in the /etc/group file and add potential users to the group.

   Normally, this group is named dba. Verify that root and oracle_id are members of the dba group and add entries as necessary for other DBA users. Ensure that the group IDs are the same on all the nodes that run Sun Cluster HA for Oracle. For example:

   dba:*:520:root,oracle

   You can make the group entries in a network name service (for example, NIS or NIS+). You should also make entries in the local /etc files to eliminate dependency on the network name service.

6. On each node, create an entry for the Oracle user ID (oracle_id).

   Normally, oracle_id is oracle. The following command line updates the /etc/passwd and /etc/shadow files as required.

   # useradd -u 120 -g dba -d /Oracle-home oracle

   Ensure that oracle_id is the same on all the nodes that run Sun Cluster HA for Oracle.

How to Install the Oracle Software

1. Become superuser on a node in the cluster.

2. Note the requirements for Oracle installation.

   You can install Oracle binaries on either the local disks of the physical hosts or on the cluster file system. For more information about installation locations, see "Preparing to Install Sun Cluster HA for Oracle".

   If you plan to install Oracle software on the cluster file system, you must first start Sun Cluster and take ownership of the disk device group.

3. Install the Oracle software.

   Regardless of where the Oracle software is to be installed, on each node, modify the /etc/system files according to standard Oracle installation procedures. Reboot afterward.

   Log in as oracle_id to ensure ownership of the entire directory before performing this step. For instructions on installing Oracle software, refer to the appropriate Oracle installation and configuration guides.

How to Verify the Oracle Installation

1. Verify that $ORACLE_HOME/bin/oracle is owned by the oracle_id user and the dba group.

2. Verify that the $ORACLE_HOME/bin/oracle permissions are set as follows:

   -rwsr-s--x

3. Verify that the listener binaries exist in $ORACLE_HOME/bin.

Where to Go from Here

When you have completed the work in this section, go to "Creating an Oracle Database".

Creating an Oracle Database

Complete both procedures in this section to configure and create the initial Oracle database in a Sun Cluster configuration. If you are creating and configuring additional databases, perform only the procedure, "How to Create an Oracle Database".

How to Configure Oracle Database Access

1. Configure the disk devices for use by your volume manager.

   For more information, see the appendix for your volume manager in the Sun Cluster 3.0 Installation Guide.

2. If you are using Solstice DiskSuite, set up UFS logging or raw mirrored meta devices on all the nodes that are running Sun Cluster HA for Oracle.

   If you are using raw devices to contain the databases, change the owner, group, and mode of each of the raw mirrored meta devices and verify the changes. If you are not using raw devices, skip this step. Instructions for configuring raw devices are provided in the Sun Cluster 3.0 Installation Guide.

   If you are creating raw devices, type the following commands for each device on each node that can master the Oracle resource group:

   # chown oracle_id /dev/md/disk_device_group/rdsk/dn # chgrp dba_id /dev/md/disk_device_group/rdsk/dn # chmod 600 /dev/md/disk_device_group/rdsk/dn

   Verify that the changes have taken effect.

   # ls -lL /dev/md/disk_device_group/rdsk/dn

3. If you are using VERITAS Volume Manager, set up UFS logs or raw devices on all the nodes.

   For information about Solaris UFS logging, see the mount_ufs(1M) man page and the Solaris Transition Guide. If you are using raw devices to contain the databases, change the owner, group, and mode of each device. If not, skip this step.

   If you are creating raw devices, type the following command for each raw device:

   # vxedit -g disk_device_group set user=oracle_id \ group=dba mode=600 volume_name

   Verify that the changes have taken effect.

   # ls -lL /dev/vx/rdsk/disk_device_group/volume_name

   Next, re-register the disk device group with the cluster. This step is necessary to keep the VxVM namespace consistent throughout the cluster. If you are using Solstice DiskSuite, you can skip this step.

   # scconf -c -D name=disk_device_group

How to Create an Oracle Database

1. Prepare database configuration files.

   Place all database-related files (data files, redolog files, and control files) on either shared raw global devices or the cluster file system. For information on installation locations, refer to "Preparing to Install Sun Cluster HA for Oracle".

   Within the init$ORACLE_SID.ora or config$ORACLE_SID.ora file, you might need to modify the assignments for control_files and background_dump_dest to specify the locations of the control files and alert files.

   ---

   Note -

   If you are using Solaris authentication for database logins, set the remote_os_authent variable in the init$ORACLE_SID.ora file to True.

   ---

2. Create the database.

   Start the Oracle installer and select the option to create a database. Alternatively, depending on your Oracle version, you can create the database by using the Oracle svrmgrl command.

   During creation, ensure that all database-related files are placed in the appropriate location: either on shared global devices or on the cluster file system.

3. Verify that the file names of your control files match the file names in your configuration files.

4. Create the v$sysstat view.

   Run the catalog scripts that create the v$sysstat view. The Sun Cluster fault monitoring scripts use this view.

Where to Go from Here

When you have completed the work in this section, go to "Setting Up Oracle Database Permissions".

Setting Up Oracle Database Permissions

Use this procedure to set up Oracle database permissions.

How to Set Up Oracle Database Permissions

Depending on which authentication method you choose, Oracle authentication or Solaris authentication, perform either Step 1 or Step 2 of this procedure.

1. Enable access for the user and password to be used for fault monitoring.

   You must complete this step if you do not enable Solaris authentication, as described in Step 2.

   For all supported Oracle releases, enable access by typing the following script into the screen displayed by the srvmgrl(1M) command.

   # svrmgrl connect internal; grant connect, resource to user identified by passwd; alter user user default tablespace system quota 1m on system; grant select on v_$sysstat to user; grant create session to user; grant create table to user; disconnect; exit;

2. Grant permission for the database to use Solaris authentication.

   Perform this step if you choose not to use Step 1.

   ---

   Note -

   The user for which you enable Solaris authentication is the user who owns the files under $ORACLE_HOME. The following code sample shows that the user oracle owns these files.

   ---

   # svrmgrl connect internal; create user ops$oracle identified by externally default tablespace system quota 1m on system; grant connect, resource to ops$oracle; grant select on v_$sysstat to ops$oracle; grant create session to ops$oracle; grant create table to ops$oracle; disconnect; exit;

3. Configure NET8 for Sun Cluster.

   The listener.ora and tnsnames.ora files must be accessible from all the nodes in the cluster. You can place these files under the cluster file system or in the local file system of each node that can potentially run the Oracle resources.

   Sun Cluster HA for Oracle imposes no restrictions on the listener name-it can be any valid Oracle listener name.

   The following code sample identifies the lines in listener.ora that are updated.

   LISTENER = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP) (HOST = logicalhostname) <- use logical host name (PORT = 1527) ) ) . . SID_LIST_LISTENER = . . (SID_NAME = SID) <- Database name, default is ORCL

   The following code sample identifies the lines in tnsnames.ora that are updated on client machines.

   service_name = . . (ADDRESS = (PROTOCOL = TCP) (HOST = logicalhostname) <- logical host name (PORT = 1527) <- must match port in LISTENER.ORA ) ) (CONNECT_DATA = (SID = <SID>)) <- database name, default is ORCL

   The following example shows how to update the listener.ora and tnsnames.ora files given the following Oracle instances.

   Instance	Logical Host	Listener
   ora8	hadbms3	LISTENER-ora8
   ora7	hadbms4	LISTENER-ora7

   The corresponding listener.ora entries are:

   LISTENER-ora7 = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP) (HOST = hadbms4) (PORT = 1530) ) ) SID_LIST_LISTENER-ora7 = (SID_LIST = (SID_DESC = (SID_NAME = ora7) ) ) LISTENER-ora8 = (ADDRESS_LIST = (ADDRESS= (PROTOCOL=TCP) (HOST=hadbms3)(PORT=1806)) ) SID_LIST_LISTENER-ora8 = (SID_LIST = (SID_DESC = (SID_NAME = ora8) ) )

   The corresponding tnsnames.ora entries are:

   ora8 = (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP) (HOST = hadbms3) (PORT = 1806)) ) (CONNECT_DATA = (SID = ora8)) ) ora7 = (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP) (HOST = hadbms4) (PORT = 1530)) ) (CONNECT_DATA = (SID = ora7)) )

4. Verify that Sun Cluster is installed and running on all nodes.

   # scstat

Where to Go from Here

To register and configure the Sun Cluster HA for Oracle data service, go to "Installing Sun Cluster HA for Oracle Packages".

Installing Sun Cluster HA for Oracle Packages

The scinstall(1M) utility installs SUNWscor, the Sun Cluster HA for Oracle data service package, on a cluster. You can install specific data service packages from the Sun Cluster data service CD by using interactive scinstall, or you can install all data service packages on the CD by using the -s option to non-interactive scinstall. The preferred method is to use interactive scinstall, as described in the following procedure.

The data service packages might have been installed as part of your initial Sun Cluster installation. If not, use this procedure to install them now.

How to Install Sun Cluster HA for Oracle Packages

You need the Sun Cluster data service CD to complete this procedure. Perform this procedure on all cluster nodes that run Sun Cluster HA for Oracle.

1. Load the data service CD into the CD-ROM drive.

2. Run scinstall with no options.

   This step starts scinstall in interactive mode.

3. Select the menu option: "Add support for new data service to this cluster node."

   You can then load software for any data services that exist on the CD.

4. Exit scinstall and unload the CD from the drive.

Where to Go from Here

See "Registering and Configuring Sun Cluster HA for Oracle" to register Sun Cluster HA for Oracle and configure the cluster for the data service.

Registering and Configuring Sun Cluster HA for Oracle

Sun Cluster HA for Oracle is registered and configured as a failover data service. You must register the data service and configure resource groups and resources for the Oracle server and listener. For details on resources and resource groups, see Chapter 1, Planning for Sun Cluster Data Services and the Sun Cluster 3.0 Concepts document.

How to Register and Configure Sun Cluster HA for Oracle

To register and configure the Sun Cluster HA for Oracle data service, use the Cluster Module of Sun Management Center or the following command-line procedure.

To perform this procedure, you must have the following information:

• The names of the cluster nodes that master the data service.

• The logical host name to be used by clients to access the data service. Normally, you set up this IP address up when you install the cluster. For details, see the section on setting up logical host names in the Sun Cluster 3.0 Installation Guide.

• The path to the Oracle application binaries for the resources that you plan to configure.

Perform this procedure on any cluster member.

1. Become superuser on a node in the cluster.

2. Register the resource types for the data service.

   For Sun Cluster HA for Oracle, you register two resource types, SUNW.oracle_server and SUNW.oracle_listener, as follows:

   # scrgadm -a -t SUNW.oracle_server # scrgadm -a -t SUNW.oracle_listener

   -a

   Adds the data service resource type.

   -t SUNW.oracle_type

   Specifies the predefined resource type name for your data service.

3. Create a failover resource group to hold the network and application resources.

   You can optionally select the set of nodes on which the data service can run with the -h option, as follows:

   # scrgadm -a -g resource-group-name [-h nodelist]

   -g resource-group-name

   Specifies the name of the resource group. This name can be your choice but must be unique for resource groups within the cluster.

   -h nodelist

   Specifies an optional comma-separated list of physical node names or IDs that identify potential masters. The order here determines the order in which the nodes are considered as primary during failover.

   ---

   Note -

   Use -h to specify the order of the node list. If all the nodes in the cluster are potential masters, you need not use the -h option.

   ---

4. Verify that all logical host names that you are using have been added to your name service database.

   You should have done this verification as part of the Sun Cluster installation.

   ---

   Note -

   To avoid any failures because of name service lookup, ensure that all logical host names are present in the server's and client's /etc/hosts file.

   ---

5. Add a logical host name to the failover resource group.

   # scrgadm -a -L -g resource-group-name -l logical-hostname \ [-j resource-name] [-n network-interface-id-list]

   -l logical-hostname

   Specifies a logical hostname.

   -j resource-name

   An optional name for the logical host name resource. If a name is not specified, the default resource name is the first name to appear after the -l option.

   -n network-interface-id-list

   An optional comma-separated list that identifies the NAFO groups on each node. All nodes in nodelist of the resource group must be represented in the network-interface-list. If you do not specify this option, scrgadm attempts to discover a net adapter on the subnet identified by the hostname list for each node in nodelist.

6. Create Oracle application resources in the failover resource group.

   # scrgadm -a -j resource-name -g resource-group-name \ -t SUNW.oracle_server \ -x Connect_string=user/passwd \ -x ORACLE_SID=instance-name \ -x ORACLE_HOME=Oracle-home \ -x Alert_log_file=path-to-log

   # scrgadm -a -j resource-name -g resource-group-name \ -t SUNW.oracle_listener \ -x LISTENER_NAME=listener-name \ -x ORACLE_HOME=Oracle-home

   -j resource-name

   Specifies the name of the resource to add.

   -g resource-group-name

   Specifies the name of the resource group into which the resources are to be placed.

   -t SUNW.oracle_server/listener

   Specifies the type of the resource to add.

   -x Alert_log_file=path-to-log

   Sets the path under $ORACLE_HOME for the server message log.

   -x Connect_string=user/passwd

   The user and password used by the fault monitor to connect to the database. These settings must agree with the permissions you set up in "How to Set Up Oracle Database Permissions". If you are using Solaris authorization, type a slash (/) instead of the user name and password.

   -x ORACLE_SID=instance-name

   Sets the Oracle system identifier.

   -x LISTENER_NAME=listener-name

   Sets the name of the Oracle listener instance. This name must match the corresponding entry in listener.ora.

   -x ORACLE_HOME=Oracle-home

   Sets the path to the Oracle home directory.

   ---

   Note -

   Optionally, you can set additional extension properties that belong to the Oracle data service to override the default value. See "Configuring Sun Cluster HA for Oracle Extension Properties" for a list of extension properties.

   ---

7. Enable the resource and fault monitoring, then move the resource group into a managed state and bring it online.

   # scswitch -Z -g resource-group-name

   -Z

   Enables the resource and monitor, moves the resource group to the managed state, and brings it online.

   -g resource-group-name

   Specifies the name of the resource group.

Example-Registering Sun Cluster HA for Oracle

The following example shows how to register Sun Cluster HA for Oracle on a two- node cluster.

Cluster Information
Node names: phys-schost-1, phys-schost-2
Logical Hostname: schost-1
Resource group: oracle-rg (failover resource group), Oracle Resources: oracle-server, oracle-listener,Oracle Instances: ora-lsnr (listener), ora-srvr (server)
 
(Add the failover resource group to contain all the resources.)
# scrgadm -a -g oracle-rg
 
(Add the logical hostname resource to the resource group.)
# scrgadm -a -L -g oracle-rg -l schost-1 
 
(Register the Oracle resource types)
# scrgadm -a -t SUNW.oracle_server
# scrgadm -a -t SUNW.oracle_listener
 
(Add the Oracle application resources to the resource group.)
# scrgadm -a -j oracle-server -g oracle-rg \
-t SUNW.oracle_server -x ORACLE_HOME=/global/oracle \
-x Alert_log_file=/global/oracle/message-log \
-x ORACLE_SID=ora-srvr -x Connect_string=scott/tiger
 
# scrgadm -a -j oracle-listener -g oracle-rg \
-t SUNW.oracle_listener -x ORACLE_HOME=/global/oracle \
-x LISTENER_NAME=ora-lsnr
 
(Bring the resource group online.)
# scswitch -Z -g oracle-rg

How to Configure SUNW.HAStorage Resource Type

The SUNW.HAStorage resource type synchronizes actions between HA storage and data service. Because Sun Cluster HA for Oracle is disk-intensive, we strongly recommend that you set up SUNW.HAStorage.

For details on the background, see the SUNW.HAStorage(5) man page and "Relationship Between Resource Groups and Disk Device Groups". For the procedure, see "How to Set Up SUNW.HAStorage Resource Type for New Resources".

Verifying the Sun Cluster HA for Oracle Installation

Perform the following verification tests to ensure the Sun Cluster HA for Oracle was installed correctly.

These sanity checks ensure that the Oracle instance can be started by all the nodes that run Sun Cluster HA for Oracle and can be accessed by the other nodes in the configuration. Perform these sanity checks to isolate any problems starting Oracle from the Sun Cluster HA for Oracle data service.

How to Verify the Sun Cluster HA for Oracle Installation

1. Log in to the node monitoring the resource group that contains the Oracle resources and set the Oracle environment variables.

   Log in as oracle_id to the node that currently masters the Oracle resource group and set the environment variables ORACLE_SID and ORACLE_HOME.

2. Confirm that you can start the Oracle instance from this node.

3. Confirm that you can connect to the Oracle instance.

   Use the sqlplus command with the tns_service variable defined in the tnsnames.ora file:

   # sqlplus user/passwd@tns_service

4. Shut down the Oracle instance.

   Because the Oracle instance is under Sun Cluster control, Sun Cluster restarts it afterward.

5. Switch the resource group that contains the Oracle database resource to another node in the cluster.

   For example:

   # scswitch -z -g rg-name -h phys-nodename

6. Log in to the node now that contains the resource group and repeat the checks listed in Step 1.

   Log in as oracle_id to the new master node and confirm interactions with the Oracle instance.

Oracle Clients

Clients must always refer to the database by using the logical host name (an IP address that can move between physical nodes during failover), not the physical host name (a machine name).

For example, in the tnsnames.ora file, you must specify the logical host name as the host on which the database instance is running. See "How to Set Up Oracle Database Permissions".

Oracle client-server connections cannot survive a Sun Cluster HA for Oracle switchover. The client application must be prepared to handle disconnection and reconnection or recovery as appropriate. A transaction monitor might simplify the application. Further, Sun Cluster HA for Oracle node recovery time is application dependent.

Configuring Sun Cluster HA for Oracle Extension Properties

This section describes how to configure Sun Cluster HA for Oracle extension properties.

For details on all Sun Cluster properties, see Appendix A, Standard Properties.

How to Configure Sun Cluster HA for Oracle Extension Properties

Typically, you configure extension properties by using the Cluster Module of Sun Management Center or the command line scrgadm -x parameter=value at the time you create the Oracle server and listener resources. You can also configure them later by using the procedures described in Chapter 9, Administering Data Service Resources.

Some extension properties can be updated dynamically and others only when the resource is created or when it is disabled. The Tunable column in the following two tables indicates when the property can be updated.

The required extension property for creating an Oracle listener resource is ORACLE_HOME. Table 2-3 describes the extension properties that you can set for the Oracle listener resource.

Table 2-4 describes the extension properties that you can set for the Oracle server.The only extension properties that you are required to set for the Oracle server are ORACLE_HOME, ORACLE_SID, Alert_log_file, and Connect_string.