Sun Cluster Data Service for Oracle RAC Guide for Solaris OS

ProcedureHow to Add Sun Cluster Support for Oracle RAC to Selected Nodes

Perform this procedure if you are adding nodes to a cluster and you require Sun Cluster Support for Oracle RAC to run on the nodes. Perform this procedure from only one node.

This procedure provides the long forms of the Sun Cluster maintenance commands. Most commands also have short forms. Except for the forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands, in Sun Cluster Data Services Planning and Administration Guide for Solaris OS.

This task involves adding the selected nodes from the following resource groups in the following order:

Before You Begin

Ensure that the required Sun Cluster Support for Oracle RAC software packages are installed on each node to which you are adding Sun Cluster Support for Oracle RAC. For more information, see Installing the Sun Cluster Support for Oracle RAC Packages.

  1. Become superuser on any cluster node.

  2. Add the nodes to any resource groups that contain scalable file-system mount-point resources.

    If no resource groups that contain scalable file-system mount-point resources are configured, omit this step.

    For each resource group to which you are adding nodes, run the following command:


    # clresourcegroup add-node -S -n nodelist scal-mp-rg
    
    -n nodelist

    Specifies a comma-separated list of cluster nodes to which you are adding Sun Cluster Support for Oracle RAC.

    scal-mp-rg

    Specifies the name of the resource group to which you are adding nodes.

  3. Add the nodes to the RAC framework resource group.


    # clresourcegroup add-node -S -n nodelist rac-fmwk-rg
    
    -n nodelist

    Specifies a comma-separated list of cluster nodes to which you are adding Sun Cluster Support for Oracle RAC.

    rac-fmwk-rg

    Specifies the name of the resource group to which you are adding nodes.

  4. Add the nodes to any scalable device groups that you are using for Oracle files.

    If you are not using any scalable device groups for Oracle files, omit this step.

    How to perform this step depends on the type of the scalable device group.

    • For each Solaris Volume Manager for Sun Cluster multi-owner disk set, type the following command:


      # metaset -s set-name -M -a -h nodelist
      
      -s set-name

      Specifies the Solaris Volume Manager for Sun Cluster multi-owner disk set to which you are adding nodes.

      -h nodelist

      Specifies a space-separated list of cluster nodes that you are adding to the multi-owner disk set.

    • For each VxVM shared-disk group, use VERITAS commands to add the nodes to the VxVM shared-disk group.

      For more information, see your VxVM documentation.

  5. Add the nodes to any resource groups that contain scalable device group resources.

    If no resource groups that contain scalable device group resources are configured, omit this step.

    For each resource group to which you are adding nodes, run the following command:


    # clresourcegroup add-node -S -n nodelist scal-dg-rg
    
    -n nodelist

    Specifies a comma-separated list of cluster nodes to which you are adding Sun Cluster Support for Oracle RAC.

    scal-dg-rg

    Specifies the name of the resource group to which you are adding nodes.

  6. Mount each shared file system that is to be accessed from the nodes that you are adding.

    If no shared file systems are to be accessed from the nodes that you are adding, omit this step.

    For each file system that you are mounting, type the following command:


    # mount mount-point
    
    mount-point

    Specifies the mount point of the file system that you are mounting.

  7. Add the nodes to any resource groups that contain resources for the Sun StorEdge QFS metadata server.

    If no resource groups that contain resources for the Sun StorEdge QFS metadata server are configured, omit this step.

    For each resource group to which you are adding nodes, run the following command:


    # clresourcegroup add-node -n nodelist qfs-mds-rg
    
    -n nodelist

    Specifies a comma-separated list of cluster nodes to which you are adding Sun Cluster Support for Oracle RAC.

    qfs-mds-rg

    Specifies the name of the resource group to which you are adding nodes.

  8. Bring online all resource groups to which you added nodes in Step 5.

    These resource groups contain scalable device group resources.

    If no resource groups that contain scalable device group resources are configured, omit this step.

    For each resource group that you are bringing online, type the following command:


    # clresourcegroup online scal-dg-rg
    
    scal-dg-rg

    Specifies the name of the resource group that you are bringing online.

  9. (Oracle 10g R2 only) Start the Oracle CRS.

    If you are using Oracle 9i or Oracle 10g R1, omit this step.


    # /etc/init.d/init.crs start
    Startup will be queued to init within 30 seconds.
  10. (Oracle 9i only) Add the nodes to all resource groups that contain logical hostname resources for each RAC database that is to run on the nodes.

    If you are using Oracle 10g R1 or Oracle 10g R2 , omit this step. For Oracle 10g R1 and Oracle 10g R2, no resource groups for logical hostname resources are configured.

    For each resource group to which you are adding nodes, run the following command:


    # clresourcegroup add-node -n nodelist lh-rg
    
    -n nodelist

    Specifies a comma-separated list of cluster nodes to which you are adding Sun Cluster Support for Oracle RAC.

    lh-rg

    Specifies the name of the resource group to which you are adding nodes.

  11. (Oracle 9i and Oracle 10g R2 only) Add the nodes to the resource group for each RAC database that is to run on the nodes.

    If you are using Oracle 10g R1, omit this step. For Oracle 10g R1, no resource groups for RAC databases are configured.

    For each resource group to which you are adding nodes, run the following command:


    # clresourcegroup add-node -S -n nodelist rac-db-rg
    
    -n nodelist

    Specifies a comma-separated list of cluster nodes to which you are adding Sun Cluster Support for Oracle RAC.

    rac-db-rg

    Specifies the name of the resource group to which you are adding nodes.

  12. (Oracle 10g R2 only) For each node that you are adding, create the Oracle CRS resources that are required to represent Sun Cluster resources.

    Create an Oracle CRS resource for each Sun Cluster resource for scalable device groups and scalable file-system mount points on which Oracle components depend. For more information, see How to Create an Oracle CRS Resource for Interoperation With Sun Cluster.

  13. (Oracle 9i and Oracle 10g R2 only) Modify each resource for RAC databases to set a value of each per-node property for each node that you are adding.

    If you are using Oracle 10g R1, omit this step. For Oracle 10g R1, no resource groups for RAC databases are configured.

    For each resource that you are modifying, perform these steps:

    1. Disable the resource.


      # clresource disable rac-db-rs
      
      rac-db-rs

      Specifies the name of the RAC database resource that you are disabling.

    2. Set a value of each per-node property for each node that you are adding.

      The per-node properties of each resource type for RAC databases are shown in the following table.

      Resource Type 

      Properties 

      SUNW.scalable_rac_server_proxy

      oracle_sid

      SUNW.scalable_rac_listener

      listener_name

      SUNW.scalable_rac_server

      alert_log_file

      oracle_sid

      For information about extension properties of resource types for RAC databases, see the following sections:


      # clresource set \
      -p property{node}=value[…]\
      [-p property{node}=value[…]][…]\
      rac-db-rs
      
      property

      Specifies the name of a per-node property that you are setting.

      node

      Specifies the node for which you are setting a value for property.

      value

      Specifies the value to which you are setting property for node.

      rac-db-rs

      Specifies the name of the RAC database resource whose per-node properties you are setting.

    3. Enable the resource.


      # clresource enable rac-db-rs
      
      rac-db-rs

      Specifies the name of the RAC database resource that you are enabling.

  14. (Oracle 9i and Oracle 10g R2 only) Bring online each resource group for RAC databases.

    If you are using Oracle 10g R1, omit this step. For Oracle 10g R1, no resource groups for RAC databases are configured.

    For each resource group that you are bringing online, run the following command:


    # clresourcegroup online rac-db-rg
    
    rac-db-rg

    Specifies the name of the resource group that you are bringing online.


Example 6–3 Adding Sun Cluster Support for Oracle RAC to Selected Nodes

This example shows the sequence of operations that is required to add Sun Cluster Support for Oracle RAC to nodes pclus3 and pclus4 of a four-node cluster.

The configuration of Sun Cluster Support for Oracle RAC in this example is as follows:

The configuration of resource groups in this example is shown in the following table.

Resource Group 

Purpose 

rac-framework-rg

RAC framework resource group. 

scaldg-rg

Resource group for scalable device-group resources. 

qfsmds-rg

Resource group for Sun StorEdge QFS metadata server resources. 

scalmnt-rg

Resource group for scalable file-system mount-point resources. 

rac_server_proxy-rg

RAC database resource group. 

The resource groups that are required for this configuration are shown in Figure A–2.

  1. To add the nodes to the resource group that contains scalable file-system mount-point resources, the following command is run:


    # clresourcegroup add-node -S -n pclus3,pclus4 scalmnt-rg
    
  2. To add the nodes to the RAC framework resource group, the following command is run:


    # clresourcegroup add-node -S -n pclus3,plcus4 rac-framework-rg
    
  3. To add the nodes to the Solaris Volume Manager for Sun Cluster multi-owner disk set oradg, the following command is run:


    # metaset -s oradg -M -a -h  pclus3 pclus4
    
  4. To add the nodes to the resource group that contains scalable device group resources, the following command is run:


    # clresourcegroup add-node -S -n pclus3,pclus4 scaldg-rg
    
  5. To mount the shared file systems that are to be accessed from the nodes that are being added, the following commands are run:


    # mount /db_qfs/OraData
    # mount /db_qfs/OraHome
    
  6. To add the nodes to the resource group that contains resources for the Sun StorEdge QFS metadata server, the following command is run:


    # clresourcegroup add-node -n pclus3,pclus4 qfsmds-rg
    
  7. To bring online the resource group that contains scalable device group resources, the following command is run:


    # clresourcegroup online scaldg-rg
    
  8. To start the Oracle CRS and to verify the correct startup of the Oracle CRS, the following commands are run:


    # /etc/init.d/init.crs start
    Startup will be queued to init within 30 seconds.
    # /db_qfs/OraHome/crs/bin/crsctl check crs
    CSS appears healthy
    CRS appears healthy
    EVM appears healthy
  9. To add the nodes to the resource group for the RAC database, the following command is run:


    # clresourcegroup add-node -S -n pclus3,pclus4 rac_server_proxy-rg
    

    After the addition of the nodes to the resource group for the RAC database, the required Oracle CRS resources are created. The creation of these Oracle CRS resources is beyond the scope of this example.

  10. To set required per-node properties for the RAC database resource, the following commands are run:


    # clresource disable rac_server_proxy-rs
    # clresource set -p oracle_sid\{3\}=swb3 -p \
    oracle_sid\{4\}=swb4 rac_server_proxy-rs
    # clresource enable rac_server_proxy-rs
    

    The per-node property oracle_sid is set to swb3 on node pclus3 and to swb4 on node pclus4.

  11. To bring online the resource group for the RAC database, the following command is run:


    # clresourcegroup online rac_server_proxy-rg