Sun Cluster 3.1 10/03 System Administration Guide

Shutting Down and Booting a Cluster Overview

The Sun Cluster scshutdown(1M) command stops cluster services in an orderly fashion and cleanly shuts down the entire cluster. You might do use the scshutdown command when moving the location of a cluster. You can also use the command to shut down the cluster if you have data corruption caused by an application error.


Note –

Use the scshutdown command instead of the shutdown or halt commands to ensure proper shutdown of the entire cluster. The Solaris shutdown command is used with the scswitch(1M) command to shut down individual nodes. See How to Shut Down a Cluster or Shutting Down and Booting a Single Cluster Node for more information.


The scshutdown command stops all nodes in a cluster by:

  1. Taking all running resource groups offline.

  2. Unmounting all cluster file systems.

  3. Shutting down active device services.

  4. Running init 0 and bringing all nodes to the OBP ok prompt.


Note –

If necessary, you can boot a node in non-cluster mode so that the node does not participate in cluster membership. Non-cluster mode is useful when installing cluster software or for performing certain administrative procedures. See How to Boot a Cluster Node in Non-Cluster Mode for more information.


Table 3–1 Task List: Shutting Down and Booting a Cluster

Task 

For Instructions 

Stop the cluster 

    -Use scshutdown(1M)

How to Shut Down a Cluster

Start the cluster by booting all nodes.  

The nodes must have a working connection to the cluster interconnect to attain cluster membership. 

How to Boot a Cluster

Reboot the cluster 

    - Use scshutdown

At the ok prompt, boot each node individually with the boot(1M) command.

The nodes must have a working connection to the cluster interconnect to attain cluster membership. 

How to Reboot a Cluster

How to Shut Down a Cluster


Caution – Caution –

Do not use send brk on a cluster console to shut down a cluster node. The command is not supported within a cluster. If you use send brk with the go at the ok prompt to reboot, the node panics.


  1. If your cluster is running Oracle® Parallel Server or Real Application Clusters, shut down all instances of the database.

    Refer to the Oracle Parallel Server/Real Application Clusters product documentation for shutdown procedures.

  2. Become superuser on any node in the cluster.

  3. Shut down the cluster immediately to OBP.

    From a single node in the cluster, type the following command.


    # scshutdown -g0 -y
    

  4. Verify that all nodes have reached the ok prompt.

    Do not power off any nodes until all cluster nodes are at the ok prompt.

  5. If necessary, power off the nodes.

Example—Shutting Down a Cluster

The following example shows the console output when stopping normal cluster operation and bringing down all nodes to the ok prompt. The -g 0 option sets the shutdown grace period to zero, -y provides an automatic yes response to the confirmation question. Shutdown messages also appear on the consoles of the other nodes in the cluster.


# scshutdown -g0 -y
May 2 10:08:46 phys-schost-1 cl_runtime: 
WARNING: CMM monitoring disabled.
phys-schost-1# 
INIT: New run level: 0
The system is coming down.  Please wait.
System services are now being stopped.
/etc/rc0.d/K05initrgm: Calling scswitch -S (evacuate)
The system is down.
syncing file systems... done
Program terminated
ok 

Where to Go From Here

See How to Boot a Cluster to restart a cluster that has been shut down.

How to Boot a Cluster

  1. To start a cluster whose nodes have been shut down and are at the ok prompt, boot(1M) each node.

    If you make configuration changes between shutdowns, start the node with the most current configuration first. Except in this situation, the boot order of the nodes does not matter.


    ok boot
    

    Messages are displayed on the booted nodes' consoles as cluster components are activated.


    Note –

    Cluster nodes must have a working connection to the cluster interconnect to attain cluster membership.


  2. Verify that the nodes booted without error and are online.

    The scstat(1M) command reports the nodes' status.


    # scstat -n
    


    Note –

    If a cluster node's /var file system fills up, Sun Cluster might not be able to restart on that node. If this problem arises, see How to Repair a Full /var File System.


Example—Booting a Cluster

The following example shows the console output when booting node phys-schost-1 into the cluster. Similar messages appear on the consoles of the other nodes in the cluster.


ok boot
Rebooting with command: boot 
...
Hostname: phys-schost-1
Booting as part of a cluster
NOTICE: Node 1 with votecount = 1 added.
NOTICE: Node 2 with votecount = 1 added.
NOTICE: Node 3 with votecount = 1 added.
...
NOTICE: Node 1: attempting to join cluster
...
NOTICE: Node 2 (incarnation # 937690106) has become reachable.
NOTICE: Node 3 (incarnation # 937690290) has become reachable.
NOTICE: cluster has reached quorum.
NOTICE: node 1 is up; new incarnation number = 937846227.
NOTICE: node 2 is up; new incarnation number = 937690106.
NOTICE: node 3 is up; new incarnation number = 937690290.
NOTICE: Cluster members:   1  2  3
...

How to Reboot a Cluster

Run the scshutdown(1M) command to shut down the cluster, then boot the cluster with the boot(1M) command on each node.

  1. (Optional). For a cluster that is running Oracle Parallel Server/Real Application Clusters, shut down all instances of the database.

    Refer to the Oracle Parallel Server/Real Application Clusters product documentation for shutdown procedures.

  2. Become superuser on any node in the cluster.

  3. Shut down the cluster to OBP.

    From a single node in the cluster, type the following command.


    # scshutdown -g0 -y 
    

    Each node is shut down to the ok prompt.


    Note –

    Cluster nodes must have a working connection to the cluster interconnect to attain cluster membership.


  4. Boot each node.

    The order in which the nodes are booted does not matter unless you make configuration changes between shutdowns. If you make configuration changes between shutdowns, start the node with the most current configuration first.


    ok boot
    

    Messages appear on the booted nodes' consoles as cluster components are activated.

  5. Verify that the nodes booted without error and are online.

    The scstat command reports the nodes' status.


    # scstat -n
    


    Note –

    If a cluster node's /var file system fills up, Sun Cluster might not be able to restart on that node. If this problem arises, see How to Repair a Full /var File System.


Example—Rebooting a Cluster

The following example shows the console output when stopping normal cluster operation, bringing down all nodes to the ok prompt, then restarting the cluster. The -g 0 option sets the grace period to zero, -y provides an automatic yes response to the confirmation question. Shutdown messages also appear on the consoles of other nodes in the cluster.


# scshutdown -g0 -y
May 2 10:08:46 phys-schost-1 cl_runtime: 
WARNING: CMM monitoring disabled.
phys-schost-1# 
INIT: New run level: 0
The system is coming down.  Please wait.
...
The system is down.
syncing file systems... done
Program terminated
ok boot
Rebooting with command: boot 
...
Hostname: phys-schost-1
Booting as part of a cluster
...
NOTICE: Node 1: attempting to join cluster
...
NOTICE: Node 2 (incarnation # 937690106) has become reachable.
NOTICE: Node 3 (incarnation # 937690290) has become reachable.
NOTICE: cluster has reached quorum.
...
NOTICE: Cluster members:   1  2  3
...
NOTICE: Node 1: joined cluster
...
The system is coming up.  Please wait.
checking ufs filesystems
...
reservation program successfully exiting
Print services started.
volume management starting.
The system is ready.
phys-schost-1 console login:
NOTICE: Node 1: joined cluster
...
The system is coming up.  Please wait.
checking ufs filesystems
...
reservation program successfully exiting
Print services started.
volume management starting.
The system is ready.
phys-schost-1 console login: