Sun Cluster System Administration Guide for Solaris OS

Chapter 6 Administering Quorum

This chapter provides the procedures for administering quorum devices within Sun Cluster and Sun Cluster quorum servers. For information about quorum concepts, see Quorum and Quorum Devices in Sun Cluster Concepts Guide for Solaris OS.

Administering Quorum Devices

A quorum device is a shared storage device or quorum server that is shared by two or more nodes and that contributes votes that are used to establish a quorum. This section provides the procedures for administering quorum devices.

You can use the clquorum(1CL) command to perform all quorum device administrative procedures. In addition, you can accomplish some procedures by using the clsetup(1CL) interactive utility or the Sun Cluster Manager GUI. Whenever possible, quorum procedures are described in this section by using the clsetup utility. The Sun Cluster Manager online help describes how to perform quorum procedures by using the GUI. When you work with quorum devices, keep in mind the following guidelines:


Note –

The clsetup command is an interactive interface to the other Sun Cluster commands. When clsetup runs, the command generates the appropriate specific commands, in this case clquorum commands. These generated commands are shown in the examples at the end of the procedures.


To view the quorum configuration, use clquorum show. The clquorum list command displays the names of quorum devices in the cluster. The clquorum status command provides status and vote count information.

Most examples shown in this section are from a three-node cluster.

Table 6–1 Task List: Administering Quorum

Task 

For Instructions 

Add a quorum device to a cluster by using clsetup(1CL)

Adding a Quorum Device

Remove a quorum device from a cluster by using clsetup (to generate clquorum)

How to Remove a Quorum Device

Remove the last quorum device from a cluster by using clsetup (to generate clquorum)

How to Remove the Last Quorum Device From a Cluster

Replace a quorum device in a cluster by using the add and remove procedures 

How to Replace a Quorum Device

Modify a quorum device list by using the add and remove procedures 

How to Modify a Quorum Device Node List

Put a quorum device into maintenance state by using clsetup (to generate clquorum)

(While in maintenance state, the quorum device does not participate in voting to establish the quorum.) 

How to Put a Quorum Device Into Maintenance State

Reset the quorum configuration to its default state by using clsetup (to generate clquorum)

How to Bring a Quorum Device Out of Maintenance State

List the quorum devices and vote counts by using the clquorum(1CL) command

How to List the Quorum Configuration

Dynamic Reconfiguration With Quorum Devices

You must consider a few issues when completing dynamic reconfiguration (DR) operations on quorum devices in a cluster.

To remove a quorum device, you must complete the following steps, in the order indicated.

Table 6–2 Task Map: Dynamic Reconfiguration With Quorum Devices

Task 

For Instructions 

1. Enable a new quorum device to replace the one being removed. 

Adding a Quorum Device

2. Disable the quorum device to be removed. 

How to Remove a Quorum Device

3. Perform the DR remove operation on the device being removed. 

Sun Enterprise 10000 DR Configuration Guide and the Sun Enterprise 10000 Dynamic Reconfiguration Reference Manual (from the Solaris 9 on Sun Hardware, and Solaris 10 on Sun Hardware collections.)

Adding a Quorum Device

This section provides procedures to add a quorum device. For information about determining the number of quorum vote counts necessary for your cluster, recommended quorum configurations, and failure fencing, see Quorum and Quorum Devices in Sun Cluster Concepts Guide for Solaris OS.


Caution – Caution –

Do not add a disk that is currently configured as a quorum device to a Solaris ZFS storage pool. When a configured quorum device is added to a Solaris ZFS storage pool, the disk is relabeled as an EFI disk and quorum configuration information is lost and the disk no longer provides a quorum vote to the cluster. Once a disk is in a storage pool, that disk can then be configured as a quorum device. Or, you can unconfigure the disk, add it to the storage pool, then reconfigure the disk as a quorum device.


The Sun Cluster software supports the following types of quorum devices: SCSI, Network Appliance (NetApp) NAS, and Sun Cluster Quorum Server. Procedures for adding these devices are provided in the following sections:


Note –

You cannot configure replicated disks as quorum devices. If you attempt to add a replicated disk as a quorum device, you will receive the following error message, after which the command exits with an error code.


Disk-name is a replicated device. Replicated devices cannot be
configured as quorum devices.

You can also accomplish these procedures by using the Sun Cluster Manager GUI. See the Sun Cluster Manager online help for more information.

See the clsetup(1CL) and clquorum(1CL) man pages for information about the commands used in the following procedures.

ProcedureHow to Add a SCSI Quorum Device

To complete this procedure, identify a disk drive by its device ID (DID), which is shared by the nodes. Use the cldevice show command to see the list of DID names. Refer to the cldevice(1CL) man page for additional information.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  2. Start the clsetup utility.


    # clsetup
    

    The clsetup Main Menu is displayed.

  3. Type the number that corresponds to the option for Quorum.

    The Quorum Menu is displayed.

  4. Type the number that corresponds to the option for adding a quorum device, then type yes when the clsetup utility asks you to confirm the quorum device that you are adding.

    The clsetup utility asks what type of quorum device you want to add.

  5. Type the number that corresponds to the option for a SCSI quorum device.

    The clsetup utility asks which global device you want to use.

  6. Type the global device you are using.

    The clsetup utility asks you to confirm that the new quorum device should be added to the global device you specified.

  7. Type yes to continue adding the new quorum device.

    If the new quorum device is added successfully, the clsetup utility displays a message to that effect.

  8. Verify that the quorum device has been added.


    # clquorum list -v
    

Example 6–1 Adding a SCSI Quorum Device

The following example shows the clquorum command generated by clsetup when it adds a SCSI quorum device and a verification step.


Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization 
on any cluster node.

[Start the clsetup utility:]
# clsetup
[Select Quorum>Add a quorum device]
[Answer the questions when prompted.]
[You will need the following information.]
  [Information:              Example:]
  [SCSI device               scsi]
  [Global device		       d20]

[Verify that the clquorum command was completed successfully:]
 clquorum add d20
 
    Command completed successfully.
[Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is added:]
# clquorum list -v
 
Quorums         Type
-------         ----
d20             scsi
scphyshost-1    node
scphyshost-2    node

ProcedureHow to Add a Sun NAS Quorum Device

To complete this procedure, identify a disk drive by its device ID (DID), which is shared by the nodes. Use the cldevice show command to see the list of DID names. Refer to the cldevice(1CL) man page for additional information.


Note –

Sun Cluster supports a Sun NAS quorum device only on a two-node cluster configuration.


This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

  1. Use the Sun NAS GUI to set up an iSCSI device on the Sun NAS filer.

    1. Create a file volume that is approximately 50 Mbytes in size.


      File Volume Operations -> Create File Volume
    2. For each node, create an iSCSI access list.


      iSCSI Configuration -> Configure Access List
      1. Use the name of the cluster as the iSCSI access list name.

      2. Add the initiator node name of each cluster node to the access list. CHAP and IQN are not needed.

    3. Configure the iSCSI LUN


      iSCSI Configuration -> Configure iSCSI LUN

      You can use the name of the backing file volume as the name of the LUN. Add the access list for each node to the LUN.

  2. On each of the cluster nodes, discover the iSCSI LUN and set the iSCSI access list to static configuration.


    # iscsiadm modify discovery -s enable
    
    	# iscsiadm list discovery
    	Discovery:
    	        Static: enabled
    	        Send Targets: disabled
    	        iSNS: disabled
    
    	#  iscsiadm add static-config iqn.LUNName,IPAddress_of_NASDevice
    # devfsadm -i iscsi
    # cldevice refresh
    
  3. From one cluster node, configure the DIDs for the iSCSI LUN.


    # /usr/cluster/bin/scgdevs
  4. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  5. Start the clsetup utility.


    # clsetup
    

    The clsetup Main Menu is displayed.

  6. Type the number that corresponds to the option for Quorum.

    The Quorum Menu is displayed.

  7. Type the number that corresponds to the option for adding a quorum device, then type yes when the clsetup utility asks you to confirm the quorum device that you are adding.

    The clsetup utility asks what type of quorum device you want to add.

  8. Type the number that corresponds to the option for a SCSI quorum device.

    The clsetup utility asks which global device you want to use.

  9. Type the global device you are using.

    The clsetup utility asks you to confirm that the new quorum device should be added to the global device you specified.

  10. Type yes to continue adding the new quorum device.

    If the new quorum device is added successfully, the clsetup utility displays a message to that effect.

  11. Verify that the quorum device has been added.


    # clquorum list -v
    

Example 6–2 Adding a Sun NAS Quorum Device

The following example shows the clquorum command generated by clsetup when it adds a Sun NAS quorum device and a verification step.


Add an iSCSI device on the Sun NAS filer.
Use the Sun NAS GUI to create a file volume that is approximately 50mb in size.
File Volume Operations -> Create File Volume
For each node, create an iSCSI access list.
iSCSI Configuration -> Configure Access List
Add the initiator node name of each cluster node to the access list. 
*** Need GUI or command syntax for this step. ***
Configure the iSCSI LUN
iSCSI Configuration -> Configure iSCSI LUN
On each of the cluster nodes, discover the iSCSI LUN and set the iSCSI access list to static configuration.
iscsiadm modify discovery -s enable
iscsiadm list discovery
Discovery:
   Static: disable
   Send Targets: enables
   iSNS: disabled
iscsiadm add status-config
iqn.1986-03.com.sun0-1:000e0c66efe8.4604DE16.thinquorum,10.11.160.20
devsadm -i iscsi
From one cluster node, configure the DID devices for the iSCSI LUN.
/usr/cluster/bin/scgdevs
Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization 
on any cluster node.

[Start the clsetup utility:]
# clsetup
[Select Quorum>Add a quorum device]
[Answer the questions when prompted.]
[You will need the following information.]
  [Information:              Example:]
  [SCSI device               scsi]
  [Global device		       d20]

[Verify that the clquorum command was completed successfully:]
 clquorum add d20
 
    Command completed successfully.
[Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is added:]
# clquorum list -v
 
Quorums         Type
-------         ----
d20             scsi
scphyshost-1    node
scphyshost-2    node

ProcedureHow to Add a Network Appliance Network-Attached Storage (NAS) Quorum Device

When you use a Network Appliance (NetApp) network-attached storage (NAS) device as a quorum device, the following are required:

See the following Sun Cluster documentation for information about installing a NetApp NAS storage device in a Sun Cluster environment: Sun Cluster 3.1 - 3.2 With Network-Attached Storage Devices Manual for Solaris OS.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

  1. Make sure that all Sun Cluster nodes are online and can communicate with the NetApp clustered filer.

  2. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  3. Start the clsetup utility.


    # clsetup
    

    The clsetup Main Menu is displayed.

  4. Type the number that corresponds to the option for Quorum.

    The Quorum Menu is displayed.

  5. Type the number that corresponds to the option for adding a quorum device. Then type yes to confirm that you are adding a quorum device.

    The clsetup utility asks what type of quorum device you want to add.

  6. Type the number that corresponds to the option for a netapp_nas quorum device. Then type yes to confirm that you are adding a netapp_nas quorum device.

    The clsetup utility asks you provide the name of the new quorum device.

  7. Type the name of the quorum device you are adding.

    The quorum device name can be any name you choose. The name is only used to process future administrative commands.

    The clsetup utility asks you to provide the name of the filer for the new quorum device.

  8. Type the name of the filer of the new quorum device.

    This name is the network-accessible name or address of the filer.

    The clsetup utility asks you to provide the LUN ID for the filer.

  9. Type the ID of the quorum device LUN on the filer.

    The clsetup utility asks if to the new quorum device should be added on the filer.

  10. Type yes to continue adding the new quorum device.

    If the new quorum device is added successfully, the clsetup utility displays a message to that effect.

  11. Verify that the quorum device has been added.


    # clquorum list -v
    

Example 6–3 Adding a NetApp NAS Quorum Device

The following example shows the clquorum command generated by clsetup when it adds a NetApp NAS quorum device. The example also shows a verification step.


Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any cluster node.

[Start the clsetup utility:]
# clsetup
[Select Quorum>Add a quorum device]
[Answer the questions when prompted.]
[You will need the following information.]
  [Information:                Example:]
  [Quorum Device               Netapp_nas quorum device]
  [Name:        		         qd1]
  [Filer:                      nas1.sun.com]
  [LUN ID:                     0]

[Verify that the clquorum command was completed successfully:]
 clquorum add -t netapp_nas -p filer=nas1.sun.com,-p lun_id=0 qd1
 
    Command completed successfully.
[Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is added:]
# clquorum list -v
 
Quorums         Type
-------         ----
qd1             netapp_nas
scphyshost-1    node
scphyshost-2    node

ProcedureHow to Add a Quorum Server Quorum Device

Before You Begin

Before you can add a Sun Cluster Quorum Server as a quorum device, the Sun Cluster Quorum Server software must be installed on the host machine and the quorum server must be started and running. For information about installing and starting the quorum server, see the Sun Cluster Quorum Server User’s Guide.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  2. Make sure that all Sun Cluster nodes are online and can communicate with the Sun Cluster Quorum Server.

    1. Ensure that network switches that are directly connected to cluster nodes meet one of the following criteria:

      • The switch supports Rapid Spanning Tree Protocol (RSTP).

      • Fast port mode is enabled on the switch.

      One of these features is required to ensure immediate communication between cluster nodes and the quorum server. If this communication is significantly delayed by the switch, the cluster interprets this prevention of communication as loss of the quorum device.

    2. If the public network uses variable-length subnetting, also called Classless Inter-Domain Routing (CIDR), modify the following files on each node.

      If you use classful subnets, as defined in RFC 791, you do not need to perform these steps.

      1. Add to the /etc/inet/netmasks file an entry for each public subnet that the cluster uses.

        The following is an example entry which contains a public-network IP address and netmask:


        10.11.30.0	255.255.255.0
      2. Append netmask + broadcast + to the hostname entry in each /etc/hostname.adapter file.


        nodename netmask + broadcast +
        
    3. On each node in the cluster, add the quorum server host name to the /etc/inet/hosts file or the /etc/inet/ipnodes file.

      Add a host name-to-address mapping to the file, such as the following.


      ipaddress qshost1
      
      ipaddress

      The IP address of the computer where the quorum server is running.

      qshost1

      The host name of the computer where the quorum server is running.

    4. If you use a naming service, add the quorum server host's name-to-address mapping to the name-service database.

  3. Start the clsetup utility.


    # clsetup
    

    The clsetup Main Menu is displayed.

  4. Type the number that corresponds to the option for Quorum.

    The Quorum Menu is displayed.

  5. Type the number that corresponds to the option for adding a quorum device. Then type yes to confirm that you are adding a quorum device.

    The clsetup utility asks what type of quorum device you want to add.

  6. Type the number that corresponds to the option for a quorum-server quorum device. Then type yes to confirm that you are adding a quorum-server quorum device.

    The clsetup utility asks you to provide the name of the new quorum device.

  7. Type the name of the quorum device you are adding.

    The quorum device name can be any name you choose. The name is only used to process future administrative commands.

    The clsetup utility asks you to provide the name of the filer for the new quorum device.

  8. Type the name of the host of the quorum server.

    This name specifies the IP address of the machine where the quorum server runs or the host name of the machine on the network.

    Depending on the IPv4 or IPv6 configuration of the host, the IP address of the machine must be specified in the /etc/hosts file, the /etc/inet/ipnodes file, or both.


    Note –

    The machine you specify must be reachable by all cluster nodes and must run the quorum server.


    The clsetup utility asks you to provide the port number of the quorum server.

  9. Type the port number that is used by the quorum server to communicate with the cluster nodes.

    The clsetup utility asks you to confirm that the new quorum device should be added.

  10. Type yes to continue adding the new quorum device.

    If the new quorum device is added successfully, the clsetup utility displays a message to that effect.

  11. Verify that the quorum device has been added.


    # clquorum list -v
    

Example 6–4 Adding a Quorum Server Quorum Device

The following example shows the clquorum command that is generated by clsetup when it adds a quorum server quorum device. The example also shows a verification step.


Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on 
any cluster node.

[Start the clsetup utility:]
# clsetup
[Select Quorum>Add a quorum device]
[Answer the questions when prompted.]
[You will need the following information.]
  [Information:                Example:]
  [Quorum Device               quorum_server quorum device]
  [Name:        		         qd1]
  [Host Machine Name:          10.11.124.84]
  [Port Number:                9001]

[Verify that the clquorum command was completed successfully:]
 clquorum add -t quorum_server -p qshost=10.11.124.84,-p port=9001 qd1
 
    Command completed successfully.
[Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is added:]
# clquorum list -v
 
Quorums         Type
-------         ----
qd1             quorum_server
scphyshost-1    node
scphyshost-2    node
 
-- Quorum Summary --
 
  Quorum votes possible: 5
  Quorum votes needed:   3
  Quorum votes present:  5
 
-- Quorum Votes by Node --
 
                    Node Name           Present Possible Status
                    ---------           ------- -------- ------
  Node votes:       phys-schost-1       1        1       Online
  Node votes:       phys-schost-2       1        1       Online
 
-- Quorum Votes by Device --
 
                   Device Name         Present Possible Status
                   -----------         ------- -------- ------
 Device votes:     qd1                  1        1       Online
 Device votes:     /dev/did/rdsk/d3s2   1        1       Online
 Device votes:     /dev/did/rdsk/d4s2   1        1       Online

Removing or Replacing a Quorum Device

This section provides the following procedures for removing or replacing a quorum device:

ProcedureHow to Remove a Quorum Device

You can also accomplish this procedure by using the Sun Cluster Manager GUI. See the Sun Cluster Manager online help for more information.

When a quorum device is removed, it no longer participates in the voting to establish quorum. Note that all two-node clusters require that at least one quorum device be configured. If this is the last quorum device on a cluster, clquorum(1CL) will fail to remove the device from the configuration.


Note –

If the device you intend to remove is the last quorum device in the cluster, see the procedure How to Remove the Last Quorum Device From a Cluster.


This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node in the cluster.

  2. Determine the quorum device to be removed.


    # clquorum list -v
    
  3. Execute the clsetup(1CL) utility.


    # clsetup
    

    The Main Menu is displayed.

  4. Type the number that corresponds to the option for Quorum.

  5. Type the number that corresponds to the option to remove a quorum device.

    Answer the questions displayed during the removal process.

  6. Quit clsetup.

  7. Verify that the quorum device is removed.


    # clquorum list -v
    

Example 6–5 Removing a Quorum Device

This example shows how to remove a quorum device from a cluster with two or more quorum devices configured.


Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on 
any cluster node.

[Determine the quorum device to be removed:]
# clquorum list -v
[Start the clsetup utility:]
# clsetup
[Select Quorum>Remove a quorum device]
[Answer the questions when prompted.] 
[Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is removed:]
# clquorum list -v
 
 Quorums         Type
-------         ----
scphyshost-1    node
scphyshost-2    node
scphyshost-3    node

Troubleshooting

If you lose communications between the cluster and the quorum server host while removing a quorum server quorum device, you must clean up stale configuration information on the quorum server host. For instructions on performing this cleanup, see the Sun Cluster Quorum Server User’s Guide.

ProcedureHow to Remove the Last Quorum Device From a Cluster

This procedure removes the last quorum device from a cluster. This procedure is only required if you have a two node cluster and you are removing one of the nodes. If the device you intend to remove is not the last quorum device in a two-node cluster, use the previous procedure, How to Remove a Quorum Device.


Note –

All two-node clusters require at least one configured quorum device. If this is the last quorum device on a two-node cluster, the cluster must be placed into install mode before clquorum(1CL) enables you to remove the device from the configuration. This procedure should only be performed if a node is being removed from the cluster.


This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node in the cluster.

  2. Place the node to be removed in maintenance state.

    See How to Put a Node Into Maintenance State.

  3. Place the cluster in install mode.


    # cluster set -p installmode=enabled
    
  4. Remove the quorum device by using the clquorum command.

    The clsetup(1CL) cluster-administration menu options are not available while the cluster is in install mode.


    # clquorum remove qd1
    
  5. Verify that the quorum device has been removed.


    # clquorum list -v
    

Example 6–6 Removing the Last Quorum Device

This example shows how to remove the last remaining quorum device in a cluster configuration.


[Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on 
any cluster node.]
[Place the cluster in install mode:]
# cluster set -p installmode=enabled
[Remove the quorum device:]
# clquorum remove d3
[Verify that the quorum device has been removed:]
# clquorum list -v
 Quorums         Type
-------         ----
scphyshost-1    node
scphyshost-2    node
scphyshost-3    node

ProcedureHow to Replace a Quorum Device

Use this procedure to replace an existing quorum device with another quorum device. You can replace a quorum device with a similar device type, such as replacing a NAS device with another NAS device, or you can replace the device with a dissimilar device, such as replacing a NAS device with a shared disk.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

  1. Configure a new quorum device.

    You need to first add a new quorum device to the configuration to take the place of the old device. See Adding a Quorum Device to add a new quorum device to the cluster.

  2. Remove the device that you are replacing as a quorum device.

    See How to Remove a Quorum Device to remove the old quorum device from the configuration.

  3. If the quorum device is a failed disk, replace the disk.

    Refer to the hardware procedures for your disk enclosure in the Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS.

Maintaining Quorum Devices

This section provides the following procedures for maintaining quorum devices:

ProcedureHow to Modify a Quorum Device Node List

You can use the clsetup(1CL) utility to add a node to or remove a node from the node list of an existing quorum device. To modify a quorum device's node list, you must remove the quorum device, modify the physical connections of nodes to the quorum device you removed, then add the quorum device to the cluster configuration again. When a quorum device is added, clquorum(1CL) automatically configures the node-to-disk paths for all nodes attached to the disk.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  2. Determine the name of the quorum device you are modifying.


    # clquorum list -v
    
  3. Start the clsetup utility.


    # clsetup
    

    The Main Menu is displayed.

  4. Type the number that corresponds to the Quorum option.

    The Quorum Menu is displayed.

  5. Type the number that corresponds to the option to remove a quorum device.

    Follow the instructions. You will be asked the name of the disk to be removed.

  6. Add or delete the physical node connections to the quorum device.

  7. Type the number that corresponds to the option to add a quorum device.

    Follow the instructions. You will be asked the name of the disk to be used as the quorum device.

  8. Verify that the quorum device has been added.


    # clquorum list -v
    

Example 6–7 Modifying a Quorum Device Node List

The following example shows how to use the clsetup utility to add nodes to or delete nodes from a quorum device node list. In this example, the quorum device name is d2, and the final result of the procedures adds another node to the quorum device node list.


[Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on 
any node in the cluster.]

[Determine the quorum device name:]
# clquorum list -v
Quorums            Type
-------            -----
d2                 scsi
sc-phys-schost-1   node
sc-phys-schost-2   node
sc-phys-schost-3   node

[Start the clsetup utility:]
# clsetup

[Type the number that corresponds with the quorum option.]
.
[Type the number that corresponds with the option to remove a quorum device.]
.
[Answer the questions when prompted.]
[You will need the following information:]

   Information:			Example:
   Quorum Device Name: 	d2

[Verify that the clquorum command completed successfully:]
 clquorum remove d2
     Command completed successfully.

[Type the number that corresponds with the Quorum option.]
.
[Type the number that corresponds with the option to add a quorum device.]
.
[Answer the questions when prompted.]
[You will need the following information:]

   Information         Example:
   quorum device name     d2

[Verify that the clquorum command was completed successfully:clquorum add d2
     Command completed successfully.

Quit the clsetup utility.

[Verify that the correct nodes have paths to the quorum device. 
In this example, note that phys-schost-3 has been added to the 
enabled hosts list.]
# clquorum show d2 | grep Hosts
=== Quorum Devices ===

Quorum Device Name:		d2
   Hosts (enabled):		phys-schost-1, phys-schost-2, phys-schost-3

[Verify that the modified quorum device is online.]
# clquorum status d2
=== Cluster Quorum ===

--- Quorum Votes by Device ---

Device Name       Present      Possible      Status
-----------       -------      --------      ------
d2                1            1             Online[Verify the quorum device is removed:]
# clquorum list -v
Quorums            Type
-------            -----
sc-phys-schost-1   node
sc-phys-schost-2   node
sc-phys-schost-3   node

ProcedureHow to Put a Quorum Device Into Maintenance State

Use the clquorum(1CL) command to put a quorum device into maintenance state. The clsetup(1CL) utility does not currently have this capability. You can also accomplish this procedure by using the Sun Cluster Manager GUI. See the Sun Cluster Manager online help for more information.

Put a quorum device into maintenance state when taking the quorum device out of service for an extended period of time. This way, the quorum device's quorum vote count is set to zero and does not contribute to the quorum count while the device is being serviced. While in maintenance state, the quorum device's configuration information is preserved.


Note –

All two-node clusters require at least one configured quorum device. If this is the last quorum device on a two-node cluster, clquorum will fail to put the device into maintenance state.


To put a cluster node into maintenance state, see How to Put a Node Into Maintenance State.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  2. Put the quorum device into the maintenance state.


    # clquorum disable device
    
    device

    Specifies the DID name of the disk device to change, for example, d4.

  3. Verify that the quorum device is now in maintenance state.

    The output for the device you placed in maintenance state should read zero for the Quorum Device Votes.


    # clquorum status device
    

Example 6–8 Putting a Quorum Device Into Maintenance State

The following example shows how to put a quorum device into maintenance state and how to verify the results.


# clquorum disable d20
# clquorum status d20
  
=== Cluster Quorum ===

--- Quorum Votes by Device ---

Device Name       Present      Possible      Status
-----------       -------      --------      ------
d20                1            1             Offline

See Also

To re-enable the quorum device, see How to Bring a Quorum Device Out of Maintenance State.

To put a node into maintenance state, see How to Put a Node Into Maintenance State.

ProcedureHow to Bring a Quorum Device Out of Maintenance State

Run this procedure each time a quorum device is in maintenance state and you want to bring the quorum device out of maintenance state and reset the quorum vote count to the default.


Caution – Caution –

If you do not specify either the globaldev or node options, the quorum count is reset for the entire cluster.


When you configure a quorum device, Sun Cluster software assigns the quorum device a vote count of N-1 where N is the number of connected votes to the quorum device. For example, a quorum device that is connected to two nodes with nonzero vote counts has a quorum count of one (two minus one).

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  2. Reset the quorum count.


    # clquorum enable device
    
    device

    Specifies the DID name of the quorum device to reset, for example, d4.

  3. If you are resetting the quorum count because a node was in maintenance state, reboot the node.

  4. Verify the quorum vote count.


    # clquorum show +
    

Example 6–9 Resetting the Quorum Vote Count (Quorum Device)

The following example resets the quorum count for a quorum device back to the default and verifies the result.


# clquorum enable d20
# clquorum show +
  
=== Cluster Nodes ===                          

Node Name:                                      phys-schost-2
  Node ID:                                        1
  Quorum Vote Count:                              1
  Reservation Key:                                0x43BAC41300000001

Node Name:                                      phys-schost-3
  Node ID:                                        2
  Quorum Vote Count:                              1
  Reservation Key:                                0x43BAC41300000002


=== Quorum Devices ===                         

Quorum Device Name:                             d3
  Enabled:                                        yes
  Votes:                                          1
  Global Name:                                    /dev/did/rdsk/d20s2
  Type:                                           scsi
  Access Mode:                                    scsi2
  Hosts (enabled):                                phys-schost-2, phys-schost-3

ProcedureHow to List the Quorum Configuration

You can also accomplish this procedure by using the Sun Cluster Manager GUI. See the Sun Cluster Manager online help for more information.

You do not need to be superuser to list the quorum configuration. You can assume any role that provides solaris.cluster.read RBAC authorization.


Note –

When you increase or decrease the number of node attachments to a quorum device, the quorum vote count is not automatically recalculated. You can reestablish the correct quorum vote if you remove all quorum devices and then add them back into the configuration. For a two-node cluster, temporarily add a new quorum device before you remove and add back the original quorum device. Then remove the temporary quorum device.


This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

  1. Use clquorum(1CL) to list the quorum configuration.


    % clquorum show +
    

Example 6–10 Listing the Quorum Configuration


% clquorum show +

=== Cluster Nodes ===                          

Node Name:                                      phys-schost-2
  Node ID:                                        1
  Quorum Vote Count:                              1
  Reservation Key:                                0x43BAC41300000001

Node Name:                                      phys-schost-3
  Node ID:                                        2
  Quorum Vote Count:                              1
  Reservation Key:                                0x43BAC41300000002


=== Quorum Devices ===                         

Quorum Device Name:                             d3
  Enabled:                                        yes
  Votes:                                          1
  Global Name:                                    /dev/did/rdsk/d20s2
  Type:                                           scsi
  Access Mode:                                    scsi2
  Hosts (enabled):                                phys-schost-2, phys-schost-3

ProcedureHow to Repair a Quorum Device

Use this procedure to replace a malfunctioning quorum device.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

  1. Remove the disk device that you are replacing as a quorum device.


    Note –

    If the device you intend to remove is the last quorum device, you might want to first add another disk as a new quorum device. This step assures a valid quorum device if a failure occurs during the replacement procedure. See Adding a Quorum Device to add a new quorum device.


    See How to Remove a Quorum Device to remove a disk device as a quorum device.

  2. Replace the disk device.

    To replace the disk device, see the hardware procedures for the disk enclosure in the Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS.

  3. Add the replaced disk as a new quorum device.

    See Adding a Quorum Device to add a disk as a new quorum device.


    Note –

    If you added an additional quorum device in Step 1, it is now safe to remove it. See How to Remove a Quorum Device to remove the quorum device.


Administering Sun Cluster Quorum Servers

Sun Cluster Quorum Server provides a quorum device that is not a shared storage device. This section provides procedure for administering Sun Cluster quorum servers, including:

For information about installing and configuring Sun Cluster quorum servers, see How to Install and Configure Quorum Server Software in Sun Cluster Software Installation Guide for Solaris OS.

Overview of the Quorum Server Configuration File

When you install the Sun Cluster software, a default configuration file, /etc/scqsd/scqsd.conf, is created that contains information about a single default quorum server. Each line in the /etc/scqsd/scqsd.conf file has the following format:


/usr/cluster/lib/sc/scqsd [-d quorumdirectory] [-i instancename] -p port
/usr/cluster/lib/sc/scqsd

The full path to where you installed the Sun Cluster software. This value must be /usr/cluster/lib/sc/scqsd.

-d quorumdirectory

The path to the directory where the quorum server can store quorum data.

The quorum server process creates one file per cluster in this directory to store cluster-specific quorum information. By default, the value of this option is /var/scqsd. This directory must be unique for each quorum server that you configure.

-i instancename

A unique name that you choose for the quorum server instance.

-p port

The port number on which the quorum server listens for requests from the cluster. The default port is 9000.

Instance names are optional. If you specify a name for the quorum server, that name must be unique among all quorum servers in your system. If you choose to omit the instance name option, you must refer to the quorum server by the port on which it listens.

Starting and Stopping the Sun Cluster Quorum Server Software

These procedures describe how to start and stop the Sun Cluster software.

By default, these procedures start and stop a single default quorum server unless you have customized the content of the quorum server configuration file, /etc/scqsd/scqsd.conf. The default quorum server is bound on port 9000 and uses the /var/scqsd directory for quorum information.

For information about customizing the quorum server startup file, see Configuring Multiple Quorum Servers on the Same Host in Sun Cluster Quorum Server User’s Guide.

ProcedureHow to Start a Quorum Server

  1. Become superuser on the host where you want to start the Sun Cluster software.

  2. Use the clquorumserver start command to start the software.


    # /usr/cluster/bin/clquorumserver start quorumserver
    
    quorumserver

    Identifies the quorum server. You can use the port number on which the quorum server listens. If you provided an instance name in the configuration file, you can use the name instead.

    To start a single quorum server, provide either the instance name or port number. To start all quorum servers, when you have multiple quorum servers configured, use the + operand.


Example 6–11 Starting All Configured Quorum Servers

The following example starts all the configured quorum servers.


# /usr/cluster/bin/clquorumserver start +


Example 6–12 Starting a Specific Quorum Server

The following example starts the quorum server that listens on port number 2000.


# /usr/cluster/bin/clquorumserver start 2000

ProcedureHow to Stop a Quorum Server

  1. Become superuser on the host where you want to start the Sun Cluster software.

  2. Use the clquorumserver stop command to stop the software.


    # /usr/cluster/bin/clquorumserver stop quorumserver
    
    quorumserver

    Identifies the quorum server. You can use the port number on which the quorum server listens. If you provided an instance name in the configuration file, you can use that name instead.

    To stop a single quorum server, provide either the instance name or port number. To stop all quorum servers, when you have multiple quorum servers configured, use the + operand.


Example 6–13 Stopping All Configured Quorum Servers

The following example stops all the configured quorum servers.


# /usr/cluster/bin/clquorumserver stop +


Example 6–14 Stopping a Specific Quorum Server

The following example stops the quorum server that listens on port number 2000.


# /usr/cluster/bin/clquorumserver stop 2000

Displaying Information About the Quorum Server

You can display configuration information about the quorum server. For every cluster that configured the quorum server as a quorum device, this command shows the corresponding cluster name, cluster ID, list of reservation keys, and list of registration keys.

ProcedureHow to Display Information About the Quorum Server

  1. Become the superuser on the host where you want to display the quorum server information.

    Users other than the superuser require solaris.cluster.read role-based access control (RBAC) authorization. For more information about RBAC rights profiles, see the rbac(5) man page.

  2. Display the configuration information of the quorum server by using the clquorumserver command.


    # /usr/cluster/bin/clquorumserver show quorumserver
    
    quorumserver

    Identifies one or more quorum servers. You can specify the quorum server by instance name, or by port number. To display configuration information for all quorum servers, use the + operand.


Example 6–15 Displaying the Configuration of One Quorum Server

The following example displays the configuration information for the quorum server that uses port 9000. The command displays information for every cluster that has the quorum server configured as a quorum device. This information includes the cluster name and ID, and the list of reservation and registration keys on the device.

In the following example, nodes with IDs 1, 2, 3, and 4 of cluster bastille have registered their keys on the quorum server. Also, because Node 4 owns the quorum device reservation, its key is displayed in the reservation list.


# /usr/cluster/bin/clquorumserver show 9000

=== Quorum Server on port 9000 ===

   ---  Cluster bastille (id 0x439A2EFB) Reservation ---

   Node ID:                      4
     Reservation key:            0x439a2efb00000004

   ---  Cluster bastille (id 0x439A2EFB) Registrations ---

   Node ID:                      1
     Registration key:           0x439a2efb00000001

   Node ID:                      2
     Registration key:           0x439a2efb00000002

   Node ID:                      3
     Registration key:           0x439a2efb00000003

   Node ID:                      4
     Registration key:           0x439a2efb00000004


Example 6–16 Displaying the Configuration of Several Quorum Servers

The following example displays the configuration information for three quorum servers, qs1, qs2, and qs3.


# /usr/cluster/bin/clquorumserver show qs1 qs2 qs3


Example 6–17 Displaying the Configuration of All Running Quorum Servers

The following example displays the configuration information for all running quorum servers:


# /usr/cluster/bin/clquorumserver show +

Cleaning Up Stale Quorum Server Cluster Information

To remove a quorum device of type quorumserver, use the clquorum remove command as described in How to Remove a Quorum Device. Under normal operation, this command also removes the quorum server information on the quorum server host. However, if the cluster loses communications with the quorum server host, removing the quorum device does not clean up this information.

The quorum server cluster information becomes invalid in the following circumstances:


Caution – Caution –

If a quorum device of type quorumserver is not yet removed from the cluster, using this procedure to clean up a valid quorum server could compromise the cluster quorum.


ProcedureHow to Clean Up the Quorum Server Configuration Information

Before You Begin

Remove the quorum server quorum device from the cluster, as described in How to Remove a Quorum Device.


Caution – Caution –

Use this procedure only if the circumstances described in Cleaning Up Stale Quorum Server Cluster Information in Sun Cluster Quorum Server User’s Guide pertain. If a cluster is still using this quorum server, performing this procedure will compromise cluster quorum.


  1. Become the superuser on the quorum server host.

  2. Use the clquorumserver clear command to clean up the configuration file.


    # clquorumserver clear -c clustername -I clusterID quorumserver [-y]
    -c clustername

    The name of the cluster that formerly used the quorum server as a quorum device.

    You can obtain the cluster name by running cluster show on a cluster node.

    -I clusterID

    The cluster ID.

    The cluster ID is an 8-digit hexadecimal number. You can obtain the cluster ID by running cluster show on a cluster node.

    quorumserver

    An identifier for one or more quorum servers.

    The quorum server can be identified by a port number or an instance name. The port number is used by the cluster nodes to communicate with the quorum server. The instance name is specified in the quorum server configuration file, /etc/scqsd/scqsd.conf.

    -y

    Force the clquorumserver clear command to clean up cluster information from the configuration file without first prompting for confirmation.

    Use this option only if you are confident that you want outdated cluster information to be removed from the quorum server.

  3. (Optional) If no other quorum devices are configured on this server instance, stop the quorum server.

    For specific instructions, see How to Stop a Quorum Server in Sun Cluster Quorum Server User’s Guide.


Example 6–18 Cleaning Up Outdated Cluster Information From the Quorum Server Configuration

This example removes information about the cluster named sc-cluster from the quorum server that uses port 9000.


# clquorumserver clear -c sc-cluster -I 0x4308D2CF 9000
The quorum server to be unconfigured must have been removed from the cluster.  
Unconfiguring a valid quorum server could compromise the cluster quorum.  Do you 
want to continue? (yes or no) y