Sun Cluster System Administration Guide for Solaris OS

Administering Quorum Devices

A quorum device is a shared storage device or quorum server that is shared by two or more nodes and that contributes votes that are used to establish a quorum. This section provides the procedures for administering quorum devices.

You can use the clquorum(1CL) command to perform all quorum device administrative procedures. In addition, you can accomplish some procedures by using the clsetup(1CL) interactive utility or the Sun Cluster Manager GUI. Whenever possible, quorum procedures are described in this section by using the clsetup utility. The Sun Cluster Manager online help describes how to perform quorum procedures by using the GUI. When you work with quorum devices, keep in mind the following guidelines:


Note –

The clsetup command is an interactive interface to the other Sun Cluster commands. When clsetup runs, the command generates the appropriate specific commands, in this case clquorum commands. These generated commands are shown in the examples at the end of the procedures.


To view the quorum configuration, use clquorum show. The clquorum list command displays the names of quorum devices in the cluster. The clquorum status command provides status and vote count information.

Most examples shown in this section are from a three-node cluster.

Table 6–1 Task List: Administering Quorum

Task 

For Instructions 

Add a quorum device to a cluster by using clsetup(1CL)

Adding a Quorum Device

Remove a quorum device from a cluster by using clsetup (to generate clquorum)

How to Remove a Quorum Device

Remove the last quorum device from a cluster by using clsetup (to generate clquorum)

How to Remove the Last Quorum Device From a Cluster

Replace a quorum device in a cluster by using the add and remove procedures 

How to Replace a Quorum Device

Modify a quorum device list by using the add and remove procedures 

How to Modify a Quorum Device Node List

Put a quorum device into maintenance state by using clsetup (to generate clquorum)

(While in maintenance state, the quorum device does not participate in voting to establish the quorum.) 

How to Put a Quorum Device Into Maintenance State

Reset the quorum configuration to its default state by using clsetup (to generate clquorum)

How to Bring a Quorum Device Out of Maintenance State

List the quorum devices and vote counts by using the clquorum(1CL) command

How to List the Quorum Configuration

Dynamic Reconfiguration With Quorum Devices

You must consider a few issues when completing dynamic reconfiguration (DR) operations on quorum devices in a cluster.

To remove a quorum device, you must complete the following steps, in the order indicated.

Table 6–2 Task Map: Dynamic Reconfiguration With Quorum Devices

Task 

For Instructions 

1. Enable a new quorum device to replace the one being removed. 

Adding a Quorum Device

2. Disable the quorum device to be removed. 

How to Remove a Quorum Device

3. Perform the DR remove operation on the device being removed. 

Sun Enterprise 10000 DR Configuration Guide and the Sun Enterprise 10000 Dynamic Reconfiguration Reference Manual (from the Solaris 9 on Sun Hardware, and Solaris 10 on Sun Hardware collections.)

Adding a Quorum Device

This section provides procedures to add a quorum device. For information about determining the number of quorum vote counts necessary for your cluster, recommended quorum configurations, and failure fencing, see Quorum and Quorum Devices in Sun Cluster Concepts Guide for Solaris OS.


Caution – Caution –

Do not add a disk that is currently configured as a quorum device to a Solaris ZFS storage pool. When a configured quorum device is added to a Solaris ZFS storage pool, the disk is relabeled as an EFI disk and quorum configuration information is lost and the disk no longer provides a quorum vote to the cluster. Once a disk is in a storage pool, that disk can then be configured as a quorum device. You can also unconfigure the disk, add it to the storage pool, and then reconfigure the disk as a quorum device.


The Sun Cluster software supports the following types of quorum devices:

Procedures for adding these devices are provided in the following sections:


Note –

You cannot configure replicated disks as quorum devices. If you try to add a replicated disk as a quorum device, you receive the following error message and the command exits with an error code.


Disk-name is a replicated device. Replicated devices cannot be
configured as quorum devices.

A shared-disk quorum device is any attached storage device that is supported by Sun Cluster software. The shared disk is connected to two or more nodes of your cluster. If you turn fencing on, a dual-ported disk can be configured as a quorum device that uses SCSI-2 or SCSI-3 (the default is SCSI-2). If fencing is turned on and your shared device is connected to more than two nodes, you can configure your shared disk as a quorum device that uses the SCSI-3 protocol (the default protocol for more than two nodes). You can use the SCSI override flag to make the Sun Cluster software use the SCSI-3 protocol for dual-ported shared disks.

If you turn fencing off for a shared disk, you can then configure the disk as a quorum device that uses the software quorum protocol. This would be true regardless of whether the disk supports SCSI-2 or SCSI-3 protocols. Software quorum is a protocol developed by Sun Microsystems that emulates a form of SCSI Persistent Group Reservations (PGR).


Caution – Caution –

If you are using disks that do not support SCSI (such as SATA), you should turn SCSI fencing off.


For quorum devices, you can use a disk that contains user data or is a member of a device group. View the protocol that is used by the quorum subsystem with a shared disk by looking at the access-mode value for the shared disk in the output from the cluster show command.

You can also accomplish these procedures by using the Sun Cluster Manager GUI. See the Sun Cluster Manager online help for more information.

See the clsetup(1CL) and clquorum(1CL) man pages for information about the commands that are used in the following procedures.

ProcedureHow to Add a Shared Disk Quorum Device

Sun Cluster software supports shared-disk (both SCSI and SATA) devices as quorum devices. A SATA device does not support a SCSI reservation, and you must disable the SCSI reservation fencing flag and use the software quorum protocol to configure these disks as quorum devices.

To complete this procedure, identify a disk drive by its device ID (DID), which is shared by the nodes. Use the cldevice show command to see the list of DID names. Refer to the cldevice(1CL) man page for additional information.

Use this procedure to configure SCSI or SATA devices.

The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  2. Start the clsetup utility.


    # clsetup
    

    The clsetup Main Menu is displayed.

  3. Type the number that corresponds to the option for Quorum.

    The Quorum Menu is displayed.

  4. Type the number that corresponds to the option for adding a quorum device, then type yes when the clsetup utility asks you to confirm the quorum device that you are adding.

    The clsetup utility asks what type of quorum device you want to add.

  5. Type the number that corresponds to the option for a shared-disk quorum device.

    The clsetup utility asks which global device you want to use.

  6. Type the global device you are using.

    The clsetup utility asks you to confirm that the new quorum device should be added to the global device you specified.

  7. Type yes to continue adding the new quorum device.

    If the new quorum device is added successfully, the clsetup utility displays a message to that effect.

  8. Verify that the quorum device has been added.


    # clquorum list -v
    

Example 6–1 Adding a Shared Disk Quorum Device

The following example shows the clquorum command generated by clsetup when it adds a shared-disk quorum device and a verification step.


Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any cluster node.

[Start the clsetup utility:]
# clsetup
[Select Quorum>Add a quorum device]
[Answer the questions when prompted.]
[You will need the following information.]
  [Information:                       Example:]
  [Directly attached shared disk      shared_disk]
  [Global device		                  d20]

[Verify that the clquorum command was completed successfully:]
 clquorum add d20
 
    Command completed successfully.
[Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is added:]
# clquorum list -v
 
Quorum         Type
-------         ----
d20              shared_disk
scphyshost-1    node
scphyshost-2    node

ProcedureHow to Add a Sun NAS Quorum Device

To complete this procedure, identify a disk drive by its device ID (DID), which is shared by the nodes. Use the cldevice show command to see the list of DID names. Refer to the cldevice(1CL) man page for additional information.


Note –

Sun Cluster supports a Sun NAS quorum device only on a two-node cluster configuration.


The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

  1. Use the Sun NAS GUI to set up an iSCSI device on the Sun NAS filer.

    1. Create a file volume that is approximately 50 Mbytes in size.


      File Volume Operations -> Create File Volume
    2. For each node, create an iSCSI access list.


      iSCSI Configuration -> Configure Access List
      1. Use the name of the cluster as the iSCSI access list name.

      2. Add the initiator node name of each cluster node to the access list. CHAP and IQN are not needed.

    3. Configure the iSCSI LUN


      iSCSI Configuration -> Configure iSCSI LUN

      You can use the name of the backing file volume as the name of the LUN. Add the access list for each node to the LUN.

  2. On each of the cluster nodes, discover the iSCSI LUN and set the iSCSI access list to static configuration.


    # iscsiadm modify discovery -s enable
    
    	# iscsiadm list discovery
    	Discovery:
    	        Static: enabled
    	        Send Targets: disabled
    	        iSNS: disabled
    
    	#  iscsiadm add static-config iqn.LUNName,IPAddress_of_NASDevice
    # devfsadm -i iscsi
    # cldevice refresh
    
  3. From one cluster node, configure the DIDs for the iSCSI LUN.


    # /usr/cluster/bin/scgdevs
  4. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  5. Start the clsetup utility.


    # clsetup
    

    The clsetup Main Menu is displayed.

  6. Type the number that corresponds to the option for Quorum.

    The Quorum Menu is displayed.

  7. Type the number that corresponds to the option for adding a quorum device, then type yes when the clsetup utility asks you to confirm the quorum device that you are adding.

    The clsetup utility asks what type of quorum device you want to add.

  8. Type the number that corresponds to the option for a shared-disk quorum device.

    The clsetup utility asks which global device you want to use.

  9. Type the global device you are using.

    The clsetup utility asks you to confirm that the new quorum device should be added to the global device you specified.

  10. Type yes to continue adding the new quorum device.

    If the new quorum device is added successfully, the clsetup utility displays a message to that effect.

  11. Verify that the quorum device has been added.


    # clquorum list -v
    

Example 6–2 Adding a Sun NAS Quorum Device

The following example shows the clquorum command generated by clsetup when it adds a Sun NAS quorum device and a verification step.


Add an iSCSI device on the Sun NAS filer.
Use the Sun NAS GUI to create a file volume that is approximately 50mb in size.
File Volume Operations -> Create File Volume
For each node, create an iSCSI access list.
iSCSI Configuration -> Configure Access List
Add the initiator node name of each cluster node to the access list.
*** Need GUI or command syntax for this step. ***
Configure the iSCSI LUN
iSCSI Configuration -> Configure iSCSI LUN
On each of the cluster nodes, discover the iSCSI LUN and set the iSCSI access list to static configuration.
iscsiadm modify discovery -s enable
iscsiadm list discovery
Discovery:
   Static: disable
   Send Targets: enables
   iSNS: disabled
iscsiadm add status-config
iqn.1986-03.com.sun0-1:000e0c66efe8.4604DE16.thinquorum,10.11.160.20
devsadm -i iscsi
From one cluster node, configure the DID devices for the iSCSI LUN.
/usr/cluster/bin/scgdevs
/usr/cluster/bin/scgdevs
Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any cluster node.

[Start the clsetup utility:]
# clsetup
[Select Quorum>Add a quorum device]
[Answer the questions when prompted.]
[You will need the following information.]
  [Information:                           Example:]
  [Directly attached shared disk device   shared_disk]
  [Global device		                      d20

[Verify that the clquorum command was completed successfully:]
 clquorum add d20
 
    Command completed successfully.
[Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is added:]
# clquorum list -v
 
Quorum         Type
-------         ----
d20             shared_disk
scphyshost-1    node
scphyshost-2    node

ProcedureHow to Add a Network Appliance Network-Attached Storage (NAS) Quorum Device

When you use a Network Appliance (NetApp) network-attached storage (NAS) device as a quorum device, the following are required:

See the following Sun Cluster documentation for information about installing a NetApp NAS storage device in a Sun Cluster environment: Sun Cluster 3.1 - 3.2 With Network-Attached Storage Devices Manual for Solaris OS.

The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

  1. Make sure that all Sun Cluster nodes are online and can communicate with the NetApp clustered filer.

  2. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  3. Start the clsetup utility.


    # clsetup
    

    The clsetup Main Menu is displayed.

  4. Type the number that corresponds to the option for Quorum.

    The Quorum Menu is displayed.

  5. Type the number that corresponds to the option for adding a quorum device. Then type yes to confirm that you are adding a quorum device.

    The clsetup utility asks what type of quorum device you want to add.

  6. Type the number that corresponds to the option for a netapp_nas quorum device. Then type yes to confirm that you are adding a netapp_nas quorum device.

    The clsetup utility asks you provide the name of the new quorum device.

  7. Type the name of the quorum device you are adding.

    The quorum device name can be any name you choose. The name is only used to process future administrative commands.

    The clsetup utility asks you to provide the name of the filer for the new quorum device.

  8. Type the name of the filer of the new quorum device.

    This name is the network-accessible name or address of the filer.

    The clsetup utility asks you to provide the LUN ID for the filer.

  9. Type the ID of the quorum device LUN on the filer.

    The clsetup utility asks if to the new quorum device should be added on the filer.

  10. Type yes to continue adding the new quorum device.

    If the new quorum device is added successfully, the clsetup utility displays a message to that effect.

  11. Verify that the quorum device has been added.


    # clquorum list -v
    

Example 6–3 Adding a NetApp NAS Quorum Device

The following example shows the clquorum command generated by clsetup when it adds a NetApp NAS quorum device. The example also shows a verification step.


Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any cluster node.

[Start the clsetup utility:]
# clsetup
[Select Quorum>Add a quorum device]
[Answer the questions when prompted.]
[You will need the following information.]
  [Information:                Example:]
  [Quorum Device               Netapp_nas quorum device]
  [Name:        		         qd1]
  [Filer:                      nas1.sun.com]
  [LUN ID:                     0]

[Verify that the clquorum command was completed successfully:]
 clquorum add -t netapp_nas -p filer=nas1.sun.com,-p lun_id=0 qd1
 
    Command completed successfully.
[Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is added:]
# clquorum list -v
 
Quorum         Type
-------         ----
qd1             netapp_nas
scphyshost-1    node
scphyshost-2    node

ProcedureHow to Add a Quorum Server Quorum Device

Before You Begin

Before you can add a Sun Cluster Quorum Server as a quorum device, the Sun Cluster Quorum Server software must be installed on the host machine and the quorum server must be started and running. For information about installing the quorum server, see the How to Install and Configure Quorum Server Software in Sun Cluster Software Installation Guide for Solaris OS.

The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  2. Make sure that all Sun Cluster nodes are online and can communicate with the Sun Cluster Quorum Server.

    1. Ensure that network switches that are directly connected to cluster nodes meet one of the following criteria:

      • The switch supports Rapid Spanning Tree Protocol (RSTP).

      • Fast port mode is enabled on the switch.

      One of these features is required to ensure immediate communication between cluster nodes and the quorum server. If this communication is significantly delayed by the switch, the cluster interprets this prevention of communication as loss of the quorum device.

    2. If the public network uses variable-length subnetting, also called Classless Inter-Domain Routing (CIDR), modify the following files on each node.

      If you use classful subnets, as defined in RFC 791, you do not need to perform these steps.

      1. Add to the /etc/inet/netmasks file an entry for each public subnet that the cluster uses.

        The following is an example entry which contains a public-network IP address and netmask:


        10.11.30.0	255.255.255.0
      2. Append netmask + broadcast + to the hostname entry in each /etc/hostname.adapter file.


        nodename netmask + broadcast +
        
    3. On each node in the cluster, add the quorum server hostname to the /etc/inet/hosts file or the /etc/inet/ipnodes file.

      Add a hostname-to-address mapping to the file, such as the following.


      ipaddress qshost1
      
      ipaddress

      The IP address of the computer where the quorum server is running.

      qshost1

      The hostname of the computer where the quorum server is running.

    4. If you use a naming service, add the quorum server host's name-to-address mapping to the name-service database.

  3. Start the clsetup utility.


    # clsetup
    

    The clsetup Main Menu is displayed.

  4. Type the number that corresponds to the option for Quorum.

    The Quorum Menu is displayed.

  5. Type the number that corresponds to the option for adding a quorum device. Then type yes to confirm that you are adding a quorum device.

    The clsetup utility asks what type of quorum device you want to add.

  6. Type the number that corresponds to the option for a quorum-server quorum device. Then type yes to confirm that you are adding a quorum-server quorum device.

    The clsetup utility asks you to provide the name of the new quorum device.

  7. Type the name of the quorum device you are adding.

    The quorum device name can be any name you choose. The name is only used to process future administrative commands.

    The clsetup utility asks you to provide the name of the filer for the new quorum device.

  8. Type the name of the host of the quorum server.

    This name specifies the IP address of the machine where the quorum server runs or the hostname of the machine on the network.

    Depending on the IPv4 or IPv6 configuration of the host, the IP address of the machine must be specified in the /etc/hosts file, the /etc/inet/ipnodes file, or both.


    Note –

    The machine you specify must be reachable by all cluster nodes and must run the quorum server.


    The clsetup utility asks you to provide the port number of the quorum server.

  9. Type the port number that is used by the quorum server to communicate with the cluster nodes.

    The clsetup utility asks you to confirm that the new quorum device should be added.

  10. Type yes to continue adding the new quorum device.

    If the new quorum device is added successfully, the clsetup utility displays a message to that effect.

  11. Verify that the quorum device has been added.


    # clquorum list -v
    

Example 6–4 Adding a Quorum Server Quorum Device

The following example shows the clquorum command that is generated by clsetup when it adds a quorum server quorum device. The example also shows a verification step.


Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any cluster node.

[Start the clsetup utility:]
# clsetup
[Select Quorum > Add a quorum device]
[Answer the questions when prompted.]
[You will need the following information.]
  [Information:                Example:]
  [Quorum Device               quorum_server quorum device]
  [Name:        		           qd1]
  [Host Machine Name:          10.11.124.84]
  [Port Number:                9001]

[Verify that the clquorum command was completed successfully:]
 clquorum add -t quorum_server -p qshost=10.11.124.84,-p port=9001 qd1
 
    Command completed successfully.
[Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is added:]
# clquorum list -v
 
Quorum         Type
-------         ----
qd1             quorum_server
scphyshost-1    node
scphyshost-2    node

# clquorum status 

=== Cluster Quorum ===
-- Quorum Votes Summary --
 
        Needed     Present    Possible
        ------     -------    -------- 
        3          5           5
   
-- Quorum Votes by Node --
 
  Node Name        Present    Possible   Status
  ---------        -------    --------  ------
  phys-schost-1    1          1          Online
  phys-schost-2    1          1          Online
 
-- Quorum Votes by Device --
 
  Device Name       Present   Possible   Status
  -----------       -------   --------   ------
  qd1               1         1          Online
  d3s2              1         1          Online
  d4s2              1         1          Online

Removing or Replacing a Quorum Device

This section provides the following procedures for removing or replacing a quorum device:

ProcedureHow to Remove a Quorum Device

You can also accomplish this procedure by using the Sun Cluster Manager GUI. See the Sun Cluster Manager online help for more information.

When a quorum device is removed, it no longer participates in the voting to establish quorum. Note that all two-node clusters require that at least one quorum device be configured. If this is the last quorum device on a cluster, clquorum(1CL) will fail to remove the device from the configuration. If you are removing a node, remove all quorum devices connected to the node.


Note –

If the device you intend to remove is the last quorum device in the cluster, see the procedure How to Remove the Last Quorum Device From a Cluster.


The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node in the cluster.

  2. Determine the quorum device to be removed.


    # clquorum list -v
    
  3. Execute the clsetup(1CL) utility.


    # clsetup
    

    The Main Menu is displayed.

  4. Type the number that corresponds to the option for Quorum.

  5. Type the number that corresponds to the option to remove a quorum device.

    Answer the questions displayed during the removal process.

  6. Quit clsetup.

  7. Verify that the quorum device is removed.


    # clquorum list -v
    

Example 6–5 Removing a Quorum Device

This example shows how to remove a quorum device from a cluster with two or more quorum devices configured.


Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any cluster node.

[Determine the quorum device to be removed:]
# clquorum list -v
[Start the clsetup utility:]
# clsetup
[Select Quorum>Remove a quorum device]
[Answer the questions when prompted.] 
Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is removed:]
# clquorum list -v
 
 Quorum         Type
-------         ----
scphyshost-1    node
scphyshost-2    node
scphyshost-3    node

Troubleshooting

If you lose communications between the cluster and the quorum server host while removing a quorum server quorum device, you must clean up stale configuration information about the quorum server host. For instructions on performing this cleanup, see Cleaning Up Stale Quorum Server Cluster Information.

ProcedureHow to Remove the Last Quorum Device From a Cluster

This procedure removes the last quorum device from a cluster. Use this procedure only if you have a two-node cluster and you are removing one of the nodes. If one of the nodes fails, use the -F option to remove the device. Generally, you should first add the replacement quorum device and then remove the failed device. Adding a quorum device involves a node reconfiguration, which touches the failed quorum device and panics the machine. The Force option lets you remove the failed quorum device without panicking the machine.

If the device you intend to remove is not the last quorum device in a two-node cluster, follow the steps in How to Remove a Quorum Device.


Note –

All two-node clusters require at least one configured quorum device. If this is the last quorum device on a two-node cluster, the cluster must be placed into install mode before clquorum(1CL) enables you to remove the device from the configuration. This procedure should only be performed if a node is being removed from the cluster.


The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node in the cluster.

  2. Place the node to be removed in maintenance state.

    See How to Put a Node Into Maintenance State.

  3. Remove the quorum device by using the clquorumcommand. If the quorum device failed, use the -F Force option to remove the failed device.

    The clsetup(1CL) cluster-administration menu options are not available while the cluster is in install mode.


    # clquorum remove -F qd1
    
  4. Verify that the quorum device has been removed.


    # clquorum list -v
    

Example 6–6 Removing the Last Quorum Device

This example shows how to remove the last remaining quorum device in a cluster configuration.


[Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any cluster node.]
[Place the cluster in install mode:]
# cluster set -p installmode=enabled
[Remove the quorum device:]
# clquorum remove d3
[Verify that the quorum device has been removed:]
# clquorum list -v
 Quorum         Type
-------         ----
scphyshost-1    node
scphyshost-2    node
scphyshost-3    node

ProcedureHow to Replace a Quorum Device

Use this procedure to replace an existing quorum device with another quorum device. You can replace a quorum device with a similar device type, such as replacing a NAS device with another NAS device, or you can replace the device with a dissimilar device, such as replacing a NAS device with a shared disk.

The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

  1. Configure a new quorum device.

    You need to first add a new quorum device to the configuration to take the place of the old device. See Adding a Quorum Device to add a new quorum device to the cluster.

  2. Remove the device that you are replacing as a quorum device.

    See How to Remove a Quorum Device to remove the old quorum device from the configuration.

  3. If the quorum device is a failed disk, replace the disk.

    Refer to the hardware procedures for your disk enclosure in the Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS.

Maintaining Quorum Devices

This section provides the following procedures for maintaining quorum devices:

ProcedureHow to Modify a Quorum Device Node List

You can use the clsetup(1CL) utility to add a node to or remove a node from the node list of an existing quorum device. To modify a quorum device's node list, you must remove the quorum device, modify the physical connections of nodes to the quorum device you removed, then add the quorum device to the cluster configuration again. When a quorum device is added, clquorum(1CL) automatically configures the node-to-disk paths for all nodes attached to the disk.

The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  2. Determine the name of the quorum device you are modifying.


    # clquorum list -v
    
  3. Start the clsetup utility.


    # clsetup
    

    The Main Menu is displayed.

  4. Type the number that corresponds to the Quorum option.

    The Quorum Menu is displayed.

  5. Type the number that corresponds to the option to remove a quorum device.

    Follow the instructions. You will be asked the name of the disk to be removed.

  6. Add or delete the node connections to the quorum device.

  7. Type the number that corresponds to the option to add a quorum device.

    Follow the instructions. You will be asked the name of the disk to be used as the quorum device.

  8. Verify that the quorum device has been added.


    # clquorum list -v
    

Example 6–7 Modifying a Quorum Device Node List

The following example shows how to use the clsetup utility to add nodes to or delete nodes from a quorum device node list. In this example, the quorum device name is d2, and the final result of the procedures adds another node to the quorum device node list.


[Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on 
any node in the cluster.]

[Determine the quorum device name:]
# clquorum list -v
Quorum            Type
-------            -----
d2                 shared_disk
sc-phys-schost-1   node
sc-phys-schost-2   node
sc-phys-schost-3   node

[Start the clsetup utility:]
# clsetup

[Type the number that corresponds with the quorum option.]
.
[Type the number that corresponds with the option to remove a quorum device.]
.
[Answer the questions when prompted.]
[You will need the following information:]

   Information:			Example:
   Quorum Device Name: 	d2

[Verify that the clquorum command completed successfully:]
 clquorum remove d2
     Command completed successfully.

[Verify that the quorum device was removed.]
# clquorum list -v
Quorum            Type
-------            -----
sc-phys-schost-1   node
sc-phys-schost-2   node
sc-phys-schost-3   node

[Type the number that corresponds with the Quorum option.]
.
[Type the number that corresponds with the option to add a quorum device.]
.
[Answer the questions when prompted.]
[You will need the following information:]

   Information            Example:
   quorum device name     d2

[Verify that the clquorum command was completed successfully:] 
clquorum add d2
     Command completed successfully.

Quit the clsetup utility.

[Verify that the correct nodes have paths to the quorum device. 
In this example, note that phys-schost-3 has been added to the 
enabled hosts list.]
# clquorum show d2 | grep Hosts
=== Quorum Devices ===

Quorum Device Name:		d2
   Hosts (enabled):		phys-schost-1, phys-schost-2, phys-schost-3

[Verify that the modified quorum device is online.]

# clquorum status d2
=== Cluster Quorum ===

--- Quorum Votes by Device ---

Device Name       Present      Possible      Status
-----------       -------      --------      ------
d2                1            1             Online

ProcedureHow to Put a Quorum Device Into Maintenance State

Use the clquorum(1CL) command to put a quorum device into maintenance state. The clsetup(1CL) utility does not currently have this capability. You can also accomplish this procedure by using the Sun Cluster Manager GUI. See the Sun Cluster Manager online help for more information.

Put a quorum device into maintenance state when taking the quorum device out of service for an extended period of time. This way, the quorum device's quorum vote count is set to zero and does not contribute to the quorum count while the device is being serviced. While in maintenance state, the quorum device's configuration information is preserved.


Note –

All two-node clusters require at least one configured quorum device. If this is the last quorum device on a two-node cluster, clquorum will fail to put the device into maintenance state.


To put a cluster node into maintenance state, see How to Put a Node Into Maintenance State.

The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  2. Put the quorum device into the maintenance state.


    # clquorum disable device
    
    device

    Specifies the DID name of the disk device to change, for example, d4.

  3. Verify that the quorum device is now in maintenance state.

    The output for the device you placed in maintenance state should read zero for the Quorum Device Votes.


    # clquorum status device
    

Example 6–8 Putting a Quorum Device Into Maintenance State

The following example shows how to put a quorum device into maintenance state and how to verify the results.


# clquorum disable d20
# clquorum status d20
  
=== Cluster Quorum ===

--- Quorum Votes by Device ---

Device Name       Present      Possible      Status
-----------       -------      --------      ------
d20                1            1             Offline

See Also

To re-enable the quorum device, see How to Bring a Quorum Device Out of Maintenance State.

To put a node into maintenance state, see How to Put a Node Into Maintenance State.

ProcedureHow to Bring a Quorum Device Out of Maintenance State

Run this procedure each time a quorum device is in maintenance state and you want to bring the quorum device out of maintenance state and reset the quorum vote count to the default.


Caution – Caution –

If you do not specify either the globaldev or node options, the quorum count is reset for the entire cluster.


When you configure a quorum device, Sun Cluster software assigns the quorum device a vote count of N-1 where N is the number of connected votes to the quorum device. For example, a quorum device that is connected to two nodes with nonzero vote counts has a quorum count of one (two minus one).

The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

  1. Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

  2. Reset the quorum count.


    # clquorum enable device
    
    device

    Specifies the DID name of the quorum device to reset, for example, d4.

  3. If you are resetting the quorum count because a node was in maintenance state, reboot the node.

  4. Verify the quorum vote count.


    # clquorum show +
    

Example 6–9 Resetting the Quorum Vote Count (Quorum Device)

The following example resets the quorum count for a quorum device back to the default and verifies the result.


# clquorum enable d20
# clquorum show +
  
=== Cluster Nodes ===                          

Node Name:                                      phys-schost-2
  Node ID:                                        1
  Quorum Vote Count:                              1
  Reservation Key:                                0x43BAC41300000001

Node Name:                                      phys-schost-3
  Node ID:                                        2
  Quorum Vote Count:                              1
  Reservation Key:                                0x43BAC41300000002


=== Quorum Devices ===                         

Quorum Device Name:                             d3
  Enabled:                                        yes
  Votes:                                          1
  Global Name:                                    /dev/did/rdsk/d20s2
  Type:                                           shared_disk
  Access Mode:                                    scsi2
  Hosts (enabled):                                phys-schost-2, phys-schost-3

ProcedureHow to List the Quorum Configuration

You can also accomplish this procedure by using the Sun Cluster Manager GUI. See the Sun Cluster Manager online help for more information.

You do not need to be superuser to list the quorum configuration. You can assume any role that provides solaris.cluster.read RBAC authorization.


Note –

When you increase or decrease the number of node attachments to a quorum device, the quorum vote count is not automatically recalculated. You can reestablish the correct quorum vote if you remove all quorum devices and then add them back into the configuration. For a two-node cluster, temporarily add a new quorum device before you remove and add back the original quorum device. Then remove the temporary quorum device.


The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

  1. Use clquorum(1CL) to list the quorum configuration.


    % clquorum show +
    

Example 6–10 Listing the Quorum Configuration


% clquorum show +

=== Cluster Nodes ===                          

Node Name:                                      phys-schost-2
  Node ID:                                        1
  Quorum Vote Count:                              1
  Reservation Key:                                0x43BAC41300000001

Node Name:                                      phys-schost-3
  Node ID:                                        2
  Quorum Vote Count:                              1
  Reservation Key:                                0x43BAC41300000002


=== Quorum Devices ===                         

Quorum Device Name:                             d3
  Enabled:                                        yes
  Votes:                                          1
  Global Name:                                    /dev/did/rdsk/d20s2
  Type:                                           shared_disk
  Access Mode:                                    scsi2
  Hosts (enabled):                                phys-schost-2, phys-schost-3

ProcedureHow to Repair a Quorum Device

Use this procedure to replace a malfunctioning quorum device.

The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix B, Sun Cluster Object-Oriented Commands.

  1. Remove the disk device that you are replacing as a quorum device.


    Note –

    If the device you intend to remove is the last quorum device, you might want to first add another disk as a new quorum device. This step assures a valid quorum device if a failure occurs during the replacement procedure. See Adding a Quorum Device to add a new quorum device.


    See How to Remove a Quorum Device to remove a disk device as a quorum device.

  2. Replace the disk device.

    To replace the disk device, see the hardware procedures for the disk enclosure in the Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS.

  3. Add the replaced disk as a new quorum device.

    See Adding a Quorum Device to add a disk as a new quorum device.


    Note –

    If you added an additional quorum device in Step 1, it is now safe to remove it. See How to Remove a Quorum Device to remove the quorum device.