Adding a Quorum Device (Sun Cluster System Administration Guide for Solaris OS)

Sun Cluster System Administration Guide for Solaris OS

Adding a Quorum Device

This section provides procedures to add a quorum device. For information about determining the number of quorum vote counts necessary for your cluster, recommended quorum configurations, and failure fencing, see Quorum and Quorum Devices in Sun Cluster Concepts Guide for Solaris OS.

Caution –

Do not add a disk that is currently configured as a quorum device to a Solaris ZFS storage pool. When a configured quorum device is added to a Solaris ZFS storage pool, the disk is relabeled as an EFI disk and quorum configuration information is lost and the disk no longer provides a quorum vote to the cluster. Once a disk is in a storage pool, that disk can then be configured as a quorum device. Or, you can unconfigure the disk, add it to the storage pool, then reconfigure the disk as a quorum device.

The Sun Cluster software supports the following types of quorum devices: SCSI, Network Appliance (NetApp) NAS, and Sun Cluster Quorum Server. Procedures for adding these devices are provided in the following sections:

Note –

You cannot configure replicated disks as quorum devices. If you attempt to add a replicated disk as a quorum device, you will receive the following error message, after which the command exits with an error code.

Disk-name is a replicated device. Replicated devices cannot be
configured as quorum devices.

You can also accomplish these procedures by using the Sun Cluster Manager GUI. See the Sun Cluster Manager online help for more information.

See the clsetup(1CL) and clquorum(1CL) man pages for information about the commands used in the following procedures.

How to Add a SCSI Quorum Device

To complete this procedure, identify a disk drive by its device ID (DID), which is shared by the nodes. Use the cldevice show command to see the list of DID names. Refer to the cldevice(1CL) man page for additional information.

This procedure provides the long forms of the Sun Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical. For a list of the commands and their short forms, see Appendix A, Sun Cluster Object-Oriented Commands.

Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

Start the clsetup utility.
# clsetup
The clsetup Main Menu is displayed.

Type the number that corresponds to the option for Quorum.

The Quorum Menu is displayed.

Type the number that corresponds to the option for adding a quorum device, then type yes when the clsetup utility asks you to confirm the quorum device that you are adding.

The clsetup utility asks what type of quorum device you want to add.

Type the number that corresponds to the option for a SCSI quorum device.

The clsetup utility asks which global device you want to use.

Type the global device you are using.

The clsetup utility asks you to confirm that the new quorum device should be added to the global device you specified.

Type yes to continue adding the new quorum device.

If the new quorum device is added successfully, the clsetup utility displays a message to that effect.

Verify that the quorum device has been added.
# clquorum list -v

Example 6–1 Adding a SCSI Quorum Device

The following example shows the clquorum command generated by clsetup when it adds a SCSI quorum device and a verification step.

Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization 
on any cluster node.

[Start the clsetup utility:]
# clsetup
[Select Quorum>Add a quorum device]
[Answer the questions when prompted.]
[You will need the following information.]
  [Information:              Example:]
  [SCSI device               scsi]
  [Global device		       d20]

[Verify that the clquorum command was completed successfully:]
 clquorum add d20
 
    Command completed successfully.
[Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is added:]
# clquorum list -v
 
Quorums         Type
-------         ----
d20             scsi
scphyshost-1    node
scphyshost-2    node

How to Add a Sun NAS Quorum Device

Note –

Sun Cluster supports a Sun NAS quorum device only on a two-node cluster configuration.

Use the Sun NAS GUI to set up an iSCSI device on the Sun NAS filer.
1. Create a file volume that is approximately 50 Mbytes in size.
  File Volume Operations -> Create File Volume
2. For each node, create an iSCSI access list.
  iSCSI Configuration -> Configure Access List
  1. Use the name of the cluster as the iSCSI access list name.
  2. Add the initiator node name of each cluster node to the access list. CHAP and IQN are not needed.
3. Configure the iSCSI LUN
  iSCSI Configuration -> Configure iSCSI LUN
  You can use the name of the backing file volume as the name of the LUN. Add the access list for each node to the LUN.

On each of the cluster nodes, discover the iSCSI LUN and set the iSCSI access list to static configuration.

# iscsiadm modify discovery -s enable

	# iscsiadm list discovery
	Discovery:
	        Static: enabled
	        Send Targets: disabled
	        iSNS: disabled

	#  iscsiadm add static-config iqn.LUNName,IPAddress_of_NASDevice
# devfsadm -i iscsi
# cldevice refresh

From one cluster node, configure the DIDs for the iSCSI LUN.
# /usr/cluster/bin/scgdevs

Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

Start the clsetup utility.
# clsetup
The clsetup Main Menu is displayed.

Type the number that corresponds to the option for Quorum.

The Quorum Menu is displayed.

Type the number that corresponds to the option for adding a quorum device, then type yes when the clsetup utility asks you to confirm the quorum device that you are adding.

The clsetup utility asks what type of quorum device you want to add.

Type the number that corresponds to the option for a SCSI quorum device.

The clsetup utility asks which global device you want to use.

Type the global device you are using.

The clsetup utility asks you to confirm that the new quorum device should be added to the global device you specified.

Type yes to continue adding the new quorum device.

If the new quorum device is added successfully, the clsetup utility displays a message to that effect.

Verify that the quorum device has been added.
# clquorum list -v

Example 6–2 Adding a Sun NAS Quorum Device

The following example shows the clquorum command generated by clsetup when it adds a Sun NAS quorum device and a verification step.

Add an iSCSI device on the Sun NAS filer.
Use the Sun NAS GUI to create a file volume that is approximately 50mb in size.
File Volume Operations -> Create File Volume
For each node, create an iSCSI access list.
iSCSI Configuration -> Configure Access List
Add the initiator node name of each cluster node to the access list.
*** Need GUI or command syntax for this step. ***
Configure the iSCSI LUN
iSCSI Configuration -> Configure iSCSI LUN
On each of the cluster nodes, discover the iSCSI LUN and set the iSCSI access list to static configuration.
iscsiadm modify discovery -s enable
iscsiadm list discovery
Discovery:
Static: disable
Send Targets: enables
iSNS: disabled
iscsiadm add status-config
iqn.1986-03.com.sun0-1:000e0c66efe8.4604DE16.thinquorum,10.11.160.20
devsadm -i iscsi
From one cluster node, configure the DID devices for the iSCSI LUN.
/usr/cluster/bin/scgdevs
Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization
on any cluster node.

[Start the clsetup utility:]
# clsetup
[Select Quorum>Add a quorum device]
[Answer the questions when prompted.]
[You will need the following information.]
[Information: Example:]
[SCSI device scsi]
[Global device d20]

[Verify that the clquorum command was completed successfully:]
clquorum add d20
Command completed successfully.
[Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is added:]
# clquorum list -v
Quorums Type
------- ----
d20 scsi
scphyshost-1 node
scphyshost-2 node

How to Add a Network Appliance Network-Attached Storage (NAS) Quorum Device

When you use a Network Appliance (NetApp) network-attached storage (NAS) device as a quorum device, the following are required:

You must install the iSCSI license from NetApp.
You must configure an iSCSI LUN on the clustered filer for use as the quorum device.
You must configure the NetApp NAS unit to use NTP for synchronizing time.
At least one of the NTP servers selected for the clustered filer must be an NTP server for the Sun Cluster nodes.
When booting the cluster, always boot the NAS device before you boot the cluster nodes.

If you boot devices in the wrong order, your nodes cannot find the quorum device. If a node should fail in this situation, your cluster might be unable to remain in service. If a service disruption occurs, you must either reboot the entire cluster or remove the NetApp NAS quorum device and add it again.
A cluster can use a NAS device for only a single quorum device.

You can configure other shared storage if you need additional quorum devices. Additional clusters that use the same NAS device can use separate LUNs on that device as their quorum devices.

See the following Sun Cluster documentation for information about installing a NetApp NAS storage device in a Sun Cluster environment: Sun Cluster 3.1 - 3.2 With Network-Attached Storage Devices Manual for Solaris OS.

Make sure that all Sun Cluster nodes are online and can communicate with the NetApp clustered filer.

Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

Start the clsetup utility.
# clsetup
The clsetup Main Menu is displayed.

Type the number that corresponds to the option for Quorum.

The Quorum Menu is displayed.

Type the number that corresponds to the option for adding a quorum device. Then type yes to confirm that you are adding a quorum device.

The clsetup utility asks what type of quorum device you want to add.

Type the number that corresponds to the option for a netapp_nas quorum device. Then type yes to confirm that you are adding a netapp_nas quorum device.

The clsetup utility asks you provide the name of the new quorum device.

Type the name of the quorum device you are adding.

The quorum device name can be any name you choose. The name is only used to process future administrative commands.

The clsetup utility asks you to provide the name of the filer for the new quorum device.

Type the name of the filer of the new quorum device.

This name is the network-accessible name or address of the filer.

The clsetup utility asks you to provide the LUN ID for the filer.

Type the ID of the quorum device LUN on the filer.

The clsetup utility asks if to the new quorum device should be added on the filer.

Type yes to continue adding the new quorum device.

If the new quorum device is added successfully, the clsetup utility displays a message to that effect.

Verify that the quorum device has been added.
# clquorum list -v

Example 6–3 Adding a NetApp NAS Quorum Device

The following example shows the clquorum command generated by clsetup when it adds a NetApp NAS quorum device. The example also shows a verification step.

Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any cluster node.

[Start the clsetup utility:]
# clsetup
[Select Quorum>Add a quorum device]
[Answer the questions when prompted.]
[You will need the following information.]
  [Information:                Example:]
  [Quorum Device               Netapp_nas quorum device]
  [Name:        		         qd1]
  [Filer:                      nas1.sun.com]
  [LUN ID:                     0]

[Verify that the clquorum command was completed successfully:]
 clquorum add -t netapp_nas -p filer=nas1.sun.com,-p lun_id=0 qd1
 
    Command completed successfully.
[Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is added:]
# clquorum list -v
 
Quorums         Type
-------         ----
qd1             netapp_nas
scphyshost-1    node
scphyshost-2    node

How to Add a Quorum Server Quorum Device

Before You Begin

Before you can add a Sun Cluster Quorum Server as a quorum device, the Sun Cluster Quorum Server software must be installed on the host machine and the quorum server must be started and running. For information about installing and starting the quorum server, see the Sun Cluster Quorum Server User’s Guide.

Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on any node of the cluster.

Make sure that all Sun Cluster nodes are online and can communicate with the Sun Cluster Quorum Server.
1. Ensure that network switches that are directly connected to cluster nodes meet one of the following criteria:
  - The switch supports Rapid Spanning Tree Protocol (RSTP).
  - Fast port mode is enabled on the switch.
  One of these features is required to ensure immediate communication between cluster nodes and the quorum server. If this communication is significantly delayed by the switch, the cluster interprets this prevention of communication as loss of the quorum device.
2. If the public network uses variable-length subnetting, also called Classless Inter-Domain Routing (CIDR), modify the following files on each node.
  
  If you use classful subnets, as defined in RFC 791, you do not need to perform these steps.
  1. Add to the /etc/inet/netmasks file an entry for each public subnet that the cluster uses.
    
    The following is an example entry which contains a public-network IP address and netmask:
    10.11.30.0 255.255.255.0
  2. Append netmask + broadcast + to the hostname entry in each /etc/hostname.adapter file.
    nodename netmask + broadcast +
3. On each node in the cluster, add the quorum server host name to the /etc/inet/hosts file or the /etc/inet/ipnodes file.
  
  Add a host name-to-address mapping to the file, such as the following.
  ipaddress qshost1
  ipaddress
  
  The IP address of the computer where the quorum server is running.
  
  qshost1
  
  The host name of the computer where the quorum server is running.
4. If you use a naming service, add the quorum server host's name-to-address mapping to the name-service database.

Start the clsetup utility.
# clsetup
The clsetup Main Menu is displayed.

Type the number that corresponds to the option for Quorum.

The Quorum Menu is displayed.

Type the number that corresponds to the option for adding a quorum device. Then type yes to confirm that you are adding a quorum device.

The clsetup utility asks what type of quorum device you want to add.

Type the number that corresponds to the option for a quorum-server quorum device. Then type yes to confirm that you are adding a quorum-server quorum device.

The clsetup utility asks you to provide the name of the new quorum device.

Type the name of the quorum device you are adding.

The quorum device name can be any name you choose. The name is only used to process future administrative commands.

The clsetup utility asks you to provide the name of the filer for the new quorum device.

Type the name of the host of the quorum server.

This name specifies the IP address of the machine where the quorum server runs or the host name of the machine on the network.

Depending on the IPv4 or IPv6 configuration of the host, the IP address of the machine must be specified in the /etc/hosts file, the /etc/inet/ipnodes file, or both.

Note –
The machine you specify must be reachable by all cluster nodes and must run the quorum server.

The clsetup utility asks you to provide the port number of the quorum server.

Type the port number that is used by the quorum server to communicate with the cluster nodes.

The clsetup utility asks you to confirm that the new quorum device should be added.

Type yes to continue adding the new quorum device.

If the new quorum device is added successfully, the clsetup utility displays a message to that effect.

Verify that the quorum device has been added.
# clquorum list -v

Example 6–4 Adding a Quorum Server Quorum Device

The following example shows the clquorum command that is generated by clsetup when it adds a quorum server quorum device. The example also shows a verification step.

Become superuser or assume a role that provides solaris.cluster.modify RBAC authorization on 
any cluster node.

[Start the clsetup utility:]
# clsetup
[Select Quorum>Add a quorum device]
[Answer the questions when prompted.]
[You will need the following information.]
  [Information:                Example:]
  [Quorum Device               quorum_server quorum device]
  [Name:        		         qd1]
  [Host Machine Name:          10.11.124.84]
  [Port Number:                9001]

[Verify that the clquorum command was completed successfully:]
 clquorum add -t quorum_server -p qshost=10.11.124.84,-p port=9001 qd1
 
    Command completed successfully.
[Quit the clsetup Quorum Menu and Main Menu.]
[Verify that the quorum device is added:]
# clquorum list -v
 
Quorums         Type
-------         ----
qd1             quorum_server
scphyshost-1    node
scphyshost-2    node
 
-- Quorum Summary --
 
  Quorum votes possible: 5
  Quorum votes needed:   3
  Quorum votes present:  5
 
-- Quorum Votes by Node --
 
                    Node Name           Present Possible Status
                    ---------           ------- -------- ------
  Node votes:       phys-schost-1       1        1       Online
  Node votes:       phys-schost-2       1        1       Online
 
-- Quorum Votes by Device --
 
                   Device Name         Present Possible Status
                   -----------         ------- -------- ------
 Device votes:     qd1                  1        1       Online
 Device votes:     /dev/did/rdsk/d3s2   1        1       Online
 Device votes:     /dev/did/rdsk/d4s2   1        1       Online