Configuring Oracle Solaris Cluster Software on All Nodes (scinstall)

Language:

Configuring Oracle Solaris Cluster Software on All Nodes (`scinstall`)

The scinstall utility runs in two modes of installation, Typical or Custom. For the Typical installation of Oracle Solaris Cluster software, scinstall automatically specifies the following configuration defaults.

Private-network address: 172.16.0.0
Private-network netmask: 255.255.240.0
Cluster-transport adapters: Exactly two adapters
Cluster-transport switches: switch1 and switch2
Global fencing: Enabled
Installation security (DES): Limited

Complete one of the following cluster configuration worksheets to plan your Typical mode or Custom mode installation:

Typical Mode Worksheet – If you will use Typical mode and accept all defaults, complete the following worksheet.

Component	Description/Example	Answer
Cluster Name	What is the name of the cluster that you want to establish?
Cluster Nodes	List the names of the other cluster nodes planned for the initial cluster configuration.`(For a single-node cluster, press Control-D alone.)`
Cluster Transport Adapters and Cables	What are the names of the two cluster-transport adapters that attach the node to the private interconnect?	`First:` `Second:`
Quorum Configuration `(two-node cluster only)`	Do you want to disable automatic quorum device selection? `(Answer Yes if any shared storage is not qualified to be a quorum device or if you want to configure a quorum server as a quorum device.)`	Yes \| No
Check	Do you want to interrupt cluster creation for `cluster check` errors?	Yes \| No

Custom Mode Worksheet – If you will use Custom mode and customize the configuration data, complete the following worksheet.

Note - If you are installing a single-node cluster, the scinstall utility automatically assigns the default private network address and netmask, even though the cluster does not use a private network.

Component	Description/Example	Answer
Cluster Name	What is the name of the cluster that you want to establish?
Cluster Nodes	List the names of the other cluster nodes planned for the initial cluster configuration.`(For a single-node cluster, press Control-D alone.)`
Authenticating Requests to Add Nodes `(multiple-node cluster only)`	Do you need to use DES authentication?	No \| Yes
Minimum Number of Private Networks `(multiple-node cluster only)`	Should this cluster use at least two private networks?	Yes \| No
Point-to-Point Cables `(multiple-node cluster only)`	If this is a two-node cluster, does this cluster use switches?	Yes \| No
Cluster Switches `(multiple-node cluster only)`	Transport switch name: Defaults: `switch1` and `switch2`	`First:` `Second:`
Cluster Transport Adapters and Cables `(multiple-node cluster only)`	Node name (`the node from which you run scinstall`):
	Transport adapter name:	`First:` `Second:`
	Will this be a dedicated cluster transport adapter? `(Answer No if using tagged VLAN adapters.)`	`First:` Yes \| No `Second:` Yes \| No
	If no, what is the VLAN ID for this adapter?	`First:` `Second:`
	Where does each transport adapter connect to (`a switch or another adapter`)? Switch defaults: `switch1` and `switch2`	`First:` `Second:`
	If a transport switch, do you want to use the default port name?	`First:` Yes \| No `Second:` Yes \| No
	If no, what is the name of the port that you want to use?	`First:` `Second:`
	Do you want to use autodiscovery to list the available adapters for the other nodes? If no, supply the following information for each additional node:	Yes \| No
`Specify for each additional node` `(multiple-node cluster only)`	Node name:
	Transport adapter name:	`First:` `Second:`
	Will this be a dedicated cluster transport adapter? `(Answer No if using tagged VLAN adapters.)`	`First:` Yes \| No `Second:` Yes \| No
	If no, what is the VLAN ID for this adapter?	`First:` `Second:`
	Where does each transport adapter connect to (`a switch or another adapter`)? Defaults: `switch1` and `switch2`	`First:` `Second:`
	If a transport switch, do you want to use the default port name?	`First:` Yes \| No `Second:` Yes \| No
	If no, what is the name of the port that you want to use?	`First:` `Second:`
Network Address for the Cluster Transport `(multiple-node cluster only)`	Do you want to accept the default network address (`172.16.0.0`)?	Yes \| No
	If no, which private network address do you want to use?	___.___.___.___
	Do you want to accept the default netmask (`255.255.240.0`)?	Yes \| No
	If no, what are the maximum numbers of nodes, private networks, and zone clusters that you expect to configure in the cluster?	_____ nodes _____ networks _____ zone clusters
	Which netmask do you want to use? `(Choose from the values calculated by scinstall or supply your own.)`	___.___.___.___
Shared-IP or Exclusive-IP zone clusters	How many exclusive-IP or shared-IP zone clusters do you want to create on the physical cluster? `(The combined number of exclusive-IP and shared-IP zone clusters must be less than or equal to the maximum number of zone clusters that can be created on the physical cluster. The maximum number of exclusive-IP zone clusters is three.)`	______ exclusive-IP zone clusters ______ shared-IP zone clusters
Global Fencing	Do you want to disable global fencing? `(Answer No unless the shared storage does not support SCSI reservations or unless you want systems that are outside the cluster to access the shared storage.)`	`First:` Yes \| No `Second:` Yes \| No
Quorum Configuration `(two-node cluster only)`	Do you want to disable automatic quorum device selection? `(Answer Yes if any shared storage is not qualified to be a quorum device or if you want to configure a quorum server as a quorum device.)`	`First:` Yes \| No `Second:` Yes \| No
Check `(multiple-node cluster only)`	Do you want to interrupt cluster creation for `cluster check` errors?	Yes \| No
`(single-node cluster only)`	Do you want to run the `cluster check` utility to validate the cluster?	Yes \| No
Automatic Reboot `(single-node cluster only)`	Do you want `scinstall` to automatically reboot the node after installation?	Yes \| No

How to Configure Oracle Solaris Cluster Software on All Nodes (`scinstall`)

Perform this procedure from one node of the global cluster to configure Oracle Solaris Cluster software on all nodes of the cluster.

This procedure uses the interactive form of the scinstall command. For information about how to use the noninteractive forms of the scinstall command, such as when developing installation scripts, see the scinstall (1M) man page.

Follow these guidelines to use the interactive scinstall utility in this procedure:

Interactive scinstall enables you to type ahead. Therefore, do not press the Return key more than once if the next menu screen does not appear immediately.
Unless otherwise noted, you can press Control-D to return to either the start of a series of related questions or to the Main Menu.
Default answers or answers to previous sessions are displayed in brackets ([ ]) at the end of a question. Press Return to enter the response that is in brackets without typing it.

Before You Begin

Perform the following tasks:

Ensure that the Oracle Solaris OS is installed to support the Oracle Solaris Cluster software.
If the Oracle Solaris software is already installed on the node, you must ensure that the Oracle Solaris installation meets the requirements for the Oracle Solaris Cluster software and any other software that you intend to install on the cluster. See How to Install Oracle Solaris Software for more information about installing the Oracle Solaris software to meet Oracle Solaris Cluster software requirements.
Ensure that NWAM is disabled. See How to Install Oracle Solaris Cluster Software Packages for instructions.
SPARC: If you are configuring Oracle VM Server for SPARC I/O domains or guest domains as cluster nodes, ensure that the Oracle VM Server for SPARC software is installed on each physical machine and that the domains meet Oracle Solaris Cluster requirements. See How to Install Oracle VM Server for SPARC Software and Create Domains.
Ensure that Oracle Solaris Cluster software packages and updates are installed on each node. See How to Install Oracle Solaris Cluster Software Packages.
Ensure that any adapters that you want to use as tagged VLAN adapters are configured and that you have their VLAN IDs.
Have available your completed Typical Mode or Custom Mode installation worksheet. See Configuring Oracle Solaris Cluster Software on All Nodes (scinstall).

If you are using switches in the private interconnect of your new cluster, ensure that Neighbor Discovery Protocol (NDP) is disabled.
Follow the procedures in the documentation for your switches to determine whether NDP is enabled and to disable NDP.
During cluster configuration, the software checks that there is no traffic on the private interconnect. If NDP sends any packages to a private adapter when the private interconnect is being checked for traffic, the software will assume that the interconnect is not private and cluster configuration will be interrupted. NDP must therefore be disabled during cluster creation.
After the cluster is established, you can re-enable NDP on the private-interconnect switches if you want to use that feature.
On each node to configure in a cluster, assume the root role.
Alternatively, if your user account is assigned the System Administrator profile, issue commands as non-root through a profile shell, or prefix the command with the pfexec command.
Ensure that TCP wrappers for RPC are disabled on all nodes of the cluster.
The Oracle Solaris TCP wrappers for RPC feature prevents internode communication that is necessary for cluster configuration.
1. On each node, display the status of TCP wrappers for RPC.
  TCP wrappers are enabled if config/enable_tcpwrappers is set to true, as shown in the following example command output.
```
# svccfg -s rpc/bind listprop config/enable_tcpwrappers
config/enable_tcpwrappers  boolean true
```
2. If TCP wrappers for RPC are enabled on a node, disable TCP wrappers and refresh the RPC bind service.
```
# svccfg -s rpc/bind setprop config/enable_tcpwrappers = false
# svcadm refresh rpc/bind
# svcadm restart rpc/bind
```
Prepare public-network interfaces.
1. Create static IP addresses for each public-network interface.
```
# ipadm create-ip interface
# ipadm create-addr -T static -a local=address/prefix-length addrobj
```
  For more information, see How to Configure an IPv4 Interface in Configuring and Administering Network Components in Oracle Solaris 11.2 .
2. (Optional) Create IPMP groups for public-network interfaces.
  During initial cluster configuration, unless non-link-local IPv6 public network interfaces exist in the cluster, IPMP groups are automatically created based on matching subnets. These groups use transitive probes for interface monitoring and no test addresses are required.
  
  If these automatically created IPMP groups would not meet your needs, or if IPMP groups would not be created because your configuration includes one or more non-link-local IPv6 public network interfaces, do one of the following:
  - Create the IPMP groups you need before you establish the cluster.
  - After the cluster is established, use the ipadm command to edit the IPMP groups.
  For more information, see Configuring IPMP Groups in Administering TCP/IP Networks, IPMP, and IP Tunnels in Oracle Solaris 11.2 .

From one cluster node, start the scinstall utility.
```
phys-schost# scinstall
```

Type the option number for Create a New Cluster or Add a Cluster Node and press the Return key.

 *** Main Menu ***

Please select from one of the following (*) options:

* 1) Create a new cluster or add a cluster node
* 2) Print release information for this cluster node

* ?) Help with menu options
* q) Quit

Option:  1

The New Cluster and Cluster Node Menu is displayed.

Type the option number for Create a New Cluster and press the Return key.
The Typical or Custom Mode menu is displayed.

Type the option number for either Typical or Custom and press the Return key.
The Create a New Cluster screen is displayed. Read the requirements, then press Control-D to continue.

Follow the menu prompts to supply your answers from the configuration planning worksheet.
The scinstall utility installs and configures all cluster nodes and reboots the cluster. The cluster is established when all nodes have successfully booted into the cluster. Oracle Solaris Cluster installation output is logged in a /var/cluster/logs/install/scinstall.log.N file.
Verify on each node that multiuser services for the Service Management Facility (SMF) are online.
If services are not yet online for a node, wait until the state changes to online before you proceed to the next step.
```
phys-schost# svcs multi-user-server node
STATE          STIME    FMRI
online         17:52:55 svc:/milestone/multi-user-server:default
```

From one node, verify that all nodes have joined the cluster.

phys-schost# clnode status

Output resembles the following.

=== Cluster Nodes ===

--- Node Status ---

Node Name                                       Status
---------                                       ------
phys-schost-1                                   Online
phys-schost-2                                   Online
phys-schost-3                                   Online

For more information, see the clnode(1CL) man page.

Take the cluster out of installmode.
```
phys-schost# clquorum reset
```

(Optional) Enable the automatic node reboot feature.
This feature automatically reboots a node if all monitored shared-disk paths fail, provided that at least one of the disks is accessible from a different node in the cluster.

Note - At initial configuration time, disk-path monitoring is enabled by default for all discovered devices.
1. Enable automatic reboot.
```
phys-schost# clnode set -p reboot_on_path_failure=enabled
```
  -p
  Specifies the property to set
  
  reboot_on_path_failure=enable
  Enables automatic node reboot if failure of all monitored shared-disk paths occurs.
2. Verify that automatic reboot on disk-path failure is enabled.
```
phys-schost# clnode show
=== Cluster Nodes ===

Node Name:                                      node
…
reboot_on_path_failure:                          enabled
…
```
If you plan to enable RPC use of TCP wrappers, add all clprivnet0 IP addresses to the /etc/hosts.allow file on each cluster node.
Without this addition to the /etc/hosts.allow file, TCP wrappers prevent internode communication over RPC for cluster administration utilities.
1. On each node, display the IP addresses for all clprivnet0 devices on the node.
```
# /usr/sbin/ipadm show-addr
ADDROBJ           TYPE     STATE        ADDR
clprivnet0/N      static   ok           ip-address/netmask-length
…
```
2. On each cluster node, add to the /etc/hosts.allow file the IP addresses of all clprivnet0 devices in the cluster.
If you intend to use the HA for NFS data service (HA for NFS) on a highly available local file system, exclude from the automounter map all shares that are part of the highly available local file system that is exported by HA for NFS.
See Administrative Tasks Involving Maps in Managing Network File Systems in Oracle Solaris 11.2 for more information about modifying the automounter map.

Example 3-1 Configuring Oracle Solaris Cluster Software on All Nodes

The following example shows the scinstall progress messages that are logged as scinstall completes configuration tasks on the two-node cluster, schost. The cluster is installed from phys-schost-1 by using the scinstall utility in Typical Mode. The other cluster node is phys-schost-2. The adapter names are net2 and net3. The automatic selection of a quorum device is enabled.

    Log file - /var/cluster/logs/install/scinstall.log.24747

Configuring global device using lofi on pred1: done
Starting discovery of the cluster transport configuration.

The following connections were discovered:

phys-schost-1:net2  switch1  phys-schost-2:net2
phys-schost-1:net3  switch2  phys-schost-2:net3

Completed discovery of the cluster transport configuration.

Started cluster check on "phys-schost-1".
Started cluster check on "phys-schost-2".

cluster check completed with no errors or warnings for "phys-schost-1".
cluster check completed with no errors or warnings for "phys-schost-2".

Configuring "phys-schost-2" … done
Rebooting "phys-schost-2" … done

Configuring "phys-schost-1" … done
Rebooting "phys-schost-1" …

Log file - /var/cluster/logs/install/scinstall.log.24747

Troubleshooting

Unsuccessful configuration – If one or more nodes cannot join the cluster, or if the wrong configuration information was specified, first attempt to perform this procedure again. If that does not correct the problem, perform the procedure How to Unconfigure Oracle Solaris Cluster Software to Correct Installation Problems on each misconfigured node to remove it from the cluster configuration. You do not need to uninstall the Oracle Solaris Cluster software packages. Then perform this procedure again.

Next Steps

If you installed a single-node cluster, cluster establishment is complete. Go to Creating Cluster File Systems to install volume management software and configure the cluster.
If you installed a multiple-node cluster and chose automatic quorum configuration, post installation setup is complete. Go to How to Verify the Quorum Configuration and Installation Mode.
If you installed a multiple-node cluster and declined automatic quorum configuration, perform post installation setup. Go to How to Configure Quorum Devices.

If you intend to configure any quorum devices in your cluster, go to How to Configure Quorum Devices.

Otherwise, go to How to Verify the Quorum Configuration and Installation Mode.

How to Configure Oracle Solaris Cluster Software on All Nodes (scinstall)

Configuring Oracle Solaris Cluster Software on All Nodes (`scinstall`)

How to Configure Oracle Solaris Cluster Software on All Nodes (`scinstall`)