| Skip Navigation Links | |
| Exit Print View | |
|
Oracle Solaris Cluster System Administration Guide |
1. Introduction to Administering Oracle Solaris Cluster
2. Oracle Solaris Cluster and RBAC
3. Shutting Down and Booting a Cluster
4. Data Replication Approaches
5. Administering Global Devices, Disk-Path Monitoring, and Cluster File Systems
7. Administering Cluster Interconnects and Public Networks
Administering the Cluster Interconnects
Dynamic Reconfiguration With Cluster Interconnects
How to Check the Status of the Cluster Interconnect
How to Add Cluster Transport Cables, Transport Adapters, or Transport Switches
How to Remove Cluster Transport Cables, Transport Adapters, and Transport Switches
How to Enable a Cluster Transport Cable
How to Disable a Cluster Transport Cable
How to Determine a Transport Adapter's Instance Number
How to Change the Private Network Address or Address Range of an Existing Cluster
Administering the Public Network
How to Administer IP Network Multipathing Groups in a Cluster
Dynamic Reconfiguration With Public Network Interfaces
10. Configuring Control of CPU Usage
11. Patching Oracle Solaris Cluster Software and Firmware
12. Backing Up and Restoring a Cluster
13. Administering Oracle Solaris Cluster With the Graphical User Interfaces
This section provides the procedures for reconfiguring cluster interconnects, such as cluster transport adapters and cluster transport cables. These procedures require that you install Oracle Solaris Cluster software.
Most of the time, you can use the clsetup utility to administer the cluster transport for the cluster interconnects. See the clsetup(1CL) man page for more information. All cluster interconnect commands must be run in the global-cluster voting node.
For cluster software installation procedures, see the Oracle Solaris Cluster Software Installation Guide. For procedures about servicing cluster hardware components, see the Oracle Solaris Cluster 3.3 Hardware Administration Manual.
Note - You can usually choose to use the default port name, where appropriate, during cluster interconnect procedures. The default port name is the same as the internal node ID number of the node that hosts the adapter end of the cable.
Table 7-1 Task List: Administering the Cluster Interconnect
|
You must consider a few issues when completing dynamic reconfiguration (DR) operations on cluster interconnects.
All of the requirements, procedures, and restrictions that are documented for the Oracle Solaris DR feature also apply to Oracle Solaris Cluster DR support (except for the operating system quiescence operation). Therefore, review the documentation for the Oracle Solaris DR feature before using the DR feature with Oracle Solaris Cluster software. You should review in particular the issues that affect non-network IO devices during a DR detach operation.
The Oracle Solaris Cluster software rejects DR remove-board operations performed on active private interconnect interfaces.
You must completely remove an active adapter from the cluster in order to perform DR on an active cluster interconnect. Use the clsetup menu or the appropriate commands.
 | Caution - Oracle Solaris Cluster software requires that each cluster node has at least one functioning path to every other cluster node. Do not disable a private interconnect interface that supports the last path to any cluster node. |
Complete the following procedures in the order indicated when performing DR operations on public network interfaces.
Table 7-2 Task Map: Dynamic Reconfiguration with Public Network Interfaces
|
You can also accomplish this procedure by using the Oracle Solaris Cluster Manager GUI. See the Oracle Solaris Cluster Manager online help for more information.
The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.
This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical.
You do not need to be logged in as superuser to perform this procedure.
% clinterconnect status
|
Example 7-1 Checking the Status of the Cluster Interconnect
The following example shows the status of a functioning cluster interconnect.
% clinterconnect status -- Cluster Transport Paths -- Endpoint Endpoint Status -------- -------- ------ Transport path: phys-schost-1:qfe1 phys-schost-2:qfe1 Path online Transport path: phys-schost-1:qfe0 phys-schost-2:qfe0 Path online Transport path: phys-schost-1:qfe1 phys-schost-3:qfe1 Path online Transport path: phys-schost-1:qfe0 phys-schost-3:qfe0 Path online Transport path: phys-schost-2:qfe1 phys-schost-3:qfe1 Path online Transport path: phys-schost-2:qfe0 phys-schost-3:qfe0 Path online
For information about the requirements for the cluster private transport, see Interconnect Requirements and Restrictions in Oracle Solaris Cluster 3.3 Hardware Administration Manual.
You can also accomplish this procedure by using the Oracle Solaris Cluster Manager GUI. See the Oracle Solaris Cluster Manager online help for more information.
The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.
This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical.
For the procedure on installing a cluster transport cable, see the Oracle Solaris Cluster 3.3 Hardware Administration Manual.
# clsetup
The Main Menu is displayed.
Follow the instructions and type the requested information.
Follow the instructions and type the requested information.
If you plan to use any of the following adapters for the cluster interconnect, add the relevant entry to the /etc/system file on each cluster node. The entry becomes effective after the next system boot.
|
Follow the instructions and type the requested information.
# clinterconnect show node:adapter,adapternode # clinterconnect show node:adapter # clinterconnect show node:switch
Example 7-2 Adding a Cluster Transport Cable, Transport Adapter, or Transport Switch
The following example shows how to add a transport cable, transport adapter, or transport switch to a node by using the clsetup utility.
[Ensure that the physical cable is installed.] [Start the clsetup utility:] # clsetup [Select Cluster interconnect] [Select either Add a transport cable, Add a transport adapter to a node, or Add a transport switch.} [Answer the questions when prompted.] [You Will Need: ] [Information: Example:[ node names phys-schost-1 adapter names qfe2 switch names hub2 transport type dlpi [Verify that the clinterconnect command completed successfully:]Command completed successfully. Quit the clsetup Cluster Interconnect Menu and Main Menu. [Verify that the cable, adapter, and switch are added:] # clinterconnect show phys-schost-1:qfe2,hub2 ===Transport Cables === Transport Cable: phys-schost-1:qfe2@0,hub2 Endpoint1: phys-schost-2:qfe0@0 Endpoint2: ethernet-1@2 ???? Should this be hub2? State: Enabled # clinterconnect show phys-schost-1:qfe2 === Transport Adepters for qfe2 Transport Adapter: qfe2 Adapter State: Enabled Adapter Transport Type: dlpi Adapter Property (device_name): ce Adapter Property (device_instance): 0 Adapter Property (lazy_free): 1 Adapter Property (dlpi_heartbeat_timeout): 10000 Adpater Property (dlpi_heartbeat_quantum): 1000 Adapter Property (nw_bandwidth): 80 Adapter Property (bandwidth): 70 Adapter Property (ip_address): 172.16.0.129 Adapter Property (netmask): 255.255.255.128 Adapter Port Names: 0 Adapter Port STate (0): Enabled # clinterconnect show phys-schost-1:hub2 === Transport Switches === Transport Switch: hub2 Switch State: Enabled Switch Type: switch Switch Port Names: 1 2 Switch Port State(1): Enabled Switch Port State(2): Enabled
Next Steps
To check the interconnect status of your cluster transport cable see How to Check the Status of the Cluster Interconnect.
You can also accomplish this procedure by using the Oracle Solaris Cluster Manager GUI. See the Oracle Solaris Cluster Manager online help for more information.
Use the following procedure to remove cluster transport cables, transport adapters, and transport switches from a node configuration. When a cable is disabled, the two endpoints of the cable remain configured. An adapter cannot be removed if it is still in use as an endpoint on a transport cable.
| Caution - Each cluster node needs at least one functioning transport path to every other node in the cluster. No two nodes should be isolated from one another. Always verify the status of a node's cluster interconnect before disabling a cable. Only disable a cable connection after you have verified that it is redundant. That is, ensure that another connection is available. Disabling a node's last remaining working cable takes the node out of cluster membership. |
The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.
This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical.
# clinterconnect status
| Caution - If you receive an error such as “path faulted” while attempting to remove one node of a two-node cluster, investigate the problem before continuing with this procedure. Such a problem could indicate that a node path is unavailable. Removing the remaining operational path takes the node out of cluster membership and could result in a cluster reconfiguration. |
# clsetup
The Main Menu is displayed.
Follow the instructions and type the requested information. You need to know the applicable node names, adapter names, and switch names.
Follow the instructions and type the requested information. You need to know the applicable node names, adapter names, and switch names.
Note - If you are removing a physical cable, disconnect the cable between the port and the destination device.
Follow the instructions and type the requested information. You need to know the applicable node names, adapter names, and switch names.
Note - If you are removing a physical adapter from a node, see the Oracle Solaris Cluster 3.3 Hardware Administration Manual for hardware service procedures.
Follow the instructions and type the requested information. You need to know the applicable node names, adapter names, and switch names.
Note - A switch cannot be removed if any of the ports are still in use as endpoints on any transport cables.
# clinterconnect show node:adapter,adapternode # clinterconnect show node:adapter # clinterconnect show node:switch
The transport cable or adapter removed from the respective node should not appear in the output from this command.
Example 7-3 Removing a Transport Cable, Transport Adapter, or Transport Switch
The following example shows how to remove a transport cable, transport adapter, or transport switch by using the clsetup command.
[Become superuser on any node in the cluster.] [Start the utility:] # clsetup [Select Cluster interconnect.[ [Select either Remove a transport cable, Remove a transport adapter to a node, or Remove a transport switch.[ [Answer the questions when prompted.[ You Will Need: Information Example: node names phys-schost-1 adapter names qfe1 switch names hub1 [Verify that the clinterconnect command was completed successfully:] Command completed successfully. [Quit the clsetup utility Cluster Interconnect Menu and Main Menu.] [Verify that the cable, adapter, or switch is removed:] # clinterconnect show phys-schost-1:qfe2,hub2 ===Transport Cables === Transport Cable: phys-schost-2:qfe2@0,hub2 Cable Endpoint1: phys-schost-2:qfe0@0 Cable Endpoint2: ethernet-1@2 ??? Should this be hub2??? Cable State: Enabled # clinterconnect show phys-schost-1:qfe2 === Transport Adepters for qfe2 Transport Adapter: qfe2 Adapter State: Enabled Adapter Transport Type: dlpi Adapter Property (device_name): ce Adapter Property (device_instance): 0 Adapter Property (lazy_free): 1 Adapter Property (dlpi_heartbeat_timeout): 10000 Adpater Property (dlpi_heartbeat_quantum): 1000 Adapter Property (nw_bandwidth): 80 Adapter Property (bandwidth): 70 Adapter Property (ip_address): 172.16.0.129 Adapter Property (netmask): 255.255.255.128 Adapter Port Names: 0 Adapter Port STate (0): Enabled # clinterconnect show phys-schost-1:hub2 === Transport Switches === Transport Switch: hub2 Switch State: Enabled Switch Type: switch Switch Port Names: 1 2 Switch Port State(1): Enabled Switch Port State(2): Enabled
You can also accomplish this procedure by using the Oracle Solaris Cluster Manager GUI. See the Oracle Solaris Cluster Manager online help for more information.
This option is used to enable an already existing cluster transport cable.
The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.
This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical.
# clsetup
The Main Menu is displayed.
Follow the instructions when prompted. You need to provide both the node and the adapter names of one of the endpoints of the cable that you are trying to identify.
# clinterconnect show node:adapter,adapternode
Example 7-4 Enabling a Cluster Transport Cable
This example shows how to enable a cluster transport cable on adapter qfe-1, located on the node phys-schost-2.
[Become superuser on any node.] [Start the clsetup utility:] # clsetup [Select Cluster interconnect>Enable a transport cable.[ [Answer the questions when prompted.[ [You will need the following information.[ You Will Need: Information: Example: node names phys-schost-2 adapter names qfe1 switch names hub1 [Verify that the scinterconnect command was completed successfully:] clinterconnect enable phys-schost-2:qfe1 Command completed successfully. [Quit the clsetup Cluster Interconnect Menu and Main Menu.] [Verify that the cable is enabled:] # clinterconnect show phys-schost-1:qfe2,hub2 Transport cable: phys-schost-2:qfe1@0 ethernet-1@2 Enabled Transport cable: phys-schost-3:qfe0@1 ethernet-1@3 Enabled Transport cable: phys-schost-1:qfe0@0 ethernet-1@1 Enabled
You can also accomplish this procedure by using the Oracle Solaris Cluster Manager GUI. See the Oracle Solaris Cluster Manager online help for more information.
You might need to disable a cluster transport cable to temporarily shut down a cluster interconnect path. A temporary shutdown is useful when troubleshooting a cluster interconnect problem or when replacing cluster interconnect hardware.
When a cable is disabled, the two endpoints of the cable remain configured. An adapter cannot be removed if it is still in use as an endpoint in a transport cable.
| Caution - Each cluster node needs at least one functioning transport path to every other node in the cluster. No two nodes should be isolated from one another. Always verify the status of a node's cluster interconnect before disabling a cable. Only disable a cable connection after you have verified that it is redundant. That is, ensure that another connection is available. Disabling a node's last remaining working cable takes the node out of cluster membership. |
The phys-schost# prompt reflects a global-cluster prompt. Perform this procedure on a global cluster.
This procedure provides the long forms of the Oracle Solaris Cluster commands. Most commands also have short forms. Except for the long and short forms of the command names, the commands are identical.
# clinterconnect status
| Caution - If you receive an error such as “path faulted” while attempting to remove one node of a two-node cluster, investigate the problem before continuing with this procedure. Such a problem could indicate that a node path is unavailable. Removing the remaining operational path takes the node out of cluster membership and could result in a cluster reconfiguration. |
# clsetup
The Main Menu is displayed.
Follow the instructions and provide the requested information. All of the components on this cluster interconnect will be disabled. You need to provide both the node and the adapter names of one of the endpoints of the cable that you are trying to identify.
# clinterconnect show node:adapter,adapternode
Example 7-5 Disabling a Cluster Transport Cable
This example shows how to disable a cluster transport cable on adapter qfe-1, located on the node phys-schost-2.
[Become superuser on any node.] [Start the clsetup utility:] # clsetup [Select Cluster interconnect>Disable a transport cable.] [Answer the questions when prompted.] [You will need the following information.] [ You Will Need:] Information: Example: node names phys-schost-2 adapter names qfe1 switch names hub1 [Verify that the clinterconnect command was completed successfully:] Command completed successfully. [Quit the clsetup Cluster Interconnect Menu and Main Menu.] [Verify that the cable is disabled:] # clinterconnect show -p phys-schost-1:qfe2,hub2 Transport cable: phys-schost-2:qfe1@0 ethernet-1@2 Disabled Transport cable: phys-schost-3:qfe0@1 ethernet-1@3 Enabled Transport cable: phys-schost-1:qfe0@0 ethernet-1@1 Enabled
You need to determine a transport adapter's instance number to ensure that you add and remove the correct transport adapter through the clsetup command. The adapter name is a combination of the type of the adapter and the adapter's instance number.
The following screen is an example and might not reflect your hardware.
# prtdiag
...
========================= IO Cards =========================
Bus Max
IO Port Bus Freq Bus Dev,
Type ID Side Slot MHz Freq Func State Name Model
---- ---- ---- ---- ---- ---- ---- ----- --------------------------------
XYZ 8 B 2 33 33 2,0 ok xyz11c8,0-xyz11c8,d665.11c8.0.0
XYZ 8 B 3 33 33 3,0 ok xyz11c8,0-xyz11c8,d665.11c8.0.0
...The following screen is an example and might not reflect your hardware.
# grep sci /etc/path_to_inst "/xyz@1f,400/pci11c8,o@2" 0 "ttt" "/xyz@1f,4000.pci11c8,0@4 "ttt"
The following screen is an example and might not reflect your hardware.
# prtconf
...
xyz, instance #0
xyz11c8,0, instance #0
xyz11c8,0, instance #1
...Use this procedure to change a private network address or the range of network addresses used or both.
Before You Begin
Ensure that remote shell (rsh(1M)) or secure shell (ssh(1)) access for superuser is enabled to all cluster nodes.
The clnode evacuate command switches over all device groups from the specified node to the next-preferred node. The command also switches all resource groups from voting or non-voting nodes on the specified node to the next-preferred voting or non-voting node.
# clnode evacuate node # cluster shutdown -g0 -y
When run in noncluster mode, the clsetup utility displays the Main Menu for noncluster-mode operations.
The clsetup utility displays the current private-network configuration, then asks if you would like to change this configuration.
The clsetup utility displays the default private-network IP address, 172.16.0.0, and asks if it is okay to accept this default.
The clsetup utility will ask if it is okay to accept the default netmask. Skip to the next step to enter your response.
The clsetup utility will prompt for the new private-network IP address.
The clsetup utility displays the default netmask and then asks if it is okay to accept the default netmask.
The default netmask is 255.255.240.0. This default IP address range supports up to 64 nodes, 12 zone clusters, and 10 private networks in the cluster.
Then skip to the next step.
When you decline the default netmask, the clsetup utility prompts you for the number of nodes and private networks, and zone clusters that you expect to configure in the cluster.
From these numbers, the clsetup utility calculates two proposed netmasks:
The first netmask is the minimum netmask to support the number of nodes, private networks, and zone clusters that you specified.
The second netmask supports twice the number of nodes, private networks, and zone clusters that you specified, to accommodate possible future growth.
On SPARC based systems, run the following command.
ok boot
On x86 based systems, run the following commands.
When the GRUB menu is displayed, select the appropriate Oracle Solaris entry and press Enter. The GRUB menu appears similar to the following:
GNU GRUB version 0.95 (631K lower / 2095488K upper memory) +-------------------------------------------------------------------------+ | Solaris 10 /sol_10_x86 | | Solaris failsafe | | | +-------------------------------------------------------------------------+ Use the ^ and v keys to select which entry is highlighted. Press enter to boot the selected OS, 'e' to edit the commands before booting, or 'c' for a command-line.
# cluster status -t node
|