1. Introduction to Oracle Solaris Cluster Hardware
2. Installing and Configuring the Terminal Concentrator
3. Installing Cluster Interconnect Hardware and Configuring VLANs
4. Maintaining Cluster Interconnect Hardware
5. Installing and Maintaining Public Network Hardware
Public Network Hardware: Requirements When Using Jumbo Frames
Installing Public Network Hardware
Installing Public Network Hardware: Where to Go From Here
Maintaining Public Network Hardware in a Running Cluster
Adding Public Network Adapters
Adding Public Network Adapters: Where to Go From Here
Replacing Public Network Adapters
Replacing Public Network Adapters: Where to Go From Here
Removing Public Network Adapters
SPARC: Sun Gigabit Ethernet Adapter Considerations
6. Maintaining Platform Hardware
7. Campus Clustering With Oracle Solaris Cluster Software
The software driver for the Sun GigaSwift Ethernet adapter is known as the Cassini Ethernet (ce) driver. The Oracle Solaris Cluster software supports the ce driver for cluster interconnect and public network applications. Consult your Oracle service representative for details about the network interface products that are supported.
When you use the ce Sun Ethernet driver for the private cluster interconnect, add the following kernel parameters to the /etc/system file on all the nodes in the cluster to avoid communication problems over the private cluster interconnect.
set ce:ce_taskq_disable=1 set ce:ce_ring_size=1024 set ce:ce_comp_ring_size=4096
If you do not set these three kernel parameters when using the ce driver for the private cluster interconnect, one or more of the cluster nodes might panic due to a loss of communication between the nodes of the cluster. In these cases, check for the following panic messages.
Reservation conflict CMM: Cluster lost operational quorum; aborting CMM: Halting to prevent split brain with node name
If you are using the ce driver and your cluster interconnect uses a back-to-back connection, do not disable auto-negotiation. If you must disable auto-negotiation, when you want to force 1000 Mbit operation for example, manually specify the link master, or clock master, for the connection.
When manually specifying the link master, you must set one side of the back-to-back connection to provide the clock signal and the other side to use this clock signal. Use the ndd(1M) command to manually specify the link master and follow the guidelines listed below.
Set the link_master or master_cfg_value parameter to 1 (clock master) on one side of the back-to-back connection and to 0 on the other side.
Specify the link_master parameter for ce driver versions up to and including 1.118.
Specify the master_cfg_value parameter for ce driver versions that are released after 1.118.
Set the master_cfg_value parameter to 1.
To determine the version of the ce driver, use the modinfo command, as shown in the following example.
# modinfo | grep ce 84 78068000 4e016 222 1 ce (CE Ethernet Driver v1.148)
Example 5-1 Using the ndd Command When You Want to Force 1000 Mbit Operation
This example shows how to use the ndd command when you want to force 1000 Mbit operation with a back-to-back connection and the version of the ce driver is lower than or equal to 1.118.
# ndd -set /dev/ce link_master 0
This example shows how to use the ndd command when you want to force 1000 Mbit operation with a back-to-back connection and the version of the ce driver is greater than or equal to 1.119.
# ndd -set /dev/ce master_cfg_enable 1 # ndd -set /dev/ce master_cfg_value 0
If you are using jumbo frames, you must edit the ce.conf file to configure them, as explained in the Sun GigaSwift documentation.
The driver documentation instructs you to grep certain entries from the /etc/path_to_inst file to determine your entries for the ce.conf file. An entry modified for an Oracle Solaris Cluster node resembles the following:
# grep ce /etc/path_to_inst "/node@1/pci@8,600000/network@1" 0 "ce"
When editing the ce.conf file, remove the /node@nodeID identifier prefix from the entries that you put into the driver configuration file. For the example above, the entry to put into the configuration file is:
"/pci@8,600000/network@1" 0 "ce"