This section contains requirements on interconnect operation when using certain special features.
Heartbeat packets that are sent over the cluster interconnect are not IP based. As a result, these packets cannot be routed. If you install a router between two cluster nodes that are connected through cluster interconnects, heartbeat packets cannot find their destination. Your cluster consequently fails to work correctly.
To ensure that your cluster works correctly, you must set up the cluster interconnect in the same layer 2 (data link) network and in the same broadcast domain. The cluster interconnect must be located in the same layer 2 network and broadcast domain even if the cluster nodes are located in different, remote data centers. Cluster nodes that are arranged remotely are described in more detail in Chapter 7, Campus Clustering With Sun Cluster Software.
An interconnect path is one network step in the cluster private network: from a node to a node, from a node to a switch, or from the switch to another node. Each path in your cluster interconnect must use the same networking technology, whether Ethernet or peripheral component interconnect-scalable coherent interface (PCI-SCI).
All interconnect paths must also operate at the same speed. This means, for example, that if you are using Ethernet components that are capable of operating at different speeds, and if your cluster configuration does not allow these components to automatically negotiate a common network speed, you must configure them to operate at the same speed.
If you use Scalable Data Services and jumbo frames on your public network, ensure that the Maximum Transfer Unit (MTU) of the private network is the same size or larger than the MTU of your public network.
Scalable services cannot forward public network packets that are larger than the MTU size of the private network. The scalable services application instances will not receive those packets.
Consider the following information when configuring jumbo frames:
The maximum MTU size for an InfiniBand interface is typically less than the maximum MTU size for an Ethernet interface.
If you use switches in your private network, ensure they are configured to the MTU sizes of the private network interfaces.
For information about how to configure jumbo frames, see the documentation that shipped with your network interface card. See your Solaris OS documentation or contact your Sun sales representative for other Solaris restrictions.
The following requirements and guidelines apply to Sun Cluster Geographic Edition configurations that use InfiniBand adapters:
A two-node cluster must use InfiniBand switches. You cannot directly connect the InfiniBand adapters to each other.
Sun InfiniBand switches support up to nine nodes in a cluster.
Jumbo frames are not supported on a cluster that uses InfiniBand adapters.
If only one InfiniBand adapter is installed on a cluster node, each of its two ports must be connected to a different InfiniBand switch.
If two InfiniBand adapters are installed in a cluster node, leave the second port on each adapter unused. For example, connect port 1 on HCA 1 to switch 1 and connect port 1 on HCA 2 to switch 2.
VLANs are not supported on a cluster that uses InfiniBand switches.
Do not place a Scalable Coherent Interface (SCI) card in the 33 MHz PCI slot (slot 1) of the hot swap PCI+ (hsPCI+) I/O assembly. This placement can cause a system panic.