Clustering Considerations for Networking
Network device, datalink, and interface failures do not cause a clustered subsystem controller to fail. To protect against network failures inside or outside of the appliance, IPMP and/or LACP should be used. A comprehensive approach to availability requires the correct configuration of the network and a network-wide plan for redundancy.
Figure 4 Clustering for Networking
Network interfaces can be configured as either singleton or private resources, provided they have a static IP configuration. Interfaces configured using DHCP must be private and using DHCP in clusters is discouraged. When configured as a singleton resource, all datalinks and devices used to construct an interface can be active on only one controller at a time. Likewise, corresponding devices on each controller must be attached to the same networks in order for service to be provided in a failed-over state. An example of this is shown in the previous diagram.
For a cluster to operate correctly when you construct network interfaces from devices and datalinks, it is essential that each singleton interface has a device using the same identifier and capabilities available on both controllers. Since device identifiers depend on the device type and the order in which they are first detected by the appliance, clustered controllers MUST have identical hardware installed. Each slot in both controllers must be populated with identical hardware and slots must be populated in the same order on both controllers. Your qualified Oracle reseller or service representative can assist in planning hardware upgrades that meet these requirements.
A route is always bound explicitly to a single network interface. Routes are represented within the resource manager as symbiotes and can become active only when the interfaces to which they are bound are operational. Therefore, a route bound to an interface which is currently in standby mode (exported) has no effect until the interface is activated during the takeover process. This is important when two pools are configured and are made available to a common subnet. If a subnet is home to a router that is used by the appliances to reach one or more other networks, a separate route (for example, a second default route), must be configured and bound to each of the active and standby interfaces attached to that subnet.
Example:
-
Interface e1000g3 is assigned to 'alice' and e1000g4 is assigned to 'bob'.
-
Each interface has an address in the 172.16.27.0/24 network and can be used to provide service to clients in the 172.16.64.0/22 network, reachable via 172.16.27.1.
-
Two routes should be created to 172.16.64.0/22 via 172.16.27.1; one should be bound to e1000g3 and the other to e1000g4.
It is a good idea to assign each clustered controller an IP address used only for administration (most likely on a dedicated management network) and to designate the interface as a private resource. This ensures that it is possible to reach a functioning controller from the management network even if it is in a AKCS_STRIPPED state and awaiting failback. This is important if services such as LDAP and Active Directory are in use and require access to other network resources when the controller is not providing service. If this is not practical, the service processor should be attached to a reliable network and/or serial terminal concentrator so that the controller can be managed using the system console.
If neither of these actions is taken, it is impossible to manage or monitor a newly-booted controller until failback is completed. You may want to monitor or manage the controller that is providing service for a particular storage pool. This is likely to be useful when you want to modify some aspect of the storage itself such as modifying a share property or create a new LUN. This can be done by using one of the service interfaces to perform administrative tasks or by allocating a separate singleton interface to be used only for managing the pool to which it is matched. In either case, the interface should be assigned to the same controller as the pool it is used to manage.
Impact to NFSv4.1 clients - Certain networking changes in a cluster configuration can adversely affect the servicing of requests for NFSv4.1 clients. If the relationship between an IP address and its owner changes, then the best practice is to remount the filesystems from the client. Unlike NFSv4.0, NFSv4.1 protocol enables client connections over multiple IP addresses to be associated with the same NFSv4.1 protocol lease. When the relationship between an IP address and its owner changes, the group of IP addresses that failover together is no longer the same, forcing the client to re-establish the lease relationships by remounting the filesystems.
Related Topics