JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Oracle® ZFS Storage Appliance Administration Guide
Oracle Technology Network
Print View
search filter icon
search icon

Document Information

Using This Documentation

Chapter 1 Oracle ZFS Storage Appliance Overview

Chapter 2 Status

Chapter 3 Initial Configuration

Chapter 4 Network Configuration

Chapter 5 Storage Configuration

Chapter 6 Storage Area Network Configuration

Chapter 7 User Configuration

Chapter 8 Setting ZFSSA Preferences

Chapter 9 Alert Configuration

Chapter 10 Cluster Configuration

Cluster Features and Benefits

Cluster Disadvantages

Cluster Terminology

Understanding Clustering

Cluster Interconnect I/O

Understanding Cluster Resource Management

Cluster Takeover and Failback

Configuration Changes in a Clustered Environment

Clustering Considerations for Storage

Clustering Considerations for Networking

Private Local IP Interfaces

Clustering Considerations for Infiniband

Clustering Redundant Path Scenarios

Preventing 'Split-Brain' Conditions

Estimating and Reducing Takeover Impact

Cluster Configuration Using the BUI

Configuring Clustering

Unconfiguring Clustering

Configuring Clustering Using the CLI

Shutting Down a Clustered Configuration

Shutdown the Stand-by Head

Unconfiguring Clustering

Cluster Node Cabling

ZS3-2 Cluster Cabling

ZS3-4 and 7x20 Cluster Cabling

Storage Shelf Cabling

Cluster Configuration BUI Page

Chapter 11 ZFSSA Services

Chapter 12 Shares, Projects, and Schema

Chapter 13 Replication

Chapter 14 Shadow Migration

Chapter 15 CLI Scripting

Chapter 16 Maintenance Workflows

Chapter 17 Integration


Clustering Considerations for Networking

Network device, datalink, and interface failures do not cause a clustered subsystem head to fail. To protect against network failures inside or outside of the appliance, IPMP and/or LACP should be used. A comprehensive approach to availability requires the correct configuration of the network and a network-wide plan for redundancy.

Figure 10-5  Clustering for Networking


Network interfaces can be configured as either singleton or private resources, provided they have a static IP configuration. Interfaces configured using DHCP must be private and using DHCP in clusters is discouraged. When configured as a singleton resource, all datalinks and devices used to construct an interface can be active on only one head at a time. Likewise, corresponding devices on each head must be attached to the same networks in order for service to be provided in a failed-over state. An example of this is shown in the previous diagram.

For a cluster to operate correctly when you construct network interfaces from devices and datalinks, it is essential that each singleton interface has a device using the same identifier and capabilities available on both heads. Since device identifiers depend on the device type and the order in which they are first detected by the appliance, clustered heads MUST have identical hardware installed. Each slot in both heads must be populated with identical hardware and slots must be populated in the same order on both heads. Your qualified Oracle reseller or service representative can assist in planning hardware upgrades that meet these requirements.

A route is always bound explicitly to a single network interface. Routes are represented within the resource manager as symbiotes and can become active only when the interfaces to which they are bound are operational. Therefore, a route bound to an interface which is currently in standby mode (exported) has no effect until the interface is activated during the takeover process. This is important when two pools are configured and are made available to a common subnet. If a subnet is home to a router that is used by the appliances to reach one or more other networks, a separate route (for example, a second default route), must be configured and bound to each of the active and standby interfaces attached to that subnet.


It is a good idea to assign each clustered head an IP address used only for administration (most likely on a dedicated management network) and to designate the interface as a private resource. This ensures that it is possible to reach a functioning head from the management network even if it is in a AKCS_STRIPPED state and awaiting failback. This is important if services such as LDAP and Active Directory are in use and require access to other network resources when the head is not providing service. If this is not practical, the service processor should be attached to a reliable network and/or serial terminal concentrator so that the head can be managed using the system console.

If neither of these actions is taken, it is impossible to manage or monitor a newly-booted head until failback is completed. You may want to monitor or manage the head that is providing service for a particular storage pool. This is likely to be useful when when you want to modify some aspect of the storage itself such as modifying a share property or create a new LUN. This can be done by using one of the service interfaces to perform administrative tasks or by allocating a separate singleton interface to be used only for managing the pool to which it is matched. In either case, the interface should be assigned to the same head as the pool it is used to manage.