Chapter 1 Planning the Sun Cluster Configuration

This chapter provides planning information and guidelines for installing a Sun Cluster configuration.

The following overview information is in this chapter:

• Finding Sun Cluster Installation Tasks

• Planning the Solaris OS

• Planning the Sun Cluster Environment

• Planning the Global Devices, Device Groups, and Cluster File Systems

• Planning Volume Management

Finding Sun Cluster Installation Tasks

The following table shows where to find instructions for various installation tasks for Sun Cluster software installation and the order in which you should perform the tasks.

Table 1–1 Sun Cluster Software Installation Task Information

Task	Instructions
Set up cluster hardware.	Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS Documentation that shipped with your server and storage devices
Plan cluster software installation.	Chapter 1, Planning the Sun Cluster Configuration Installation and Configuration Worksheets
Install software packages. Optionally, install and configure Sun StorEdgeTM QFS software.	Installing the Software Sun StorEdge QFS Installation and Upgrade Guide
Establish a new cluster or a new cluster node.	Establishing a New Cluster or New Cluster Node
Configure Solaris Volume Manager software.	Configuring Solaris Volume Manager Software Solaris Volume Manager documentation
Install and configure VERITAS Volume Manager (VxVM) software.	Installing and Configuring VxVM Software VxVM documentation
Configure cluster file systems, if used.	How to Create Cluster File Systems
(Optional) On the Solaris 10 OS, create non-global zones.	How to Create a Non-Global Zone on a Cluster Node
(Optional) SPARC: Install and configure the Sun Cluster module to Sun Management Center.	SPARC: Installing the Sun Cluster Module for Sun Management Center Sun Management Center documentation
Plan, install, and configure resource groups and data services. Create highly available local file systems, if used.	Sun Cluster Data Services Planning and Administration Guide for Solaris OS
Develop custom data services.	Sun Cluster Data Services Developer’s Guide for Solaris OS
Upgrade to Sun Cluster 3.2 software.	Chapter 8, Upgrading Sun Cluster Software Configuring Solaris Volume Manager Software or Installing and Configuring VxVM Software Volume manager documentation  SPARC: How to Upgrade Sun Cluster Module Software for Sun Management Center

Planning the Solaris OS

This section provides the following guidelines for planning Solaris software installation in a cluster configuration.

• Guidelines for Selecting Your Solaris Installation Method

• Solaris OS Feature Restrictions

• Solaris Software Group Considerations

• System Disk Partitions

• Guidelines for Non-Global Zones in a Cluster

For more information about Solaris software, see your Solaris installation documentation.

Guidelines for Selecting Your Solaris Installation Method

You can install Solaris software from a local DVD-ROM or from a network installation server by using the JumpStart^TM installation method. In addition, Sun Cluster software provides a custom method for installing both the Solaris OS and Sun Cluster software by using the JumpStart installation method. If you are installing several cluster nodes, consider a network installation.

See How to Install Solaris and Sun Cluster Software (JumpStart) for details about the scinstall JumpStart installation method. See your Solaris installation documentation for details about standard Solaris installation methods.

Solaris OS Feature Restrictions

Consider the following points when you plan the use of the Solaris OS in a Sun Cluster configuration:

• Solaris 10 Zones - Install Sun Cluster 3.2 framework software only in the global zone.

  To determine whether you can install a Sun Cluster data service directly in a non-global zone, see the documentation for that data service.

  If you configure non-global zones on a cluster node, the loopback file system (LOFS) must be enabled. See the information for LOFS for additional considerations.

• Loopback file system (LOFS) - During cluster creation with the Solaris 9 version of Sun Cluster software , LOFS capability is disabled by default. During cluster creation with the Solaris 10 version of Sun Cluster software, LOFS capability is not disabled by default.

  If the cluster meets both of the following conditions, you must disable LOFS to avoid switchover problems or other failures:

  • Sun Cluster HA for NFS is configured on a highly available local file system.

  • The automountd daemon is running.

  If the cluster meets only one of these conditions, you can safely enable LOFS.

  If you require both LOFS and the automountd daemon to be enabled, exclude from the automounter map all files that are part of the highly available local file system that is exported by Sun Cluster HA for NFS.

• Interface groups - Solaris interface groups are not supported in a Sun Cluster configuration. The Solaris interface groups feature is disabled by default during Solaris software installation. Do not re-enable Solaris interface groups. See the ifconfig(1M) man page for more information about Solaris interface groups.

• Power-saving shutdown - Automatic power-saving shutdown is not supported in Sun Cluster configurations and should not be enabled. See the pmconfig(1M) and power.conf(4) man pages for more information.

• IP Filter - Sun Cluster software only supports filtering with Solaris IP Filter for failover services. Do not use IP Filter with scalable services. For more information about using IP Filter with failover services, see Using Solaris IP Filtering with Sun Cluster in Sun Cluster 3.2 Release Notes for Solaris OS.

Solaris Software Group Considerations

Sun Cluster 3.2 software requires at least the End User Solaris Software Group. However, other components of your cluster configuration might have their own Solaris software requirements as well. Consider the following information when you decide which Solaris software group you are installing.

• Servers - Check your server documentation for any Solaris software requirements. For example , Sun Enterprise^TM 10000 servers require the Entire Solaris Software Group Plus OEM Support.

• SCI-PCI adapters - To use SCI-PCI adapters, which are available for use in SPARC based clusters only, or the Remote Shared Memory Application Programming Interface (RSMAPI), ensure that you install the RSMAPI software packages, which are SUNWrsm and SUNWrsmo, and for the Solaris 9 OS on SPARC based platforms also SUNWrsmx and SUNWrsmox. The RSMAPI software packages are included only in some Solaris software groups. For example, the Developer Solaris Software Group includes the RSMAPI software packages but the End User Solaris Software Group does not.

  If the software group that you install does not include the RSMAPI software packages, install the RSMAPI software packages manually before you install Sun Cluster software. Use the pkgadd(1M) command to manually install the software packages. See the Section (3RSM) man pages for information about using the RSMAPI.

• Additional Solaris packages - You might need to install other Solaris software packages that are not part of the End User Solaris Software Group. The Apache HTTP server packages are one example. Third-party software, such as ORACLE®, might also require additional Solaris software packages. See your third-party documentation for any Solaris software requirements.

---

Tip –

To avoid the need to manually install Solaris software packages, install the Entire Solaris Software Group Plus OEM Support.

---

System Disk Partitions

Add this information to the appropriate Local File System Layout Worksheet.

When you install the Solaris OS, ensure that you create the required Sun Cluster partitions and that all partitions meet minimum space requirements.

• swap – The combined amount of swap space that is allocated for Solaris and Sun Cluster software must be no less than 750 Mbytes. For best results, add at least 512 Mbytes for Sun Cluster software to the amount that is required by the Solaris OS. In addition, allocate any additional swap amount that is required by applications that are to run on the cluster node.

  ---

  Note –

  If you create an additional swap file, do not create the swap file on a global device. Only use a local disk as a swap device for the node.

  ---

• /globaldevices – Create a file system at least 512 Mbytes large that is to be used by the scinstall(1M) utility for global devices.

• Volume manager – Create a 20-Mbyte partition on slice 7 for volume manager use. If your cluster uses VERITAS Volume Manager (VxVM) and you intend to encapsulate the root disk, you need to have two unused slices available for use by VxVM.

To meet these requirements, you must customize the partitioning if you are performing interactive installation of the Solaris OS.

See the following guidelines for additional partition planning information:

• Guidelines for the Root (/) File System

• Guidelines for the /globaldevices File System

• Volume Manager Requirements

Guidelines for the Root (/) File System

As with any other system running the Solaris OS, you can configure the root (/), /var, /usr, and /opt directories as separate file systems. Or, you can include all the directories in the root (/) file system. The following describes the software contents of the root (/), /var, /usr, and /opt directories in a Sun Cluster configuration. Consider this information when you plan your partitioning scheme.

• root (/) – The Sun Cluster software itself occupies less than 40 Mbytes of space in the root (/) file system. Solaris Volume Manager software requires less than 5 Mbytes, and VxVM software requires less than 15 Mbytes. To configure ample additional space and inode capacity, add at least 100 Mbytes to the amount of space you would normally allocate for your root ( /) file system. This space is used for the creation of both block special devices and character special devices used by the volume management software. You especially need to allocate this extra space if a large number of shared disks are in the cluster.

• /var – The Sun Cluster software occupies a negligible amount of space in the /var file system at installation time. However, you need to set aside ample space for log files. Also, more messages might be logged on a clustered node than would be found on a typical standalone server. Therefore, allow at least 100 Mbytes for the /var file system.

• /usr – Sun Cluster software occupies less than 25 Mbytes of space in the /usr file system. Solaris Volume Manager and VxVM software each require less than 15 Mbytes.

• /opt – Sun Cluster framework software uses less than 2 Mbytes in the /opt file system. However, each Sun Cluster data service might use between 1 Mbyte and 5 Mbytes. Solaris Volume Manager software does not use any space in the /opt file system. VxVM software can use over 40 Mbytes if all of its packages and tools are installed.

  In addition, most database and applications software is installed in the /opt file system.

  SPARC: If you use Sun Management Center software to monitor the cluster, you need an additional 25 Mbytes of space on each node to support the Sun Management Center agent and Sun Cluster module packages.

Guidelines for the /globaldevices File System

Sun Cluster software requires you to set aside a special file system on one of the local disks for use in managing global devices. This file system is later mounted as a cluster file system. Name this file system /globaldevices, which is the default name that is recognized by the scinstall(1M) command.

The scinstall command later renames the file system /global/.devices/node@nodeid, where nodeid represents the number that is assigned to a node when it becomes a cluster member. The original /globaldevices mount point is removed.

The /globaldevices file system must have ample space and ample inode capacity for creating both block special devices and character special devices. This guideline is especially important if a large number of disks are in the cluster. A file system size of 512 Mbytes should suffice for most cluster configurations.

Volume Manager Requirements

If you use Solaris Volume Manager software, you must set aside a slice on the root disk for use in creating the state database replica. Specifically, set aside a slice for this purpose on each local disk. But, if you only have one local disk on a node, you might need to create three state database replicas in the same slice for Solaris Volume Manager software to function properly. See your Solaris Volume Manager documentation for more information.

If you use VERITAS Volume Manager (VxVM) and you intend to encapsulate the root disk, you need to have two unused slices that are available for use by VxVM. Additionally, you need to have some additional unassigned free space at either the beginning or the end of the disk. See your VxVM documentation for more information about root disk encapsulation.

Example - Sample File-System Allocations

Table 1–2 shows a partitioning scheme for a cluster node that has less than 750 Mbytes of physical memory. This scheme is to be installed with the End User Solaris Software Group, Sun Cluster software, and the Sun Cluster HA for NFS data service. The last slice on the disk, slice 7, is allocated with a small amount of space for volume-manager use.

This layout allows for the use of either Solaris Volume Manager software or VxVM software. If you use Solaris Volume Manager software, you use slice 7 for the state database replica. If you use VxVM, you later free slice 7 by assigning the slice a zero length. This layout provides the necessary two free slices, 4 and 7, as well as provides for unused space at the end of the disk.

Table 1–2 Example File-System Allocation

Slice	Contents	Size Allocation	Description
0	/	6.75GB	Remaining free space on the disk after allocating space to slices 1 through 7. Used for the Solaris OS, Sun Cluster software, data-services software, volume-manager software, Sun Management Center agent and Sun Cluster module agent packages, root file systems, and database and application software.
1	swap	1GB	512 Mbytes for the Solaris OS.  512 Mbytes for Sun Cluster software.
2	overlap	8.43GB	The entire disk.
3	/globaldevices	512MB	The Sun Cluster software later assigns this slice a different mount point and mounts the slice as a cluster file system.
4	unused	-	Available as a free slice for encapsulating the root disk under VxVM.
5	unused	-	-
6	unused	-	-
7	volume manager	20MB	Used by Solaris Volume Manager software for the state database replica, or used by VxVM for installation after you free the slice.

Guidelines for Non-Global Zones in a Cluster

For information about the purpose and function of Solaris 10 zones in a cluster, see Support for Solaris Zones on Sun Cluster Nodes in Sun Cluster Concepts Guide for Solaris OS.

Consider the following points when you create a Solaris 10 non-global zone, simply referred to as a zone, on a cluster node.

• Unique zone name - The zone name must be unique in the node. Do not specify the same name to more than one zone on the same node.

• Reusing a zone name on multiple nodes - To simplify cluster administration, you can use the same name for a zone on each node where resource groups are to be brought online in that zone.

• Private IP addresses - Do not attempt to use more private IP addresses than are available in the cluster.

• Mounts - Do not include global mounts in zone definitions. Include only loopback mounts.

• Failover services - In multiple-node clusters, while Sun Cluster software permits you to specify different zones on the same node in a failover resource group's node list, doing so is useful only during testing. If a single node hosts all zones in the node list, the node becomes a single point of failure for the resource group. For highest availability, zones in a failover resource group's node list should be on different nodes.

  In single-node clusters, there is no functional risk if you specify multiple zones in a failover resource group's node list.

• Scalable services - Do not create non-global zones for use in the same scalable service on the same node. Each instance of the scalable service must run on a different cluster node.

• LOFS - Solaris Zones requires that the loopback file system (LOFS) be enabled. However, the Sun Cluster HA for NFS data service requires that LOFS be disabled, to avoid switchover problems or other failures. If you configure both non-global zones and Sun Cluster HA for NFS in your cluster, do one of the following to prevent possible problems in the data service:

  • Disable the automountd daemon.

  • Exclude from the automounter map all files that are part of the highly available local file system that is exported by Sun Cluster HA for NFS.

Planning the Sun Cluster Environment

This section provides guidelines for planning and preparing the following components for Sun Cluster software installation and configuration:

• Licensing

• Software Patches

• Public Network IP Addresses

• Console-Access Devices

• Logical Addresses

• Public Networks

• Guidelines for NFS

• Service Restrictions

• Sun Cluster Configurable Components

For detailed information about Sun Cluster components, see the Sun Cluster Overview for Solaris OS and the Sun Cluster Concepts Guide for Solaris OS.

Licensing

Ensure that you have available all necessary license certificates before you begin software installation. Sun Cluster software does not require a license certificate, but each node installed with Sun Cluster software must be covered under your Sun Cluster software license agreement.

For licensing requirements for volume-manager software and applications software, see the installation documentation for those products.

Software Patches

After installing each software product, you must also install any required patches.

• For information about current required patches, see Patches and Required Firmware Levels in Sun Cluster 3.2 Release Notes for Solaris OS or consult your Sun service provider.

• For general guidelines and procedures for applying patches, see Chapter 10, Patching Sun Cluster Software and Firmware, in Sun Cluster System Administration Guide for Solaris OS.

Public Network IP Addresses

For information about the use of public networks by the cluster, see Public Network Adapters and Internet Protocol (IP) Network Multipathing in Sun Cluster Concepts Guide for Solaris OS.

You must set up a number of public network IP addresses for various Sun Cluster components, depending on your cluster configuration. Each node in the cluster configuration must have at least one public-network connection to the same set of public subnets.

The following table lists the components that need public network IP addresses assigned. Add these IP addresses to the following locations:

• Any naming services that are used

• The local /etc/inet/hosts file on each cluster node, after you install Solaris software

• For Solaris 10, the local /etc/inet/ipnodes file on each cluster node, after you install Solaris software

Table 1–3 Sun Cluster Components That Use Public Network IP Addresses

Component	Number of IP Addresses Needed
Administrative console	1 IP address per subnet.
Cluster nodes	1 IP address per node, per subnet.
Domain console network interface (Sun FireTM 15000)	1 IP address per domain.
(Optional) Non-global zones	1 IP address per subnet.
Console-access device	1 IP address.
Logical addresses	1 IP address per logical host resource, per subnet.
Quorum server	1 IP address.

For more information about planning IP addresses, see System Administration Guide: IP Services (Solaris 9 or Solaris 10).

Console-Access Devices

You must have console access to all cluster nodes. If you install Cluster Control Panel software on an administrative console, you must provide the hostname and port number of the console-access device that is used to communicate with the cluster nodes.

• A terminal concentrator is used to communicate between the administrative console and the cluster node consoles.

• A Sun Enterprise 10000 server uses a System Service Processor (SSP) instead of a terminal concentrator.

• A Sun Fire server uses a system controller instead of a terminal concentrator.

For more information about console access, see the Sun Cluster Concepts Guide for Solaris OS.

Alternatively, if you connect an administrative console directly to cluster nodes or through a management network, you instead provide the hostname of each cluster node and its serial port number that is used to connect to the administrative console or the management network.

Logical Addresses

Each data-service resource group that uses a logical address must have a hostname specified for each public network from which the logical address can be accessed.

For more information, see the Sun Cluster Data Services Planning and Administration Guide for Solaris OS. For additional information about data services and resources, also see the Sun Cluster Overview for Solaris OS and the Sun Cluster Concepts Guide for Solaris OS.

Public Networks

Public networks communicate outside the cluster. Consider the following points when you plan your public-network configuration:

• Separation of public and private network - Public networks and the private network (cluster interconnect) must use separate adapters, or you must configure tagged VLAN on tagged-VLAN capable adapters and VLAN-capable switches to use the same adapter for both the private interconnect and the public network.

• Minimum - All cluster nodes must be connected to at least one public network. Public-network connections can use different subnets for different nodes.

• Maximum - You can have as many additional public-network connections as your hardware configuration allows.

• Scalable services - All nodes that run a scalable service must either use the same subnet or set of subnets or use different subnets that are routable among themselves.

• IPv4 - Sun Cluster software supports IPv4 addresses on the public network.

• IPv6 - Sun Cluster software supports IPv6 addresses on the public network under the following conditions or restrictions:

  • Sun Cluster software does not support IPv6 addresses on the public network if the private interconnect uses SCI adapters.

  • Sun Cluster software supports IPv6 addresses for both failover and scalable data services.

• IPMP groups - Each public network adapter that is used for data-service traffic must belong to an IP network multipathing (IPMP) group. If a public-network adapter is not used for data-service traffic, you do not have to configure it in an IPMP group.

  In the Sun Cluster 3.2 release, the scinstall utility no longer automatically configures a single-adapter IPMP group on each unconfigured public-network adapter during Sun Cluster creation. Instead, the scinstall utility automatically configures a multiple-adapter IPMP group for each set of public-network adapters in the cluster that uses the same subnet. On the Solaris 10 OS, these groups are probe based. However, the scinstall utility ignores adapters that are already configured in an IPMP group. If any adapter in an IPMP group that the scinstall utility configures will not be used for data-service traffic, you can remove that adapter from the group.

  For guidelines and instructions to configure IPMP groups, follow the procedures in Part VI, IPMP, in System Administration Guide: IP Services. To modify IPMP groups after cluster installation, follow the guidelines in How to Administer IP Network Multipathing Groups in a Cluster in Sun Cluster System Administration Guide for Solaris OS and procedures in Administering IPMP (Tasks) in System Administration Guide: IP Services (Solaris 9 or Solaris 10).

• Local MAC address support - All public network adapters must use network interface cards (NICs) that support local MAC address assignment. Local MAC address assignment is a requirement of IPMP.

• local-mac-address setting - The local-mac-address? variable must use the default value true for Ethernet adapters. Sun Cluster software does not support a local-mac-address? value of false for Ethernet adapters. This requirement is a change from Sun Cluster 3.0, which did require a local-mac-address? value of false.

For more information about public-network interfaces, see Sun Cluster Concepts Guide for Solaris OS.

Guidelines for NFS

Consider the following points when you plan the use of Network File System (NFS) in a Sun Cluster configuration.

• NFS client - No Sun Cluster node can be an NFS client of a Sun Cluster HA for NFS-exported file system that is being mastered on a node in the same cluster. Such cross-mounting of Sun Cluster HA for NFS is prohibited. Use the cluster file system to share files among cluster nodes.

• NFSv3 protocol - If you are mounting file systems on the cluster nodes from external NFS servers, such as NAS filers, and you are using the NFSv3 protocol, you cannot run NFS client mounts and the Sun Cluster HA for NFS data service on the same cluster node. If you do, certain Sun Cluster HA for NFS data-service activities might cause the NFS daemons to stop and restart, interrupting NFS services. However, you can safely run the Sun Cluster HA for NFS data service if you use the NFSv4 protocol to mount external NFS file systems on the cluster nodes.

• Locking - Applications that run locally on the cluster must not lock files on a file system that is exported through NFS. Otherwise, local blocking (for example, flock(3UCB) or fcntl(2)) might interfere with the ability to restart the lock manager ( lockd(1M)). During restart, a blocked local process might be granted a lock which might be intended to be reclaimed by a remote client. This would cause unpredictable behavior.

• NFS security features - Sun Cluster software does not support the following options of the share_nfs(1M) command:

  • secure

  • sec=dh

  However, Sun Cluster software does support the following security features for NFS:

  • The use of secure ports for NFS. You enable secure ports for NFS by adding the entry set nfssrv:nfs_portmon=1 to the /etc/system file on cluster nodes.

  • The use of Kerberos with NFS. For more information, see Securing Sun Cluster HA for NFS With Kerberos V5 in Sun Cluster Data Service for NFS Guide for Solaris OS.

Service Restrictions

Observe the following service restrictions for Sun Cluster configurations:

• Routers - Do not configure cluster nodes as routers (gateways). If the system goes down, the clients cannot find an alternate router and cannot recover.

• NIS+ servers - Do not configure cluster nodes as NIS or NIS+ servers. There is no data service available for NIS or NIS+. However, cluster nodes can be NIS or NIS+ clients.

• Boot and install servers - Do not use a Sun Cluster configuration to provide a highly available boot or installation service on client systems.

• RARP - Do not use a Sun Cluster configuration to provide an rarpd service.

• RPC program numbers - If you install an RPC service on the cluster, the service must not use any of the following program numbers:

  • 100141

  • 100142

  • 100248

  These numbers are reserved for the Sun Cluster daemons rgmd_receptionist, fed, and pmfd, respectively.

  If the RPC service that you install also uses one of these program numbers, you must change that RPC service to use a different program number.

• Scheduling classes - Sun Cluster software does not support the running of high-priority process scheduling classes on cluster nodes. Do not run either of the following types of processes on cluster nodes:

  • Processes that run in the time-sharing scheduling class with a high priority

  • Processes that run in the real-time scheduling class

  Sun Cluster software relies on kernel threads that do not run in the real-time scheduling class. Other time-sharing processes that run at higher-than-normal priority or real-time processes can prevent the Sun Cluster kernel threads from acquiring needed CPU cycles.

Sun Cluster Configurable Components

This section provides guidelines for the following Sun Cluster components that you configure:

• Cluster Name

• Node Names

• Zone Names

• Private Network

• Private Hostnames

• Cluster Interconnect

• Quorum Devices

Add this information to the appropriate configuration planning worksheet.

Cluster Name

Specify a name for the cluster during Sun Cluster configuration. The cluster name should be unique throughout the enterprise.

Node Names

The cluster node name is the same name that you assign to the machine when you install it with the Solaris OS. See the hosts(4) man page for information about naming requirements.

In single-node cluster installations, the default cluster name is the name of the node.

During Sun Cluster configuration, you specify the names of all nodes that you are installing in the cluster.

Zone Names

On the Solaris 10 OS, use the naming convention nodename:zonename to specify a non-global zone to a Sun Cluster command.

• The nodename is the name of the cluster node.

• The zonename is the name that you assign to the non-global zone when you create the zone on the node. The zone name must be unique on the node. However, you can use the same zone name on different nodes, because the different node name in nodename:zonename makes the complete non-global zone name unique in the cluster.

To specify the global zone, you only need to specify the node name.

Private Network

---

Note –

You do not need to configure a private network for a single-node cluster. The scinstall utility automatically assigns the default private-network address and netmask, even though a private network is not used by the cluster.

---

Sun Cluster software uses the private network for internal communication among nodes and among non-global zones that are managed by Sun Cluster software. A Sun Cluster configuration requires at least two connections to the cluster interconnect on the private network. When you configure Sun Cluster software on the first node of the cluster, you specify the private-network address and netmask in one of the following ways:

• Accept the default private-network address (172.16.0.0) and netmask (255.255.248.0). This IP address range supports a combined maximum of 64 nodes and non-global zones and a maximum of 10 private networks.

  ---

  Note –

  The maximum number of nodes that an IP address range can support does not reflect the maximum number of nodes that the hardware configuration can support.

  ---

• Specify a different allowable private-network address and accept the default netmask.

• Accept the default private-network address and specify a different netmask.

• Specify both a different private-network address and a different netmask.

If you choose to specify a different netmask, the scinstall utility prompts you for the number of nodes and the number of private networks that you want the IP address range to support. The number of nodes that you specify should also include the expected number of non-global zones that will use the private network.

The utility calculates the netmask for the minimum IP address range that will support the number of nodes and private networks that you specified. The calculated netmask might support more than the supplied number of nodes, including non-global zones, and private networks. The scinstall utility also calculates a second netmask that would be the minimum to support twice the number of nodes and private networks. This second netmask would enable the cluster to accommodate future growth without the need to reconfigure the IP address range.

The utility then asks you what netmask to choose. You can specify either of the calculated netmasks or provide a different one. The netmask that you specify must minimally support the number of nodes and private networks that you specified to the utility.

---

Note –

To change the private-network address and netmask after the cluster is established, see How to Change the Private Network Address or Address Range of an Existing Cluster in Sun Cluster System Administration Guide for Solaris OS. You must bring down the cluster to make these changes.

Changing the cluster private IP address range might be necessary to support the addition of nodes, non-global zones, or private networks.

---

If you specify a private-network address other than the default, the address must meet the following requirements:

• Address and netmask sizes - The private network address cannot be smaller than the netmask. For example, you can use a private network address of 172.16.10.0 with a netmask of 255.255.255.0. But you cannot use a private network address of 172.16.10.0 with a netmask of 255.255.0.0.

• Acceptable addresses - The address must be included in the block of addresses that RFC 1918 reserves for use in private networks. You can contact the InterNIC to obtain copies of RFCs or view RFCs online at http://www.rfcs.org.

• Use in multiple clusters - You can use the same private network address in more than one cluster. Private IP network addresses are not accessible from outside the cluster.

• IPv6 - Sun Cluster software does not support IPv6 addresses for the private interconnect. The system does configure IPv6 addresses on the private network adapters to support scalable services that use IPv6 addresses. But internode communication on the private network does not use these IPv6 addresses.

See Planning Your TCP/IP Network (Tasks), in System Administration Guide: IP Services (Solaris 9 or Solaris 10) for more information about private networks.

Private Hostnames

The private hostname is the name that is used for internode communication over the private-network interface. Private hostnames are automatically created during Sun Cluster configuration. These private hostnames follow the naming convention clusternodenodeid -priv, where nodeid is the numeral of the internal node ID. During Sun Cluster configuration, the node ID number is automatically assigned to each node when the node becomes a cluster member. After the cluster is configured, you can rename private hostnames by using the clsetup(1CL) utility.

For the Solaris 10 OS, the creation of a private hostname for a non-global zone is optional. There is no required naming convention for the private hostname of a non-global zone.

Cluster Interconnect

The cluster interconnects provide the hardware pathways for private-network communication between cluster nodes. Each interconnect consists of a cable that is connected in one of the following ways:

• Between two transport adapters

• Between a transport adapter and a transport switch

For more information about the purpose and function of the cluster interconnect, see Cluster Interconnect in Sun Cluster Concepts Guide for Solaris OS.

---

Note –

You do not need to configure a cluster interconnect for a single-node cluster. However, if you anticipate eventually adding nodes to a single-node cluster configuration, you might want to configure the cluster interconnect for future use.

---

During Sun Cluster configuration, you specify configuration information for one or two cluster interconnects.

• The use of two cluster interconnects provides higher availability than one interconnect. If the number of available adapter ports is limited, you can use tagged VLANs to share the same adapter with both the private and public network. For more information, see the guidelines for tagged VLAN adapters in Transport Adapters.

• The use of one cluster interconnect reduces the number of adapter ports that is used for the private interconnect but provides less availability. In addition, the cluster would spend more time in automatic recovery if the single private interconnect fails.

You can configure additional cluster interconnects after the cluster is established by using the clsetup(1CL) utility.

For guidelines about cluster interconnect hardware, see Interconnect Requirements and Restrictions in Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS. For general information about the cluster interconnect, see Cluster-Interconnect Components in Sun Cluster Overview for Solaris OS and Sun Cluster Concepts Guide for Solaris OS.

Transport Adapters

For the transport adapters, such as ports on network interfaces, specify the transport adapter names and transport type. If your configuration is a two-node cluster, you also specify whether your interconnect is a point-to-point connection (adapter to adapter) or uses a transport switch.

Consider the following guidelines and restrictions:

• IPv6 - Sun Cluster software does not support IPv6 communications over the private interconnects.

• Local MAC address assignment - All private network adapters must use network interface cards (NICs) that support local MAC address assignment. Link-local IPv6 addresses, which are required on private network adapters to support IPv6 public network addresses, are derived from the local MAC addresses.

• Tagged VLAN adapters – Sun Cluster software supports tagged Virtual Local Area Networks (VLANs) to share an adapter between the private cluster interconnect and the public network. To configure a tagged VLAN adapter for the cluster interconnect, specify the adapter name and its VLAN ID (VID) in one of the following ways:

  • Specify the usual adapter name, which is the device name plus the instance number or physical point of attachment (PPA). For example, the name of instance 2 of a Cassini Gigabit Ethernet adapter would be ce2. If the scinstall utility asks whether the adapter is part of a shared virtual LAN, answer yes and specify the adapter's VID number.

  • Specify the adapter by its VLAN virtual device name. This name is composed of the adapter name plus the VLAN instance number. The VLAN instance number is derived from the formula (1000*V)+N, where V is the VID number and N is the PPA.

    As an example, for VID 73 on adapter ce2, the VLAN instance number would be calculated as (1000*73)+2. You would therefore specify the adapter name as ce73002 to indicate that it is part of a shared virtual LAN.

  For information about configuring VLAN in a cluster, see Configuring VLANs as Private Interconnect Networks in Sun Cluster 3.1 - 3.2 Hardware Administration Manual for Solaris OS. For general information about VLAN, see Solaris 9 9/05 Sun Hardware Platform Guide.

• SBus SCI adapters – The SBus Scalable Coherent Interface (SCI) is not supported as a cluster interconnect. However, the SCI-PCI interface is supported.

• Logical network interfaces - Logical network interfaces are reserved for use by Sun Cluster software.

See the scconf_trans_adap_*(1M) family of man pages for information about a specific transport adapter.

Transport Switches

If you use transport switches, such as a network switch, specify a transport switch name for each interconnect. You can use the default name switchN, where N is a number that is automatically assigned during configuration, or create another name.

Also specify the switch port name or accept the default name. The default port name is the same as the internal node ID number of the node that hosts the adapter end of the cable. However, you cannot use the default port name for certain adapter types, such as SCI-PCI.

---

Note –

Clusters with three or more nodes must use transport switches. Direct connection between cluster nodes is supported only for two-node clusters.

---

If your two-node cluster is direct connected, you can still specify a transport switch for the interconnect.

---

Tip –

If you specify a transport switch, you can more easily add another node to the cluster in the future.

---

Quorum Devices

Sun Cluster configurations use quorum devices to maintain data and resource integrity. If the cluster temporarily loses connection to a node, the quorum device prevents amnesia or split-brain problems when the cluster node attempts to rejoin the cluster. For more information about the purpose and function of quorum devices, see Quorum and Quorum Devices in Sun Cluster Concepts Guide for Solaris OS.

During Sun Cluster installation of a two-node cluster, you can choose to let the scinstall utility automatically configure a SCSI quorum device. This quorum device is chosen from the available shared SCSI storage disks. The scinstall utility assumes that all available shared SCSI storage disks are supported to be quorum devices.

If you want to use a quorum server or a Network Appliance NAS device as the quorum device, you configure it after scinstall processing is completed.

After installation, you can also configure additional quorum devices by using the clsetup(1CL) utility.

---

Note –

You do not need to configure quorum devices for a single-node cluster.

---

If your cluster configuration includes third-party shared storage devices that are not supported for use as quorum devices, you must use the clsetup utility to configure quorum manually.

Consider the following points when you plan quorum devices.

• Minimum – A two-node cluster must have at least one quorum device, which can be a shared SCSI disk, a quorum server, or a Network Appliance NAS device. For other topologies, quorum devices are optional.

• Odd-number rule – If more than one quorum device is configured in a two-node cluster, or in a pair of nodes directly connected to the quorum device, configure an odd number of quorum devices. This configuration ensures that the quorum devices have completely independent failure pathways.

• Distribution of quorum votes - For highest availability of the cluster, ensure that the total number of votes that are contributed by quorum devices is less than the total number of votes that are contributed by nodes. Otherwise, the nodes cannot form a cluster if all quorum devices are unavailable, even if all nodes are functioning.

• Connection – You must connect a quorum device to at least two nodes.

• SCSI fencing protocol – When a SCSI quorum device is configured, its SCSI protocol is automatically set to SCSI-2 in a two-node cluster or SCSI-3 in cluster with three or more nodes. You cannot change the SCSI protocol of a device after it is configured as a quorum device.

• Replicated devices – Sun Cluster software does not support replicated devices as quorum devices.

• ZFS storage pools - Do not add a configured quorum device to a Zettabyte File System (ZFS) storage pool. When a configured quorum device is added to a ZFS storage pool, the disk is relabeled as an EFI disk and quorum configuration information is lost. The disk can then no longer provide a quorum vote to the cluster.

  Once a disk is in a storage pool, you can configure that disk as a quorum device. Or, you can unconfigure the quorum device, add it to the storage pool, then reconfigure the disk as a quorum device.

For more information about quorum devices, see Quorum and Quorum Devices in Sun Cluster Concepts Guide for Solaris OS and Quorum Devices in Sun Cluster Overview for Solaris OS.

Planning the Global Devices, Device Groups, and Cluster File Systems

This section provides the following guidelines for planning global devices and for planning cluster file systems:

• Global Devices

• Device Groups

• Cluster File Systems

• Choosing Mount Options for Cluster File Systems

• Mount Information for Cluster File Systems

Global Devices

For information about the purpose and function of global devices, see Devices in Sun Cluster Overview for Solaris OS and Global Devices in Sun Cluster Concepts Guide for Solaris OS.

Sun Cluster software does not require any specific disk layout or file system size. Consider the following points when you plan your layout for global devices.

• Mirroring – You must mirror all global devices for the global device to be considered highly available. You do not need to use software mirroring if the storage device provides hardware RAID as well as redundant paths to disks.

• Disks – When you mirror, lay out file systems so that the file systems are mirrored across disk arrays.

• Availability – You must physically connect a global device to more than one node in the cluster for the global device to be considered highly available. A global device with multiple physical connections can tolerate a single-node failure. A global device with only one physical connection is supported, but the global device becomes inaccessible from other nodes if the node with the connection is down.

• Swap devices - Do not create a swap file on a global device.

• Non-global zones - Global devices are not directly accessible from a non-global zone. Only cluster-file-system data is accessible from a non-global zone.

Device Groups

For information about the purpose and function of device groups, see Devices in Sun Cluster Overview for Solaris OS and Device Groups in Sun Cluster Concepts Guide for Solaris OS.

Add this planning information to the Device Group Configurations Worksheet.

Consider the following points when you plan device groups.

• Failover – You can configure multihost disks and properly configured volume-manager devices as failover devices. Proper configuration of a volume-manager device includes multihost disks and correct setup of the volume manager itself. This configuration ensures that multiple nodes can host the exported device. You cannot configure tape drives, CD-ROMs or DVD-ROMs, or single-ported devices as failover devices.

• Mirroring – You must mirror the disks to protect the data from disk failure. See Mirroring Guidelines for additional guidelines. See Configuring Solaris Volume Manager Software or Installing and Configuring VxVM Software and your volume-manager documentation for instructions about mirroring.

• Storage-based replication - Disks in a device group must be either all replicated or none replicated. A device group cannot use a mix of replicated and nonreplicated disks.

Cluster File Systems

For information about the purpose and function of cluster file systems, see Cluster File Systems in Sun Cluster Overview for Solaris OS and Cluster File Systems in Sun Cluster Concepts Guide for Solaris OS.

---

Note –

You can alternatively configure highly available local file systems. This can provide better performance to support a data service with high I/O, or to permit use of certain file-system features that are not supported in a cluster file system.. For more information, see Enabling Highly Available Local File Systems in Sun Cluster Data Services Planning and Administration Guide for Solaris OS.

---

Consider the following points when you plan cluster file systems.

• Quotas - Quotas are not supported on cluster file systems. However, quotas are supported on highly available local file systems.

• Non-global zones - If a cluster file system is to be accessed from a non-global zone, it must first be mounted in the global zone. The cluster file system is then mounted in the non-global zone by using a loopback mount. Therefore, the loopback file system (LOFS) must be enabled in a cluster that contains non-global zones.

• Loopback file system (LOFS) - During cluster creation with the Solaris 9 version of Sun Cluster software, LOFS is disabled by default. During cluster creation with the Solaris 10 version of Sun Cluster software, LOFS is enabled by default.

  You must manually disable LOFS on each cluster node if the cluster meets both of the following conditions:

  • Sun Cluster HA for NFS is configured on a highly available local file system.

  • The automountd daemon is running.

  If the cluster meets both of these conditions, you must disable LOFS to avoid switchover problems or other failures. If the cluster meets only one of these conditions, you can safely enable LOFS.

  If you require both LOFS and the automountd daemon to be enabled, exclude from the automounter map all files that are part of the highly available local file system that is exported by Sun Cluster HA for NFS.

• Process accounting log files - Do not locate process accounting log files on a cluster file system or on a highly available local file system. A switchover would be blocked by writes to the log file, which would cause the node to hang. Use only a local file system to contain process accounting log files.

• Communication endpoints - The cluster file system does not support any of the file-system features of Solaris software by which one would put a communication endpoint in the file-system namespace.

  • Although you can create a UNIX domain socket whose name is a path name into the cluster file system, the socket would not survive a node failover.

  • Any FIFOs or named pipes that you create on a cluster file system would not be globally accessible.

  Therefore, do not attempt to use the fattach command from any node other than the local node.

• Device special files - Neither block special files nor character special files are supported in a cluster file system. To specify a path name to a device node in a cluster file system, create a symbolic link to the device name in the /dev directory. Do not use the mknod command for this purpose.

• atime - Cluster file systems do not maintain atime.

• ctime - When a file on a cluster file system is accessed, the update of the file's ctime might be delayed.

• Installing applications - If you want the binaries of a highly available application to reside on a cluster file system, wait to install the application until after the cluster file system is configured. Also, if the application is installed by using the Sun Java System installer program and the application depends on any shared components, install those shared components on all nodes in the cluster that are not installed with the application.

Choosing Mount Options for Cluster File Systems

This section describes requirements and restrictions for the following types of cluster file systems:

• UFS Cluster File Systems

• VxFS Cluster File Systems

---

Note –

You can alternatively configure these and other types of file systems as highly available local file systems. For more information, see Enabling Highly Available Local File Systems in Sun Cluster Data Services Planning and Administration Guide for Solaris OS.

---

Follow these guidelines to determine what mount options to use when you create your cluster file systems.

UFS Cluster File Systems

Mount Option	Usage	Description
global	Required	This option makes the file system globally visible to all nodes in the cluster.
logging	Required	This option enables logging.
forcedirectio	Conditional	This option is required only for cluster file systems that will host Oracle RAC RDBMS data files, log files, and control files.  Note – Oracle RAC is supported for use only in SPARC based clusters.
onerror=panic	Required	You do not have to explicitly specify the onerror=panic mount option in the /etc/vfstab file. This mount option is already the default value if no other onerror mount option is specified. Note – Only the onerror=panic mount option is supported by Sun Cluster software. Do not use the onerror=umount or onerror=lock mount options. These mount options are not supported on cluster file systems for the following reasons: Use of the onerror=umount or onerror=lock mount option might cause the cluster file system to lock or become inaccessible. This condition might occur if the cluster file system experiences file corruption. The onerror=umount or onerror=lock mount option might cause the cluster file system to become unmountable. This condition might thereby cause applications that use the cluster file system to hang or prevent the applications from being killed. A node might require rebooting to recover from these states.
syncdir	Optional	If you specify syncdir, you are guaranteed POSIX-compliant file system behavior for the write() system call. If a write() succeeds, then this mount option ensures that sufficient space is on the disk. If you do not specify syncdir, the same behavior occurs that is seen with UFS file systems. When you do not specify syncdir, performance of writes that allocate disk blocks, such as when appending data to a file, can significantly improve. However, in some cases, without syncdir you would not discover an out-of-space condition (ENOSPC) until you close a file. You see ENOSPC on close only during a very short time after a failover. With syncdir, as with POSIX behavior, the out-of-space condition would be discovered before the close.

See the mount_ufs(1M) man page for more information about UFS mount options.

VxFS Cluster File Systems

Mount Option	Usage	Description
global	Required	This option makes the file system globally visible to all nodes in the cluster.
log	Required	This option enables logging.

See the VxFS mount_vxfs man page and Overview of the Administering Cluster File Systems in Sun Cluster System Administration Guide for Solaris OS for more information about VxFS mount options.

Mount Information for Cluster File Systems

Consider the following points when you plan mount points for cluster file systems.

• Mount-point location – Create mount points for cluster file systems in the /global directory, unless you are prohibited by other software products. By using the /global directory, you can more easily distinguish cluster file systems, which are globally available, from local file systems.

• SPARC: VxFS mount requirement – If you use VERITAS File System (VxFS), globally mount and unmount a VxFS file system from the primary node. The primary node is the node that masters the disk on which the VxFS file system resides. This method ensures that the mount or unmount operation succeeds. A VxFS file-system mount or unmount operation that is performed from a secondary node might fail.

• The following VxFS features are not supported in a Sun Cluster 3.2 cluster file system. They are, however, supported in a local file system.

  • Quick I/O

  • Snapshots

  • Storage checkpoints

  • VxFS-specific mount options:

    • convosync (Convert O_SYNC)

    • mincache

    • qlog, delaylog, tmplog

  • VERITAS cluster file system (requires VxVM cluster feature & VERITAS Cluster Server)

  Cache advisories can be used, but the effect is observed on the given node only.

  All other VxFS features and options that are supported in a cluster file system are supported by Sun Cluster 3.2 software. See VxFS documentation for details about VxFS options that are supported in a cluster configuration.

• Nesting mount points – Normally, you should not nest the mount points for cluster file systems. For example, do not set up one file system that is mounted on /global/a and another file system that is mounted on /global/a/b. To ignore this rule can cause availability and node boot-order problems. These problems would occur if the parent mount point is not present when the system attempts to mount a child of that file system. The only exception to this rule is if the devices for the two file systems have the same physical node connectivity. An example is different slices on the same disk.

• forcedirectio - Sun Cluster software does not support the execution of binaries off cluster file systems that are mounted by using the forcedirectio mount option.

Planning Volume Management

Add this planning information to the Device Group Configurations Worksheet and the Volume-Manager Configurations Worksheet. For Solaris Volume Manager, also add this planning information to the Volumes Worksheet (Solaris Volume Manager).

This section provides the following guidelines for planning volume management of your cluster configuration:

• Guidelines for Volume-Manager Software

• Guidelines for Solaris Volume Manager Software

• Guidelines for VERITAS Volume Manager Software

• File-System Logging

• Mirroring Guidelines

Sun Cluster software uses volume-manager software to group disks into device groups which can then be administered as one unit. Sun Cluster software supports Solaris Volume Manager software and VERITAS Volume Manager (VxVM) software that you install or use in the following ways.

Table 1–4 Supported Use of Volume Managers With Sun Cluster Software

Volume-Manager Software	Requirements
Solaris Volume Manager	You must install Solaris Volume Manager software on all nodes of the cluster, regardless of whether you use VxVM on some nodes to manage disks.
SPARC: VxVM with the cluster feature	You must install and license VxVM with the cluster feature on all nodes of the cluster.
VxVM without the cluster feature	You are only required to install and license VxVM on those nodes that are attached to storage devices which VxVM manages.
Both Solaris Volume Manager and VxVM	If you install both volume managers on the same node, you must use Solaris Volume Manager software to manage disks that are local to each node. Local disks include the root disk. Use VxVM to manage all shared disks.

See your volume-manager documentation and Configuring Solaris Volume Manager Software or Installing and Configuring VxVM Software for instructions about how to install and configure the volume-manager software. For more information about the use of volume management in a cluster configuration, see Multihost Devices in Sun Cluster Concepts Guide for Solaris OS and Device Groups in Sun Cluster Concepts Guide for Solaris OS.

Guidelines for Volume-Manager Software

Consider the following general guidelines when you configure your disks with volume-manager software:

• Software RAID – Sun Cluster software does not support software RAID 5.

• Mirrored multihost disks – You must mirror all multihost disks across disk expansion units. See Guidelines for Mirroring Multihost Disks for guidelines on mirroring multihost disks. You do not need to use software mirroring if the storage device provides hardware RAID as well as redundant paths to devices.

• Mirrored root – Mirroring the root disk ensures high availability, but such mirroring is not required. See Mirroring Guidelines for guidelines about deciding whether to mirror the root disk.

• Unique naming – You might have local Solaris Volume Manager or VxVM volumes that are used as devices on which the /global/.devices/node@nodeid file systems are mounted. If so, the name of each local volume on which a /global/.devices/node@nodeid file system is to be mounted must be unique throughout the cluster.

• Node lists – To ensure high availability of a device group, make its node lists of potential masters and its failback policy identical to any associated resource group. Or, if a scalable resource group uses more nodes or zones than its associated device group, make the scalable resource group's node list a superset of the device group's node list. See the resource group planning information in the Sun Cluster Data Services Planning and Administration Guide for Solaris OS for information about node lists.

• Multihost disks – You must connect, or port, all devices that are used to construct a device group to all of the nodes that are configured in the node list for that device group. Solaris Volume Manager software can automatically check for this connection at the time that devices are added to a disk set. However, configured VxVM disk groups do not have an association to any particular set of nodes.

• Hot spare disks – You can use hot spare disks to increase availability, but hot spare disks are not required.

See your volume-manager documentation for disk layout recommendations and any additional restrictions.

Guidelines for Solaris Volume Manager Software

Consider the following points when you plan Solaris Volume Manager configurations:

• Local volume names – The name of each local Solaris Volume Manager volume on which a global-devices file system, /global/.devices/node@nodeid, is mounted must be unique throughout the cluster. Also, the name cannot be the same as any device-ID name.

• Dual-string mediators – Each disk set configured with exactly two disk strings and mastered by exactly two nodes must have Solaris Volume Manager mediators configured for the disk set. A disk string consists of a disk enclosure, its physical disks, cables from the enclosure to the node or nodes, and the interface adapter cards. Observe the following rules to configure dual-string mediators:

  • You must configure each disk set with exactly two nodes that act as mediator hosts.

  • You must use the same two nodes for all disk sets that require mediators. Those two nodes must master those disk sets.

  • Mediators cannot be configured for disk sets that do not meet the two-string and two-host requirements.

  See the mediator(7D) man page for details.

• /kernel/drv/md.conf settings – SPARC: On the Solaris 9 OS, Solaris Volume Manager volumes that are used by each disk set are created in advance, at reconfiguration boot time. This reconfiguration is based on the configuration parameters that exist in the /kernel/drv/md.conf file.

  ---

  Note –

  With the Solaris 10 release, Solaris Volume Manager has been enhanced to configure volumes dynamically. You no longer need to edit the nmd and the md_nsets parameters in the /kernel/drv/md.conf file. New volumes are dynamically created, as needed.

  ---

  You must modify the nmd and md_nsets fields as follows to support a Sun Cluster configuration on the Solaris 9 OS:

  ---

  Caution –

  All cluster nodes must have identical /kernel/drv/md.conf files, regardless of the number of disk sets that are served by each node. Failure to follow this guideline can result in serious Solaris Volume Manager errors and possible loss of data.

  ---

  • md_nsets – The md_nsets field defines the total number of disk sets that can be created for a system to meet the needs of the entire cluster. Set the value of md_nsets to the expected number of disk sets in the cluster plus one additional disk set. Solaris Volume Manager software uses the additional disk set to manage the private disks on the local host.

    The maximum number of disk sets that are allowed per cluster is 32. This number allows for 31 disk sets for general use plus one disk set for private disk management. The default value of md_nsets is 4.

  • nmd – The nmd field defines the highest predicted value of any volume name that will exist in the cluster. For example, if the highest value of the volume names that are used in the first 15 disk sets of a cluster is 10, but the highest value of the volume in the 16th disk set is 1000, set the value of nmd to at least 1000. Also, the value of nmd must be large enough to ensure that enough numbers exist for each device–ID name. The number must also be large enough to ensure that each local volume name can be unique throughout the cluster.

    The highest allowed value of a volume name per disk set is 8192. The default value of nmd is 128.

  Set these fields at installation time to allow for all predicted future expansion of the cluster. To increase the value of these fields after the cluster is in production is time consuming. The value change requires a reconfiguration reboot for each node. To raise these values later also increases the possibility of inadequate space allocation in the root (/) file system to create all of the requested devices.

  At the same time, keep the value of the nmdfield and the md_nsets field as low as possible. Memory structures exist for all possible devices as determined by nmdand md_nsets, even if you have not created those devices. For optimal performance, keep the value of nmd and md_nsets only slightly higher than the number of volumes that you plan to use.

  See System Files and Startup Files in Solaris Volume Manager Administration Guide (Solaris 9 or Solaris 10) for more information about the md.conf file.

Guidelines for VERITAS Volume Manager Software

Consider the following points when you plan VERITAS Volume Manager (VxVM) configurations.

• Accessibility to nodes - You must configure all volume-manager disk groups as either Sun Cluster device groups or as local-only disk groups. If you do not configure the disk group in one of these ways, the devices in the disk group will not be accessible to any node in the cluster.

  • A device group enables a secondary node to host multihost disks if the primary node fails.

  • A local-only disk group functions outside the control of Sun Cluster software and can be accessed from only one node at a time.

• Enclosure-Based Naming – If you use Enclosure-Based Naming of devices, ensure that you use consistent device names on all cluster nodes that share the same storage. VxVM does not coordinate these names, so the administrator must ensure that VxVM assigns the same names to the same devices from different nodes. Failure to assign consistent names does not interfere with correct cluster behavior. However, inconsistent names greatly complicate cluster administration and greatly increase the possibility of configuration errors, potentially leading to loss of data.

• Root disk group – The creation of a root disk group is optional.

  A root disk group can be created on the following disks:

  • The root disk, which must be encapsulated

  • One or more local nonroot disks, which you can encapsulate or initialize

  • A combination of root and local nonroot disks

  The root disk group must be local to the node.

• Simple root disk groups – Simple root disk groups (rootdg created on a single slice of the root disk) are not supported as disk types with VxVM on Sun Cluster software. This is a general VxVM software restriction.

• Encapsulation – Disks to be encapsulated must have two disk-slice table entries free.

• Number of volumes – Estimate the maximum number of volumes any given device group can use at the time the device group is created.

  • If the number of volumes is less than 1000, you can use default minor numbering.

  • If the number of volumes is 1000 or greater, you must carefully plan the way in which minor numbers are assigned to device group volumes. No two device groups can have overlapping minor number assignments.

• Dirty Region Logging – The use of Dirty Region Logging (DRL) decreases volume recovery time after a node failure. Using DRL might decrease I/O throughput.

• Dynamic Multipathing (DMP) – The use of DMP alone to manage multiple I/O paths per node to the shared storage is not supported. The use of DMP is supported only in the following configurations:

  • A single I/O path per node to the cluster's shared storage.

  • A supported multipathing solution, such as Sun Traffic Manager, EMC PowerPath, or Hitachi HDLM, that manages multiple I/O paths per node to the shared cluster storage.

See your VxVM installation documentation for additional information.

File-System Logging

Logging is required for UFS and VxFS cluster file systems. This requirement does not apply to QFS shared file systems. Sun Cluster software supports the following choices of file-system logging:

• Solaris UFS logging – See the mount_ufs(1M) man page for more information.

• (Solaris 9 only) SPARC: Solaris Volume Manager transactional-volume logging – See Transactional Volumes (Overview) in Solaris Volume Manager Administration Guide for more information.

  ---

  Note –

  Solaris Volume Manager transactional-volume logging is removed from the Solaris 10 OS. Solaris UFS logging provides the same capabilities but superior performance, as well as lower system administration requirements and overhead.

  ---

• SPARC: VERITAS File System (VxFS) logging – See the mount_vxfs man page provided with VxFS software for more information.

The following table lists the file-system logging supported by each volume manager.

Table 1–5 Supported File System Logging Matrix

Volume Manager	Supported File System Logging
Solaris Volume Manager	Solaris UFS logging SPARC: Solaris Volume Manager transactional-volume logging (Solaris 9 only) VxFS logging
VERITAS Volume Manager	Solaris UFS logging VxFS logging

SPARC: On the Solaris 9 OS, consider the following points when you choose between Solaris UFS logging and Solaris Volume Manager transactional-volume logging for UFS cluster file systems:

• Solaris UFS logging always allocates the log by using free space on the UFS file system, and depending on the size of the file system.

  • On file systems less than 1 Gbyte, the log occupies 1 Mbyte.

  • On file systems 1 Gbyte or greater, the log occupies 1 Mbyte per Gbyte on the file system, to a maximum of 64 Mbytes.

• A transactional volume manages UFS logging. The logging device component of a transactional volume is a volume that you can mirror and stripe. You can create a maximum 1-Gbyte log size, although 64 Mbytes is sufficient for most file systems. The minimum log size is 1 Mbyte.

Mirroring Guidelines

This section provides the following guidelines for planning the mirroring of your cluster configuration:

• Guidelines for Mirroring Multihost Disks

• Guidelines for Mirroring the Root Disk

Guidelines for Mirroring Multihost Disks

To mirror all multihost disks in a Sun Cluster configuration enables the configuration to tolerate single-device failures. Sun Cluster software requires that you mirror all multihost disks across expansion units. You do not need to use software mirroring if the storage device provides hardware RAID as well as redundant paths to devices.

Consider the following points when you mirror multihost disks:

• Separate disk expansion units – Each submirror of a given mirror or plex should reside in a different multihost expansion unit.

• Disk space – Mirroring doubles the amount of necessary disk space.

• Three-way mirroring – Solaris Volume Manager software and VERITAS Volume Manager (VxVM) software support three-way mirroring. However, Sun Cluster software requires only two-way mirroring.

• Differing device sizes – If you mirror to a device of a different size, your mirror capacity is limited to the size of the smallest submirror or plex.

For more information about multihost disks, see Multihost Disk Storage in Sun Cluster Overview for Solaris OS and Sun Cluster Concepts Guide for Solaris OS.

Guidelines for Mirroring the Root Disk

Add this planning information to the Local File System Layout Worksheet.

For maximum availability, mirror root (/), /usr, /var, /opt, and swap on the local disks. Under VxVM, you encapsulate the root disk and mirror the generated subdisks. However, Sun Cluster software does not require that you mirror the root disk.

Before you decide whether to mirror the root disk, consider the risks, complexity, cost, and service time for the various alternatives that concern the root disk. No single mirroring strategy works for all configurations. You might want to consider your local Sun service representative's preferred solution when you decide whether to mirror root.

See your volume-manager documentation and Configuring Solaris Volume Manager Software or Installing and Configuring VxVM Software for instructions about how to mirror the root disk.

Consider the following points when you decide whether to mirror the root disk.

• Boot disk – You can set up the mirror to be a bootable root disk. You can then boot from the mirror if the primary boot disk fails.

• Complexity – To mirror the root disk adds complexity to system administration. To mirror the root disk also complicates booting in single-user mode.

• Backups – Regardless of whether you mirror the root disk, you also should perform regular backups of root. Mirroring alone does not protect against administrative errors. Only a backup plan enables you to restore files that have been accidentally altered or deleted.

• Quorum devices – Do not use a disk that was configured as a quorum device to mirror a root disk.

• Quorum – Under Solaris Volume Manager software, in failure scenarios in which state database quorum is lost, you cannot reboot the system until maintenance is performed. See your Solaris Volume Manager documentation for information about the state database and state database replicas.

• Separate controllers – Highest availability includes mirroring the root disk on a separate controller.

• Secondary root disk – With a mirrored root disk, the primary root disk can fail but work can continue on the secondary (mirror) root disk. Later, the primary root disk might return to service, for example, after a power cycle or transient I/O errors. Subsequent boots are then performed by using the primary root disk that is specified for the eeprom(1M) boot-device parameter. In this situation, no manual repair task occurs, but the drive starts working well enough to boot. With Solaris Volume Manager software, a resync does occur. A resync requires a manual step when the drive is returned to service.

  If changes were made to any files on the secondary (mirror) root disk, they would not be reflected on the primary root disk during boot time. This condition would cause a stale submirror. For example, changes to the /etc/system file would be lost. With Solaris Volume Manager software, some administrative commands might have changed the /etc/system file while the primary root disk was out of service.

  The boot program does not check whether the system is booting from a mirror or from an underlying physical device. The mirroring becomes active partway through the boot process, after the volumes are loaded. Before this point, the system is therefore vulnerable to stale submirror problems.