Planning the Oracle Solaris OS

This section provides the following guidelines for planning Oracle Solaris software installation in a cluster configuration.

• Guidelines for Selecting Your Oracle Solaris Installation Method

• Oracle Solaris OS Feature Restrictions

• Oracle Solaris Software Group Considerations

• System Disk Partitions

• Guidelines for Non-Global Zones in a Global Cluster

• SPARC: Guidelines for Oracle VM Server for SPARC in a Cluster

For more information about Oracle Solaris software, see your Oracle Solaris installation documentation.

Guidelines for Selecting Your Oracle Solaris Installation Method

You can install Oracle Solaris software from a local DVD-ROM or from a network installation server by using the Oracle Solaris JumpStart installation method. In addition, Oracle Solaris Cluster software provides a custom method for installing both the Oracle Solaris OS and Oracle Solaris Cluster software by using the JumpStart installation method. If you are installing several cluster nodes, consider a network installation.

See How to Install Oracle Solaris and Oracle Solaris Cluster Software (JumpStart) for details about the scinstall JumpStart installation method. See your Oracle Solaris installation documentation for details about standard Oracle Solaris installation methods.

Oracle Solaris OS Feature Restrictions

Consider the following points when you plan the use of the Oracle Solaris OS in an Oracle Solaris Cluster configuration:

• Oracle Solaris Zones – Install Oracle Solaris Cluster framework software only in the global zone.

  To determine whether you can install an Oracle Solaris Cluster data service directly in a non-global zone, see the documentation for that data service.

  If you configure non-global zones on a global-cluster node, the loopback file system (LOFS) must be enabled. See the information for LOFS for additional considerations.

• Loopback file system (LOFS) – During cluster creation, LOFS capability is enabled by default. If the cluster meets both of the following conditions, you must disable LOFS to avoid switchover problems or other failures:

  • Oracle Solaris Cluster HA for NFS (HA for NFS) is configured on a highly available local file system.

  • The automountd daemon is running.

  If the cluster meets only one of these conditions, you can safely enable LOFS.

  If you require both LOFS and the automountd daemon to be enabled, exclude from the automounter map all files that are part of the highly available local file system that is exported by HA for NFS.

• Power-saving shutdown – Automatic power-saving shutdown is not supported in Oracle Solaris Cluster configurations and should not be enabled. See the pmconfig(1M) and power.conf(4) man pages for more information.

• IP Filter feature – Oracle Solaris Cluster software does not support the Oracle Solaris IP Filter feature for scalable services, but does support Oracle Solaris IP Filter for failover services. Observe the following guidelines and restrictions when you configure Oracle Solaris IP Filter in a cluster:

  • NAT routing is not supported.

  • The use of NAT for translation of local addresses is supported. NAT translation rewrites packets on-the-wire and is therefore transparent to the cluster software.

  • Stateful filtering rules are not supported; only stateless filtering is supported. Oracle Solaris Cluster relies on IP network multipathing (IPMP) for public network monitoring, which does not work with stateful filtering rules.

• fssnap – Oracle Solaris Cluster software does not support the fssnap command, which is a feature of UFS. However, you can use the fssnap command on local systems that are not controlled by Oracle Solaris Cluster software. The following restrictions apply to fssnap support:

  • The fssnap command is supported on local files systems that are not managed by Oracle Solaris Cluster software.

  • The fssnap command is not supported on cluster file systems.

  • The fssnap command is not supported on local file systems under the control of HAStoragePlus.

Oracle Solaris Software Group Considerations

Oracle Solaris Cluster 3.3 3/13 software requires at least the End User Oracle Solaris Software Group (SUNWCuser). However, other components of your cluster configuration might have their own Oracle Solaris software requirements as well. Consider the following information when you decide which Oracle Solaris software group you are installing.

• Servers – Check your server documentation for any Oracle Solaris software requirements.

• Additional Oracle Solaris packages – You might need to install other Oracle Solaris software packages that are not part of the End User Oracle Solaris Software Group. The Apache HTTP server packages and Trusted Extensions software are two examples that require packages that are in a higher software group than End User. Third-party software might also require additional Oracle Solaris software packages. See your third-party documentation for any Oracle Solaris software requirements.

  ---

  Tip - To avoid the need to manually install Oracle Solaris software packages, install the Entire Oracle Solaris Software Group Plus OEM Support.

  ---

• Oracle Solaris package minimization – See Article 1544605.1 Solaris Cluster and Solaris OS Minimization Support Required Packages Group at for information.

System Disk Partitions

When you install the Oracle Solaris OS, ensure that you create the required Oracle Solaris Cluster partitions and that all partitions meet minimum space requirements.

• swap – The combined amount of swap space that is allocated for Oracle Solaris and Oracle Solaris Cluster software must be no less than 750 Mbytes. For best results, add at least 512 Mbytes for Oracle Solaris Cluster software to the amount that is required by the Oracle Solaris OS. In addition, allocate any additional swap amount that is required by applications that are to run on the Oracle Solaris host.

  ---

  Note - If you create an additional swap file, do not create the swap file on a global device. Use only a local disk as a swap device for the host.

  ---

• (Optional) /globaldevices – By default, a lofi device is used for the global devices namespace. However, you can alternatively create a file system at least 512 Mbytes large that is to be used by the scinstall utility for global devices. You must name this file system /globaldevices.

  Functionality and performance are equivalent for both choices. However, a lofi device provides greater ease of use and more flexibility in situations where a disk partition is not available for use.

• Volume manager – Create a 20-Mbyte partition on slice 7 for volume manager use.

To meet these requirements, you must customize the partitioning if you are performing interactive installation of the Oracle Solaris OS.

See the following guidelines for additional partition planning information:

• Guidelines for the Root (/) File System

• Guidelines for the /globaldevices File System

• Volume Manager Requirements

Guidelines for the Root (/) File System

As with any other system running the Oracle Solaris OS, you can configure the root (/), /var, /usr, and /opt directories as separate file systems. Or, you can include all the directories in the root (/) file system.

The following describes the software contents of the root (/), /var, /usr, and /opt directories in an Oracle Solaris Cluster configuration. Consider this information when you plan your partitioning scheme.

• root (/) – The Oracle Solaris Cluster software itself occupies less than 40 Mbytes of space in the root (/) file system. Solaris Volume Manager software requires less than 5 Mbytes. To configure ample additional space and inode capacity, add at least 100 Mbytes to the amount of space you would normally allocate for your root ( /) file system. This space is used for the creation of both block special devices and character special devices used by the volume management software. You especially need to allocate this extra space if a large number of shared disks are in the cluster.

  On the Oracle Solaris 10 OS, the lofi device for the global-devices namespace requires 100 MBytes of free space.

• /var – The Oracle Solaris Cluster software occupies a negligible amount of space in the /var file system at installation time. However, you need to set aside ample space for log files. Also, more messages might be logged on a clustered node than would be found on a typical stand-alone server. Therefore, allow at least 100 Mbytes for the /var file system.

• /usr – Oracle Solaris Cluster software occupies less than 25 Mbytes of space in the /usr file system. Solaris Volume Manager software requires less than 15 Mbytes.

• /opt – Oracle Solaris Cluster framework software uses less than 2 Mbytes in the /opt file system. However, each Oracle Solaris Cluster data service might use between 1 Mbyte and 5 Mbytes. Solaris Volume Manager software does not use any space in the /opt file system.

  In addition, most database and applications software is installed in the /opt file system.

Guidelines for the /globaldevices File System

Oracle Solaris Cluster software offers two choices of locations to host the global-devices namespace:

• A lofi device, which is the default

• A dedicated file system on one of the local disks

When you use a lofi device for the global-devices namespace, observe the following requirements:

• Dedicated use – The lofi device that hosts the global-devices namespace cannot be used for any other purpose. If you need a lofi device for some other use, create a new lofi device for that purpose.

• Mount requirement – The lofi device must not be unmounted.

• Namespace identification – After the cluster is configured, you can use the lofiadm command to identify the lofi device that corresponds to the global-devices namespace, /.globaldevices.

If you instead configure a dedicated /globaldevices for the global-devices namespace, observe the following guidelines and requirements:

• Location - The /globaldevices file system is usually located on your root disk. However, if you use different storage on which to locate the global-devices file system, such as a Logical Volume Manager volume, it must not be part of a Solaris Volume Manager shared disk set. This file system is later mounted as a UFS cluster file system. Name this file system /globaldevices, which is the default name that is recognized by the scinstall(1M) command.

• Required file-system type - No file-system type other than UFS is valid for the global-devices file system. Do not attempt to change the file-system type after the global-devices file system is created.

  However, a UFS global-devices file system can coexist on a node with other root file systems that use ZFS.

• Configured namespace name- The scinstall command later renames the file system /global/.devices/node@nodeid, where nodeid represents the number that is assigned to an Oracle Solaris host when it becomes a global-cluster member. The original /globaldevices mount point is removed.

• Space requirements - The /globaldevices file system must have ample space and ample inode capacity for creating both block special devices and character special devices. This guideline is especially important if a large number of disks are in the cluster. Create a file system size of at least 512 Mbytes and a density of 512, as follows:

  # newfs -i 512 globaldevices-partition

  This number of inodes should suffice for most cluster configurations.

Volume Manager Requirements

For Solaris Volume Manager software, you must set aside a slice on the root disk for use in creating the state database replica. Specifically, set aside a slice for this purpose on each local disk. But, if you have only one local disk on an Oracle Solaris host, you might need to create three state database replicas in the same slice for Solaris Volume Manager software to function properly. See your Solaris Volume Manager documentation for more information.

Example – Sample File-System Allocations

Table 1-2 shows a partitioning scheme for an Oracle Solaris host that has less than 750 Mbytes of physical memory. This scheme is to be installed with the End User Oracle Solaris Software Group, Oracle Solaris Cluster software, and the Oracle Solaris Cluster HA for NFS data service. The last slice on the disk, slice 7, is allocated with a small amount of space for volume-manager use.

If you use a lofi device for the global-devices namespace, slice 3 can be used for another purpose or left labeled as unused.

If you use Solaris Volume Manager software, you use slice 7 for the state database replica. This layout provides the necessary two free slices, 4 and 7, as well as provides for unused space at the end of the disk.

Table 1-2 Example File-System Allocation

Guidelines for Non-Global Zones in a Global Cluster

For information about the purpose and function of Oracle Solaris zones in a cluster, see Support for Oracle Solaris Zones in Oracle Solaris Cluster Concepts Guide.

For guidelines about configuring a cluster of non-global zones, see Zone Clusters.

Consider the following points when you create an Oracle Solaris 10 non-global zone, simply referred to as a zone, on a global-cluster node.

• Unique zone name – The zone name must be unique on the Oracle Solaris host.

• Reusing a zone name on multiple nodes – To simplify cluster administration, you can use the same name for a zone on each node where resource groups are to be brought online in that zone.

• Private IP addresses – Do not attempt to use more private IP addresses than are available in the cluster.

• Mounts – Do not include global mounts in zone definitions. Include only loopback mounts.

• Failover services – In multiple-host clusters, while Oracle Solaris Cluster software permits you to specify different zones on the same Oracle Solaris host in a failover resource group's node list, doing so is useful only during testing. If a single host contains all zones in the node list, the node becomes a single point of failure for the resource group. For highest availability, zones in a failover resource group's node list should be on different hosts.

  In single-host clusters, no functional risk is incurred if you specify multiple zones in a failover resource group's node list.

• Scalable services – Do not create non-global zones for use in the same scalable service on the same Oracle Solaris host. Each instance of the scalable service must run on a different host.

• Cluster file systems - For cluster file systems that use UFS, do not directly add a cluster file system to a non-global zone by using the zonecfs command. Instead, configure an HAStoragePlus resource, which manages the mounting of the cluster file system in the global zone and performs a loopback mount of the cluster file system in the non-global zone.

• LOFS – Oracle Solaris Zones requires that the loopback file system (LOFS) be enabled. However, the Oracle Solaris Cluster HA for NFS data service requires that LOFS be disabled, to avoid switchover problems or other failures. If you configure both non-global zones and Oracle Solaris Cluster HA for NFS in your cluster, do one of the following to prevent possible problems in the data service:

  • Disable the automountd daemon.

  • Exclude from the automounter map all files that are part of the highly available local file system that is exported by Oracle Solaris Cluster HA for NFS.

• Exclusive-IP zones – The following guidelines apply specifically to exclusive-IP non-global zones:

  • Logical-hostname resource groups – In a resource group that contains a LogicalHostname resource, if the node list contains any non-global zone with the ip-type property set to exclusive, all zones in that node list must have this property set to exclusive. Note that a global zone always has the ip-type property set to shared, and therefore cannot coexist in a node list that contains zones of ip-type=exclusive. This restriction applies only to versions of the Oracle Solaris OS that use the Oracle Solaris zones ip-type property.

  • IPMP groups – For all public-network adapters that are used for data-service traffic in the non-global zone, you must manually configure IPMP groups in all /etc/hostname.adapter files on the zone. This information is not inherited from the global zone. For guidelines and instructions to configure IPMP groups, follow the procedures in Part V, IPMP, in Oracle Solaris Administration: IP Services.

  • Private-hostname dependency - Exclusive-IP zones cannot depend on the private hostnames and private addresses of the cluster.

  • Shared-address resources – Shared-address resources cannot use exclusive-IP zones.

SPARC: Guidelines for Oracle VM Server for SPARC in a Cluster

Consider the following points when you create an Oracle VM Server for SPARC I/O domain or guest domain on a physically clustered machine that is SPARC hypervisor capable:

• SCSI LUN requirement – The virtual shared storage device, or virtual disk back end, of an Oracle VM Server for SPARC guest domain must be a full SCSI LUN in the I/O domain. You cannot use an arbitrary virtual device.

• Fencing – Do not export a storage LUN to more than one guest domain on the same physical machine, unless you also disable fencing for that device. Otherwise, if two different guest domains on the same machine both are visible to a device, the device will be fenced whenever one of the guest domains dies. The fencing of the device will panic any other guest domain that subsequently tries to access the device.

• Network isolation – Guest domains that are located on the same physical machine but are configured in different clusters must be network isolated from each other. Use one of the following methods:

  • Configure the clusters to use different network interfaces in the I/O domain for the private network.

  • Use different network addresses for each of the clusters.

• Networking in guest domains – Network packets to and from guest domains must traverse service domains to reach the network drivers through virtual switches. Virtual switches use kernel threads that run at system priority. The virtual-switch threads must be able to acquire needed CPU resources to perform critical cluster operations, including heartbeats, membership, checkpoints, and so forth. Configuring virtual switches with the mode=sc setting enables expedited handling of cluster heartbeat packets. However, the reliability of other critical cluster operations can be enhanced by adding more CPU resources to the service domain under the following workloads:

  • High-interrupt load, for example, due to network or disk I/O. Under extreme load, virtual switches can preclude system threads from running for a long time, including virtual-switch threads.

  • Real-time threads that are overly aggressive in retaining CPU resources. Real-time threads run at a higher priority than virtual-switch threads, which can restrict CPU resources for virtual-switch threads for an extended time.

• Non-shared storage - For non-shared storage, such as for Oracle VM Server for SPARC guest-domain OS images, you can use any type of virtual device. You can back such virtual devices by any implement in the I/O domain, such as files or volumes. However, do not copy files or clone volumes in the I/O domain for the purpose of mapping them into different guest domains of the same cluster. Such copying or cloning would lead to problems because the resulting virtual devices would have the same device identity in different guest domains. Always create a new file or device in the I/O domain, which would be assigned a unique device identity, then map the new file or device into a different guest domain.

• Exporting storage from I/O domains – If you configure a cluster that is composed of Oracle VM Server for SPARC I/O domains, do not export its storage devices to other guest domains that also run Oracle Solaris Cluster software.

• Oracle Solaris I/O multipathing – Do not run Oracle Solaris I/O multipathing software (MPxIO) from guest domains. Instead, run Oracle Solaris I/O multipathing software in the I/O domain and export it to the guest domains.

• Virtual disk multipathing - Do not configure the virtual disk multipathing feature of Oracle VM Server for SPARC on a logical domain that is configured as a cluster node.

• Private-interconnect IP address range – The private network is shared by all guest domains that are created on the same physical machine and it is visible to all these domains. Before you specify a private-network IP address range to the scinstall utility for use by a guest-domain cluster, ensure that the address range is not already in use by another guest domain on the same physical machine.

For more information about Oracle VM Server for SPARC, see the Logical Domains (LDoms) 1.0.3 Administration Guide.