How to Create a Zone Cluster (clsetup)

How to Create a Zone Cluster (`clsetup`)

Create a global cluster. See Establishing the Global Cluster.
Read the guidelines and requirements for creating a zone cluster. See Zone Clusters.
If you plan to use a zone cluster configuration profile when creating a solaris or labeled brand zone cluster, ensure that the file is created and the file name has the .xml extension. See the Example section of the clzonecluster(8CL) man page for an example of the profile contents.
If the zone cluster will use Trusted Extensions, ensure that you have installed, configured, and enabled Trusted Extensions as described in How to Install and Configure Trusted Extensions.
If the cluster does not have sufficient subnets available to add a zone cluster, you must modify the private IP address range to provide the needed subnets. For more information, see How to Change the Private Network Address or Address Range of an Existing Cluster in Administering an Oracle Solaris Cluster 4.4 Configuration.
Have available the following information:
- The unique name to assign to the zone cluster.
  
  Note:
  If Trusted Extensions is enabled, the zone cluster name must be the same name as a Trusted Extensions security label that has the security levels that you want to assign to the zone cluster. Create a separate zone cluster for each Trusted Extensions security label that you want to use.
- The zone path that the nodes of the zone cluster will use. For more information, see the description of the zonepath property in Configurable Resource Types and Global Properties in Oracle Solaris Zones Configuration Resources. By default, whole-root zones are created.
- The name of each node in the global cluster on which to create a zone-cluster node.
- The zone public hostname, or host alias, that you assign to each zone-cluster node.
- If applicable, the public-network IP address that each zone-cluster node uses. Specifying an IP address and NIC for each zone cluster node is required if the zone cluster will be used in a Disaster Recovery Framework configuration. Otherwise, this requirement is optional. For more information about this Disaster Recovery Framework requirement, see Disaster Recovery Framework.
- If applicable, the name of the public network management object that each zone-cluster node uses to connect to the public network. For a solaris10 branded exclusive-IP zone cluster, you can only use an IPMP group as the public network management object.
Note:
If you do not configure an IP address for each zone cluster node, two things will occur:
- That specific zone cluster will not be able to configure NAS devices for use in the zone cluster. The cluster uses the IP address of the zone cluster node when communicating with the NAS device, so not having an IP address prevents cluster support for fencing NAS devices.
- The cluster software will activate any Logical Host IP address on any NIC.

Tip:

While in the clsetup utility, you can press the < key to return to a previous screen.

You can also use Oracle Solaris Cluster Manager to create a zone cluster. For the browser interface log-in instructions, see How to Access Oracle Solaris Cluster Manager in Administering an Oracle Solaris Cluster 4.4 Configuration.

Perform this procedure to create a zone cluster using the clsetup utility.

To modify the zone cluster after it is installed, see Performing Zone Cluster Administrative Tasks in Administering an Oracle Solaris Cluster 4.4 Configuration and the clzonecluster(8CL) man page.

Note:

You cannot change the zone cluster name after the zone cluster is created.

Note:

When a zone is booted into a zone cluster, do not use the zoneadm command to halt or reboot the zone. Instead, use the clzonecluster halt and clzonecluster reboot commands. The zoneadm halt and zoneadm reboot commands do not synchronize the shutdown of objects that Oracle Solaris Cluster manages. So, these halting actions might cause applications in the zone or the zone to enter a faulted state.

Assume the root role on an active member node of a global cluster.

You perform all steps of this procedure from a node of the global cluster.

Ensure that the node of the global cluster is in cluster mode.

phys-schost# clnode status
=== Cluster Nodes ===

--- Node Status ---

Node Name                                       Status
---------                                       ------
phys-schost-2                                   Online
phys-schost-1                                   Online

Start the clsetup utility.
```
phys-schost# clsetup
```
The Main Menu is displayed.
Choose the Zone Cluster menu item.
Choose the Create a Zone Cluster menu item.
Type the name of the zone cluster you want to add.

A zone cluster name can contain ASCII letters (a-z and A-Z), numbers, a dash, or an underscore. The maximum length of the name is 20 characters.

Choose the property you want to change.

You can set the following properties:

Property	Description
`zonepath=zone-cluster-node-path`	Specifies the path to the zone cluster node. For example, `/zones/sczone`.
`brand=brand-type`	Specifies the `solaris`, `solaris10`, or `labeled` zones brand used in the zone cluster. Note: To use Trusted Extensions, you must use only the `labeled` brand. To create an exclusive-IP `solaris10` brand zone cluster, set the properties using the `clzonecluster create` command as follows: cz1> set brand=solaris10 cz1> set ip-type=exclusive
`ip-type=value`	Specifies the type of network IP address used by the zone cluster. Valid ip-type values are shared and exclusive. The maximum number of exclusive-IP zone clusters is constrained by the `cluster` property `num_xip_zoneclusters`, which you can set during initial cluster installation. This value has a default of three. For more information, see the `cluster`(8CL) man page.
`enable_priv_net=value`	When set to `true`, Oracle Solaris Cluster private network communication is enabled between the nodes of the zone cluster. The Oracle Solaris Cluster private hostnames and IP addresses for the zone cluster nodes are automatically generated by the system. Private network communication is disabled if the value is set to `false`. The default value is `true`. When the enable_priv_net property is set to `true` along with the following properties, private communication occurs in the following ways: ip-type=`shared` – Communication between zone cluster nodes uses the private networks of the global cluster. ip-type=`exclusive` (`solaris` brand only) – Communication between zone cluster nodes uses the specified `privnet` resources. The `privnet` resources are either Virtual Network Interfaces (VNICs) for the Ethernet type of private network adapters, or InfiniBand (IB) partitions for the IB type of private network adapters. The VNICs or IB partitions are automatically created by the wizard over each private network adapter of the global cluster, and used to configure a zone cluster. The VNICs or IB partitions that the wizard generates use the following naming conventions: For the Ethernet type: `private-network-interface-name_zone-cluster-name_vnic0`. For the IB type: `private-network-interface-name_zone-cluster-name_ibp0`. For example, the private network interfaces of the global cluster are `net2` and `net3`, and the zone cluster name is `zone1`. If `net2` and `net3` are Ethernet type network interfaces, the two VNICs that are created for the zone cluster will have the names `net2_zone1_vnic0` and `net3_zone1_vnic0`. If `net2` and `net3` are IB type network interfaces, the two IB partitions created for the zone cluster will have the names `net2_zone1_ibp0` and `net3_zone1_ibp0`.
`enable_scalable_svc=`value	The default value for this property is set to false. This property can be set only when the zone cluster is not in a running state. You will need to halt all the zone cluster nodes before you can change the property settings. If a scalable service is to be hosted on a zone cluster, this property must be set to true before the scalable service resource group and resources are created for the zone cluster. If this property has to be set to false, any scalable service already configured for a zone cluster must be removed before setting the property to false.

For a solaris10 brand zone cluster, enter a zone root password.

A root account password is required for a solaris10 brand zone.

Choose the Zone System Resource Control property that you want to change.

You can set the following properties:

Property	Description
`max-lwps=value`	Specifies the maximum number of lightweight processes (LWPs) simultaneously available to this zone cluster.
`max-shm-memory=value`	Specifies the maximum amount of shared memory in GBytes allowed for this zone cluster.
`max-shm-ids=value`	Specifies the maximum number of shared memory IDs allowed for this zone cluster.
`max-msg-ids=value`	Specifies the maximum number of message queue IDs allowed for this zone cluster.
`max-sem-ids=value`	Specifies the maximum number of semaphore IDs allowed for this zone cluster.
`cpu-shares=value`	Specifies the number of Fair Share Scheduler (FSS) shares to allocate to this zone cluster.

Choose the Zone CPU Resource Control property that you want to change.

You can set the following properties:

Property Description

Property	Description
`scope=scope-type`	Specifies whether the ncpus property used in a zone cluster is `dedicated-cpu` or `capped-cpu`.
`ncpus=value`	Specifies the limit for the scope type. If the `scope` property is set to `dedicated-cpu`, the `ncpus` property sets a limit on the number of CPUs that should be assigned for this zone's exclusive use. The zone will create a pool and processor set when it boots. See the `pooladm`(8) and `poolcfg`(8) man pages for more information on resource pools. If the `scope` property is set to `capped-cpu`, the `ncpus` property sets a limit on the amount of CPU time that can be used by a zone cluster. The unit used translates to the percentage of a single CPU that can be used by all user threads in a zone, expressed as a fraction (for example, .75) or a mixed number (whole number and fraction, for example, 1.25). An ncpus value of 1 means 100% of a CPU. See the `pooladm`(8) and `poolcfg`(8) man pages for more information on resource pools.

scope=scope-type

Specifies whether the ncpus property used in a zone cluster is dedicated-cpu or capped-cpu.

ncpus=value

Specifies the limit for the scope type.

If the scope property is set to dedicated-cpu, the ncpus property sets a limit on the number of CPUs that should be assigned for this zone's exclusive use. The zone will create a pool and processor set when it boots. See the pooladm(8) and poolcfg(8) man pages for more information on resource pools.
If the scope property is set to capped-cpu, the ncpus property sets a limit on the amount of CPU time that can be used by a zone cluster. The unit used translates to the percentage of a single CPU that can be used by all user threads in a zone, expressed as a fraction (for example, .75) or a mixed number (whole number and fraction, for example, 1.25). An ncpus value of 1 means 100% of a CPU. See the pooladm(8) and poolcfg(8) man pages for more information on resource pools.

Choose the capped-memory property that you want to change.

You can set the following properties:

Property	Description
`physical=value`	Specifies the GByte limit for physical memory.
`swap=value`	Specifies the GByte limit for swap memory.
`locked=value`	Specifies the GByte limit for locked memory.

You can also use Oracle Solaris Cluster Manager to view the capped-cpu memory configuration of a zone cluster, as well as the dedicated-CPU configuration. For the browser interface log-in instructions, see How to Access Oracle Solaris Cluster Manager in Administering an Oracle Solaris Cluster 4.4 Configuration.

Choose a physical host from the list of available physical hosts.

You can select one or all of the available physical nodes (or hosts), and then configure one zone-cluster node at a time.

You can set the following properties:

Property	Description
`hostname=hostname`	Specifies the zone-cluster node hostname. For example, `zc-host-1`.
`address=public-network-address`	Specifies the public network address for the zone-cluster node on a shared-IP type zone cluster. For example, `192.0.2.1`.
`physical=physical-interface`	Specifies a network physical interface for the public network from the available network interfaces that are discovered on the physical nodes. For example, `sc_ipmp0` or `net0`.
`defrouter=default-router`	Specifies the default router for the network address, if your zone is configured in a different subnet. Each zone or set of zones that uses a different `defrouter` setting must be on a different subnet, for example, `192.168.0.1`. See the `zonecfg`(8) man page for more information about the `defrouter` property.
`allowed-address=ipaddresses`	Specifies the IP addresses the exclusive-IP zone can use for the physical address. If not specified, then the exclusive-IP zone can use any IP address on the associated physical interface for the net resource. See the `zonecfg`(8) man page for more information about the `allowed-address` property.
`configure-allowed-address=true\|false`	If `configure-allowed-address` is set to true, the addresses specified by `allowed-address` are automatically configured on the interface each time the zone boots. When it is set to false, the `allowed-address` will not be configured on zone boot. By default, `configure-allowed-address` is set to true when an `allowed-address` is specified. When global net resources are present, this must be set to false. See the `zonecfg`(8) man page for more information about the `configure-allowed-address` property.

Specify the network addresses for the zone cluster.

The network addresses can be used to configure a logical hostname or shared IP cluster resources in the zone cluster. The network address is in the zone cluster global scope.

At the Review Configuration screen, press Return to continue and then type c to create the zone cluster.

The results of your configuration change are displayed, similar to the following:

 >>> Result of the Creation for the Zone Cluster(sczone) <<<

The zone cluster is being created with the following configuration

/usr/cluster/bin/clzonecluster configure sczone
create
set brand=solaris
set zonepath=/zones/sczone
set ip-type=shared
set enable_priv_net=true
add capped-memory
set physical=2G
end
add node
set physical-host=phys-schost-1
set hostname=zc-host-1
add net
set address=192.0.2.1
set physical=net0
end
end
add net
set address=192.0.2.2
end

Zone cluster, zc2 has been created and configured successfully.

Continue to install the zone cluster(yes/no) ?

Type yes to continue.

The clsetup utility performs a standard configuration of a zone cluster and you cannot specify any options.
When finished, exit the clsetup utility.

Verify the zone cluster configuration.

The verify subcommand checks for the availability of the specified resources. If the clzonecluster verify command succeeds, no output is displayed.

phys-schost-1# clzonecluster verify zone-cluster-name
phys-schost-1# clzonecluster status zone-cluster-name
=== Zone Clusters ===

--- Zone Cluster Status ---

Name      Node Name   Zone HostName   Status    Zone Status
----      ---------   -------------   ------    -----------
zone
basenode1
zone-1           Offline   Configured
basenode2
zone-2           Offline   Configured

For Trusted Extensions, make the password files writable on each zone-cluster node.
From the global zone, launch the txzonemgr BUI.
```
phys-schost# txzonemgr
```
Select the global zone, then select the item, Configure per-zone name service.

If you typed No in Step 14, then install the zone cluster.

phys-schost-1# clzonecluster install options zone-cluster-name
Waiting for zone install commands to complete on all the nodes
of the zone cluster "zone-cluster-name"...

For a solaris or labeled brand zone cluster, the following options are valid.

Option Description

Option	Description
`-c config-profile.xml`	Includes system configuration information. The `-c config-profile.xml` option provides a configuration profile for all non-global zones of the zone cluster. All profiles must have a `.xml` extension. The contents of the file is derived from the `profile.xml` created by the `sysconfig`(8) `create-profile` subcommand. See the Example section of the `clzonecluster`(8CL) man page for an example of the profile contents.
`-M manifest.xml`	Specifies a custom Automated Installer manifest that you configure to install the necessary packages on all zone-cluster nodes. Use this option if the base global-cluster nodes for the zone-cluster are not all installed with the same Oracle Solaris Cluster packages but you do not want to change which packages are on the base nodes. If the `clzonecluster install` command is run without the `-M` option, zone-cluster installation fails on a base node if it is missing a package that is installed on the issuing base node.

-c config-profile.xml

Includes system configuration information. The -c config-profile.xml option provides a configuration profile for all non-global zones of the zone cluster. All profiles must have a .xml extension. The contents of the file is derived from the profile.xml created by the sysconfig(8) create-profile subcommand. See the Example section of the clzonecluster(8CL) man page for an example of the profile contents.

-M manifest.xml

Specifies a custom Automated Installer manifest that you configure to install the necessary packages on all zone-cluster nodes. Use this option if the base global-cluster nodes for the zone-cluster are not all installed with the same Oracle Solaris Cluster packages but you do not want to change which packages are on the base nodes. If the clzonecluster install command is run without the -M option, zone-cluster installation fails on a base node if it is missing a package that is installed on the issuing base node.

For a solaris10 brand zone cluster, the following options are valid when using the clzonecluster install and the clzonecluster install-cluster commands.

When using the clzonecluster install command, use either the -a option or the -d option to install the solaris10 image.

When using the clzonecluster install-cluster command, you can use the -d, -s, and -p options in the same command, to install cluster core packages, Geographic Edition software, and agents that are supported in the zone cluster, as well as patches.

Note:

For a list of agents that are currently supported in a solaris10 brand zone cluster, see Oracle Solaris Cluster 4 Compatibility Guide (http://www.oracle.com/technetwork/server-storage/solaris-cluster/overview/solariscluster4-compatibilityguide-1429037.pdf).

Option	Description
-a absolute_path_to_archive	Specifies the absolute path to a solaris10 system archive to be used as the source image. The archive has to be accessible from all the nodes where the zone cluster is configured. # clzonecluster install [-n nodename[,…]] \ -a absolute_path_to_archive \ zone-cluster-name
-d absolute_directory_path	Specifies the full directory path to the root directory of an installed solaris10 non-global zone. The path should be accessible on all the physical nodes of the cluster where the zone cluster will be installed. # clzonecluster install \ [-n nodename[,…]] \ -d absolute_directory_path zone-cluster-name
-d dvd-image-directory zone-cluster-name -p patchdir=patchdir[,patchlistfile=patchlistfile] -s {all \| software-component	Note: Oracle Solaris Cluster patch 145333-15 for SPARC and 145334–15 for x86 patches are only required when you are installing the zone cluster with either the Oracle Solaris Cluster 3.3 software or the Oracle Solaris Cluster 3.3 5/11 software. You must install a minimum of Oracle Solaris Cluster 3.3 patch 145333–15 for SPARC or 145334–15 for x86 before you install the `solaris10` brand zone cluster. Log in to My Oracle Support to retrieve the patch. Then from the global zone, use the `-p` option to install the patch. The -d option specifies the full path to a DVD image directory for an Oracle Solaris Cluster release that supports the solaris10 brand zones. The cluster software DVD directory must be accessible from the global zone of the node where you run the command. In the -p option, patchdir specifies the directory of Oracle Solaris Cluster patches, and patchlistfile is a file that contains the list of patches in the patchdir directory to install. The patchdir directory is required, and must be accessible from inside the solaris10 brand zone on all nodes of the zone cluster. For additional instructions on installing patches, log in to My Oracle Support (https://support.oracle.com) and search for ID 1278636.1, How to Find and Download any Revision of a Solaris Patch. The -s option specifies the cluster software components that include the geo edition and data services, in addition to the core packages. # clzonecluster install-cluster \ -d dvd-image-directory \ [-p patchdir=patchdir[,patchlistfile=filename] \ [-s all] \ [-n phys-schost-1[,…]] \ [-v] \ zone-cluster-name

For more information, see the clzonecluster(8CL) man page.

If in Step 18, you did not use the -c config-profile.xml option when you installed the zone cluster, perform sysid configuration.

If in Step 18, you did use the -c config-profile.xml option when you installed the zone cluster, you do not need to perform sysid configuration. Proceed to Step 21.

Note:
In the following steps, the non-global zone zcnode and zone-cluster-name share the same name.
- For an exclusive-IP labeled brand zone cluster, perform the following steps.
  
  Configure only one zone-cluster node at a time.
  1. Boot the non-global zone of one zone-cluster node.
```
phys-schost# zoneadm -z zcnode boot
```
  2. Unconfigure the Oracle Solaris instance and reboot the zone.
```
phys-schost# zlogin zcnode
zcnode# sysconfig unconfigure
zcnode# reboot
```
    The zlogin session terminates during the reboot.
  3. Issue the zlogin command and progress through the interactive screens.
```
phys-schost# zlogin -C zcnode
```
  4. When finished, exit the zone console.
    
    For information about methods to exit from a non-global zone, see How to Exit a Non-Global Zone in Creating and Using Oracle Solaris Zones.
  5. From the global zone, halt the zone-cluster node.
```
phys-schost# zoneadm -z zcnode halt
```
  6. Repeat the preceding steps for each remaining zone-cluster node.
- For a shared-IP labeled brand zone cluster, perform the following steps on each zone-cluster node.
  1. From one global-cluster node, boot the zone cluster.
```
phys-schost# clzonecluster boot zone-cluster-name
```
  2. Unconfigure the Oracle Solaris instance and reboot the zone.
```
phys-schost# zlogin zcnode
zcnode# sysconfig unconfigure
zcnode# reboot
```
    The zlogin session terminates during the reboot.
  3. Issue the zlogin command and progress through the interactive screens.
```
phys-schost# zlogin -C zcnode
```
  4. When finished, exit the zone console.
    
    For information about methods to exit from a non-global zone, see How to Exit a Non-Global Zone in Creating and Using Oracle Solaris Zones.
  5. Repeat Step b through Step d for each remaining zone-cluster node.
- For a solaris or solaris10 brand zone cluster, perform the following steps on each zone-cluster node.
  1. From one global-cluster node, boot the zone cluster.
```
phys-schost# clzonecluster boot zone-cluster-name
```
  2. Issue the zlogin command and progress through the interactive screens.
```
phys-schost# zlogin -C zcnode
```
  3. When finished, exit the zone console.
    
    For information about methods to exit from a non-global zone, see How to Exit a Non-Global Zone in Creating and Using Oracle Solaris Zones.
  4. Repeat Step b through Step c for each remaining zone-cluster node.
Boot the zone cluster.
Installation of the zone cluster might take several minutes.
```
phys-schost# clzonecluster boot zone-cluster-name
```
(Exclusive-IP zone clusters) Manually configure an IPMP group.
The clsetup utility does not automatically configure IPMP groups for exclusive-IP zone clusters. You must create an IPMP group manually before you create a logical-hostname or shared-address resource, and add the underlying public network interface to the IPMP group. Since the underlying interface might have addresses associated with it, you must move the associated addresses to the IPMP group.

In each of the nodes of the zone cluster, configure the IPMP group and add an underlying public network interface to it. Delete any address that is already associated with the underlying interface as shown in the output of the ipadm show-addr command, and create it back on the IPMP interface.
```
zcnode# ipadm create-ipmp -i interface sc_ipmp0
zcnode# ipadm show-addr interface
zcnode# ipadm delete-addr interface/name
zcnode# ipadm create-addr -T static -a IPaddress/prefix sc_ipmp0/name
```
Note:
If the zone cluster's public networking interface is created over a global zone link aggregation or a global zone VNIC that is directly backed by a link aggregation, you do not need to create IPMP groups over it.

Next Steps

To configure Oracle Solaris Cluster 3.3 data services that you installed in a solaris10 brand zone cluster, follow procedures for zone clusters in the applicable data-service guide. See Oracle Solaris Cluster 3.3 Documentation.

To complete Trusted Extensions configuration, go to How to Configure a Zone Cluster to Use Trusted Extensions.

Otherwise, add file systems or storage devices to the zone cluster. See the following sections: