This chapter provides the procedures for administering partnerships between two Sun Cluster Geographic Edition software-enabled clusters.
This chapter contains the following sections:
Before you create a partnership between two clusters, you must configure the Sun Cluster Geographic Edition software for secure communication between the two clusters. The configuration must be reciprocal. For example, you must configure the cluster cluster-paris to trust the cluster cluster-newyork, and you must also configure the cluster cluster-newyork to trust the cluster cluster-paris.
Ensure that the following conditions are met:
The cluster on which you want to create the partnership is running.
The geoadm start command must have already been run on this cluster and the partner cluster. For more information about using the geoadm start command, see Enabling the Sun Cluster Geographic Edition Software.
The cluster name of the partner cluster is known.
The host information of the partner cluster must defined in the local host file. The local cluster needs to know how to reach the partner cluster by name.
If the clusters are in different domains, include the domain name in the entry, as logicalhostname.domainname. However, the cluster name itself must not include the domain.
Log in to a cluster node.
You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Sun Cluster Geographic Edition Software and RBAC.
Import the public keys from the remote cluster to the local cluster.
Running this command on one node of the local cluster imports the keys from the remote cluster to one node of the cluster.
# geops add-trust -c remotepartnerclustername |
Specifies the logical hostname of the cluster with which to form a partnership. The logical hostname is used by the Sun Cluster Geographic Edition software and maps to the name of the remote partner cluster. For example, a remote partner cluster name might resemble the following:
cluster-paris
If the clusters are on different domains, also specify the fully qualified domain name. For example, two clusters in a partnership that have different domains might resemble the following:
cluster-paris.france cluster-newyork.usa |
When you use this option with the add-trust or remote-trust subcommand, the option specifies the alias where the public keys on the remote cluster are stored. An alias for certificates on the remote cluster has the following pattern:
remotepartnercluster.certificate[0-9]*
Keys and only keys that belong to the remote cluster should have their alias match this pattern.
For more information about the geops command, refer to the geops(1M) man page.
Repeat the preceding steps on a node of the remote partner cluster.
If you choose to use the Sun Cluster Manager, skip this step. The Sun Cluster Manager handles all nodes in a single operation.
Verify trust from one node of each cluster.
# geops verify-trust -c remotepartnerclustername[.domainname] |
This command verifies the trust from the node on which you run the command to all nodes of the partner cluster. If you choose to use the Sun Cluster Manager, it verifies the trust from all nodes of the local cluster to all nodes of the partner cluster
For a complete example of how to configure and join a partnership, see Example 5–4.
Ensure that the following conditions are met:
The cluster on which you want to remove trust is running.
The cluster name of the partner cluster is known.
The host information of the partner cluster must defined in the local host file. The local cluster needs to know how to reach the partner cluster by name.
Log in to a cluster node.
You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Sun Cluster Geographic Edition Software and RBAC.
If there is a partnership configured between the two clusters, dissolve that partnership.
Run the following command on both clusters:
# geops leave |
On all nodes of both clusters, remove all keys for the remote cluster from the truststore file on the local node.
# geops remove-trust -c remotepartnerclustername |
Perform this step on all the nodes of the local cluster, and then repeat this step on all nodes of the partner cluster.
Specifies the logical hostname of the cluster from which you want to remove the keys. The name for the remote cluster must be identical to the cluster name you specified when adding trust with the geops add-trust command. You do not need to specify the fully qualified name if the remote cluster is reachable by partial name.
When you use this option with the add-trust or remote-trust subcommand, the option specifies the alias where the public keys on the remote cluster are stored. An alias for certificates on the remote cluster has the following pattern:
remotepartnercluster.certificate[0-9]*
Keys and only keys that belong to the remote cluster should have their alias match this pattern.
For more information about the geops command, refer to the geops(1M) man page.
If you choose to use the Sun Cluster Manager, it handles all nodes of a cluster in a single operation.
Repeat the preceding steps on a node of the remote partner cluster.
The Sun Cluster Geographic Edition software enables clusters to form partnerships between clusters to provide mutual protection against disasters. The clusters in a partnership monitor each other by sending heartbeat messages to each other in the same way that nodes of a single cluster do. Unlike local clusters, the clusters in a partnership use the public network for these messages, but support additional, plug-in mechanisms as well.
You create only one partnership between two specific clusters by using the geops(1M) command. After you have created a partnership, you can use this command to modify the properties of this partnership.
When creating partnerships, ensure that the name of all the clusters in the partnership are unique. For example, if you have a cluster wholly within the domain .france, you can use hostnames like paris and grenoble. However, if you have a cross-domain cluster, you must specify the hostnames with enough qualification to identify the host on the network. You can link paris and munich with hostnames paris.france and munich.germany, and the cluster names remain paris and munich.
You cannot create a partnership between clusters paris.france and paris.texas because of a collision on the cluster name paris.
The names of the application resource groups that are managed by the Sun Cluster Geographic Edition software must be the same on both partner clusters. You can configure the names of these resource groups manually or by using the scsnapshot command.
The scsnapshot command replicates configuration data on a cluster that does not have configured resource groups, resource types, and resources. The scsnapshot command retrieves the configuration data from the cluster on which it is launched and generates a script called scriptfile. Edit the script to adapt it to the specific features of the cluster where you want to replicate the configuration data. For example, you might have to change the IP address and host names in the script. Launch the script from any node in the cluster where you want to replicate the configuration data. For more information about using this command, see the scsnapshot(1M) man page.
You can define only one partnership between two specific clusters. A single cluster can participate in other partnerships with different clusters.
Ensure that the following conditions are met:
The cluster on which you want to create the partnership is up and running.
The geoadm start command must have already been run on the this cluster and the partner cluster. For more information about using the geoadm start command, see Enabling the Sun Cluster Geographic Edition Software.
The cluster name of the partner cluster is known.
The host information of the partner cluster must defined in the local host file. The local cluster needs to know how to reach the partner cluster by name.
Security has been configured on the two clusters by installing the appropriate certificates.
See Configuring Trust Between Partner Clusters for more information.
Log in to a cluster node.
You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Sun Cluster Geographic Edition Software and RBAC.
Create the partnership.
# geops create -c remotepartnerclustername[.domainname] [-h heartbeatname] \ [-p propertysetting [-p…]] partnershipname |
Specifies the name of the remote cluster that will participate in the partnership. If clusters in the partnership are in different domains, you must also specify the domain name of the remote cluster.
This name matches the logical hostname used by the Sun Cluster Geographic Edition infrastructure on the remote cluster.
Specifies a custom heartbeat to use in the partnership to monitor the availability of the partner cluster.
If you omit this option, the default Sun Cluster Geographic Edition heartbeat is used.
Custom heartbeats are provided for special circumstances and require careful configuration. Consult your Sun specialist for assistance if your system requires the use of custom heartbeats. For more information about configuring custom heartbeats, see Chapter 6, Administering Heartbeats.
If you create a custom heartbeat, you must add at least one plug-in to prevent the partnership from remaining in degraded mode.
You must configure the custom heartbeat that you provide in this option before you run the geops command.
A custom heartbeat prevents the default heartbeat from being used during partnership creation. If you want to use the default heartbeat for your partnership, you must delete the custom heartbeat before you run the geops create command.
Specifies the value of partnership properties with a string of property=value pair statements.
Specify a description of the partnership with the Description property.
You can configure heartbeat-loss notification with the Notification_emailaddrs and Notification_actioncmd properties. For more information about configuring heartbeat-loss notification, see Configuring Heartbeat-Loss Notification.
For more information about the properties you can set, see Appendix A, Standard Sun Cluster Geographic Edition Properties.
Specifies the name of the partnership.
For information about the names and values that are supported by Sun Cluster Geographic Edition software, see Appendix B, Legal Names and Values of Sun Cluster Geographic Edition Entities.
For more information about the geops command, refer to the geops(1M) man page.
Verify that the partnership was created and the status of the partnership.
# geoadm status |
This example creates the paris-newyork-ps partnership on the cluster-paris.usa cluster.
# geops create -c cluster-newyork.usa -p Description=Transatlantic \ -p Notification_emailaddrs=sysadmin@companyX.com paris-newyork-ps # geoadm status |
For a complete example of how to configure and join a partnership, see Example 5–4.
Log in to a cluster node.
You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Sun Cluster Geographic Edition Software and RBAC.
Modify partnership properties.
# geops set-prop -p propertysetting [-p…] partnershipname |
Specifies the value of partnership properties with a string of property=value pair statements.
Specify a description of the partnership with the Description property.
You can configure heartbeat-loss notification with the Notification_emailaddrs and Notification_actioncmd properties. For more information about configuring heartbeat-loss notification, see Configuring Heartbeat-Loss Notification.
For more information about the properties you can set, see Appendix A, Standard Sun Cluster Geographic Edition Properties.
Specifies the name of the partnership.
For information about the names and values that are supported by Sun Cluster Geographic Edition software, see Appendix B, Legal Names and Values of Sun Cluster Geographic Edition Entities.
For more information about the geops command, refer to the geops(1M) man page.
Verify that your modification was made correctly.
# geops list |
This example modifies the notification email address for the cluster-paris cluster.
# geops set-prop -p Notification_emailaddrs=operations@companyX.com \ paris-newyork-ps # geops list |
When you define and configure a partnership, the partnership specifies a second cluster to be a member of that partnership. Then, you must configure this second cluster to join the partnership.
Ensure that the following conditions are met:
The local cluster is enabled to run the Sun Cluster Geographic Edition software.
The partnership you want the cluster to join is defined and configured on another cluster (cluster-paris) and the local cluster (cluster-newyork) is specified as a member of this partnership.
Security has been configured on the clusters by installing the appropriate certificates.
See Configuring Secure Cluster Communication Using Security Certificates for more information.
Log in to a node of the cluster that is joining the partnership.
You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Sun Cluster Geographic Edition Software and RBAC.
Confirm that the remote cluster that originally created the partnership, cluster-paris, can be reached at its logical hostname.
# ping lh-paris-1 |
For information about the logical hostname of the cluster, see How to Enable Sun Cluster Geographic Edition Software.
Join the partnership.
# geops join-partnership [-h heartbeatname] remoteclustername partnershipname |
Specifies a custom heartbeat to use in the partnership to monitor the availability of the partner cluster.
If you omit this option, the default Sun Cluster Geographic Edition heartbeat is used.
Custom heartbeats are provided for special circumstances and require careful configuration. Consult your Sun specialist for assistance if your system requires the use of custom heartbeats. For more information about configuring custom heartbeats, see Chapter 6, Administering Heartbeats.
If you create a custom heartbeat, you must add at least one plug-in to prevent the partnership from remaining in degraded mode.
You must configure the custom heartbeat that you provide in this option before you run the geops command.
Specifies the name of a cluster that is currently a member of the partnership that is being joined. This cluster is used to retrieve the partnership configuration information.
Specifies the name of the partnership.
For information about the names and values that are supported by Sun Cluster Geographic Edition software, see Appendix B, Legal Names and Values of Sun Cluster Geographic Edition Entities.
For more information about the geops command, refer to the geops(1M) man page.
Verify that the cluster was added to the partnership and that the partnership properties were defined correctly.
# geops list # geoadm status |
This example joins the cluster-newyork cluster in the partnership that was created on cluster-paris in Example 5–1.
# geops join-partnership cluster-paris paris-newyork-ps # geops list # geoadm status |
This example creates and configures the paris-newyork-ps partnership between clusters cluster-paris.france and cluster-newyork.usa.
On one node of cluster-paris.france, configure trust for the partnership.
phys-paris-1# geops add-trust -c cluster-newyork.usa |
On one node of cluster-newyork.usa, configure trust for the partnership.
phys-newyork-1# geops add-trust -c cluster-paris.france |
On each node of both clusters, verify that trust has been set up properly, both between the local cluster and partner cluster and among nodes of the local cluster.
phys-newyork-1# geops verify-trust -c cluster-paris.france phys-newyork-2# geops verify-trust -c cluster-paris.france phys-newyork-1# geops verify-trust phys-newyork-2# geops verify-trust phys-paris-1# geops verify-trust -c cluster-newyork.usa phys-paris-2# geops verify-trust -c cluster-newyork.usa phys-paris-1# geops verify-trust phys-paris-2# geops verify-trust |
On cluster-paris.france, create the partnership paris-newyork-ps.
cluster-paris# geops create -c cluster-newyork.usa -p Description=Transatlantic \ -p Notification_emailaddrs=sysadmin@companyX.com paris-newyork-ps |
On cluster-newyork.usa, join the partnership paris-newyork-ps.
cluster-newyork# geops join-partnership cluster-paris.france paris-newyork-ps |
Verify that the partnership has been created successfully.
# geops list # geoadm status |
When you add a new node to a cluster that is in a partnership, you must perform additional tasks on that node to make it an active participant in the Sun Cluster Geographic Edition configuration.
Perform all steps from the new node.
Add the new node to the cluster.
Follow procedures in Chapter 8, Adding and Removing a Node, in Sun Cluster System Administration Guide for Solaris OS.
Install Sun Cluster Geographic Edition, data replication, and application software on the new node.
To install Sun Cluster Geographic Edition software, see Sun Cluster Geographic Edition Installation Guide.
To install data replication and application software, see the appropriate manual for the software that you use.
If the cluster with the new node is the primary for any activated protection groups, remove application resource groups from those protection groups.
This step is necessary to avoid application downtime.
# geopg remove-resource-group resourcegroup protectiongroup |
Deactivate all protection groups that are active on this cluster locally.
# geopg stop -e local protectiongroup |
Stop the Sun Cluster Geographic Edition infrastructure.
# geoadm stop |
Re-enable the Sun Cluster Geographic Edition infrastructure.
This action recreates each Sun Cluster Geographic Edition resource group and adds all nodes in the cluster, including the new node, to the node list.
# geoadm start |
Reactivate the protection groups that you deactivated in Step 4.
# geopg start -e local protectiongroup |
Restore any application resource groups that you removed in Step 3.
# geopg add-resource-group resourcegroup protectiongroup |
When you rename a cluster that is in a partnership, the partnership becomes invalid. You must fully unconfigure the existing partnership and create a new one that uses the new cluster name.
This procedure demonstrates how to rename one of the clusters that is in a partnership. You can rename more than one of the clusters at the same time.
If the cluster that you rename belongs to more than one partnership, perform each step on all clusters that share a partnership with the cluster to rename, before you proceed to the next step in the procedure.
From one node of the cluster that you are renaming, remove resource groups from each protection group that the cluster belongs to.
This task avoids production application downtime.
# geopg remove-resource-group app-rg pg1 |
From one node of each cluster in a protection group, confirm that application resource groups have been removed.
# geopg list pg1 |
From one node of the cluster that you are renaming, stop each protection group globally.
This task stops data replication.
# geopg stop pg1 -e global |
From one node of each cluster in a protection group, delete the protection group
# geopg delete pg1 |
From one node of each cluster in a partnership, leave the partnership.
# geops leave-partnership ps1 |
From one node of each cluster, confirm that the protection group and the partnership have been removed.
# geoadm status |
From one node of each cluster, disable Sun Cluster Geographic Edition software.
# geoadm stop |
From one node of each cluster, confirm that Sun Cluster Geographic Edition software was disabled.
Verify that the geo-infrastructure, geo-clusterstate, and data-replication resource groups are deleted.
# clrg list # geoadm status |
From one node of the cluster that you are renaming, change the cluster name.
Follow cluster naming guidelines as described in Planning Required IP Addresses and Hostnames in Sun Cluster Geographic Edition Installation Guide.
# cluster rename -c new-clustername |
The name of the cluster must not include the domain. If a partnership contains clusters that are in different domains, you specify the domain to administrative commands, when necessary, by appending the domain name to the cluster name as cluster.domain. Only certain Sun Cluster Geographic Edition administrative commands require this fully qualified name when clusters in a partnership are not in the same domain.
Confirm that the cluster name is changed.
# cluster list |
On each node of both clusters, ensure that hostname entries that match the new cluster name are free and are added to the local /etc/hosts files.
If clusters in the partnership are in different domains, include the domain in the /etc/hosts entry for each cluster.
# ping new-clustername there should be no response # echo "IPaddress new-clustername" >> /etc/hosts |
From one node of each cluster, start Sun Cluster Geographic Edition software.
# geoadm start |
If Sun Cluster Geographic Edition software fails to start, and the failure is not due to problems with the new logical host, restart the common agent container on all nodes by using the cacaoadm restart command, then start Sun Cluster Geographic Edition software.
From one node of each cluster, verify that Sun Cluster Geographic Edition software is successfully started.
# geoadm status |
From one node of each cluster, add trust between the clusters.
# geops add-trust -c remotepartnerclustername[.domainname] |
From one node of each cluster, confirm that trust is added successfully.
Do not specify a domain name to the verify-trust subcommand.
# geops verify-trust -c remotepartnerclustername |
Create and join a new partnership between the clusters.
On each cluster, confirm that the new partnership is successfully created and joined.
# geoadm status |
If you did not reboot the nodes of the cluster that you renamed, restart the heartbeats on each node of the renamed cluster.
Restarting the heartbeat initiates the heartbeat to read and store the new cluster name.
# /etc/init.d/initgchb_resd stop # /etc/init.d/initgchb_resd start |
This example renames the cluster newyork, in the paris-newyork-ps partnership, to chicago. The names of the nodes in this cluster are not changed, so phys-newyork-1 becomes a node in the newly named chicago cluster. The paris-newyork-ps partnership is first unconfigured. After the cluster is renamed, a new paris-chicago-ps partnership is created with the chicago cluster as primary and the paris cluster as secondary. The two clusters belong to the same domain, so the domain name is not specified to the commands.
phys-newyork-1# geopg remove-resource-group app-rg phys-newyork-1# geopg list examplepg phys-paris-1# geopg list examplepg phys-newyork-1# geopg stop examplepg -e global phys-newyork-1# geopg delete examplepg phys-paris-1# geopg delete examplepg phys-newyork-1# geops leave-partnership paris-newyork-ps phys-paris-1# geops leave-partnership paris-newyork-ps phys-newyork-1# geoadm stop phys-paris-1# geoadm stop phys-newyork-1# clrg list phys-newyork-1# geoadm status phys-paris-1# clrg list phys-paris-1# geoadm status phys-newyork-1# cluster rename -c chicago phys-newyork-1# cluster list phys-newyork-1# ping chicago phys-newyork-1# echo "192.168.10.1 chicago" >> /etc/hosts repeat on each node of the chicago cluster phys-paris-1# ping chicago phys-paris-1# echo "192.168.20.1 chicago" >> /etc/hosts repeat on each node of the paris cluster phys-newyork-1# geoadm start phys-paris-1# geoadm start phys-newyork-1# geoadm status phys-paris-1# geoadm status phys-newyork-1# geops add-trust -c paris phys-paris-1# geops add-trust -c chicago phys-newyork-1# geops verify-trust -c paris phys-paris-1# geops verify-trust -c chicago phys-newyork-1# geops create -c paris paris-chicago-ps phys-paris-1# geops join-partnership chicago paris-chicago-ps phys-newyork-1# geoadm status phys-paris-1# geoadm status phys-newyork-1# /etc/init.d/initgchb_resd stop phys-newyork-1# /etc/init.d/initgchb_resd start repeat on each node of the chicago cluster phys-paris-1# /etc/init.d/initgchb_resd stop phys-paris-1# /etc/init.d/initgchb_resd start repeat on each node of the paris cluster |
Perform the following tasks:
Create a new protection group and replicate it to partner.
Add device groups.
Start globally.
Add resource groups to the protection group and verify the configuration.
NOTE - When you create the new protection group, pay close attention to which cluster is the primary and which is the secondary, to ensure that data replication is started in the desired direction.
Follow procedures in the appropriate data-replication guide:
You can also use the geops command to remove a cluster from a partnership and release all the resources that are associated with the partnership.
Because this command destroys the local partnership configuration information, when the last member leaves a partnership, the partnership no longer exists.
Ensure that the following conditions are met:
The local cluster is a member of the partnership you want to leave.
This partnership does not contain any protection groups.
Log in to a cluster node.
You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Sun Cluster Geographic Edition Software and RBAC.
Verify that the partnership does not have any protection groups.
# geopg list |
If you find that the partnership contains protection groups, you can delete them with the geopg delete command. For information about deleting protection groups, see one of the following data replication guides:
Remove the partnership on a node of the cluster that is a member of the partnership.
# geops leave-partnership partnershipname |
Specifies the name of the partnership
The geops leave-partnership command deletes the heartbeats configured for the partnership, including custom heartbeats.
For more information, refer to the geops(1M) man page.
In this example, the cluster-paris cluster leaves the paris-newyork-ps partnership.
phys-paris-1# geops leave-partnership paris-newyork-ps |
After the cluster-paris cluster leaves the paris-newyork-ps partnership, as described in the previous example, the only remaining member of the partnership is the cluster-newyork cluster. You can delete the paris-newyork-ps partnership by forcing the cluster-newyork cluster to leave the partnership.
phys-newyork-1# geops leave-partnership paris-newyork-ps |
Repeat this procedure on the other cluster in the partnership.
Partner clusters that become disconnected during a disaster situation might force the administrator to perform a takeover for a protection group that the partners share. When both clusters are brought online again, both partner clusters might report as the primary of the protection group. You must resynchronize the configuration information of the local protection group with the configuration information that is retrieved from the partner cluster.
If a cluster that is a member of a partnership fails, when the cluster restarts, it detects whether the partnership parameters have been modified while it was down. You decide which partnership configuration information you want to keep: the information on the cluster that failed or the information on the failover cluster. Then, resynchronize the configuration of the partnership accordingly.
You do not need to resynchronize the configuration information in the following situations if the original secondary cluster goes down and resumes operation later.
Use the geoadm status command to check whether you need to resynchronize a partnership. If the Configuration status is Synchronization Status Error, you need to synchronize the partnership. If the Local status is Partnership Error, do not resynchronize the partnership. Instead, wait until a heartbeat exchange occurs.
Ensure that the following conditions are met:
The local cluster is Sun Cluster Geographic Edition enabled.
The local cluster was an active member of the partnership before failing.
Resynchronizing a partnership overwrites the partnership configuration on the cluster where the command is run with the information from the partner cluster.
Log in to a node on the cluster that needs to be synchronized with the information retrieved from the partner cluster.
You must be assigned the Geo Management RBAC rights profile to complete this procedure. For more information about RBAC, see Sun Cluster Geographic Edition Software and RBAC.
Resynchronize the partnership.
# geops update partnershipname |
Specifies the name of the partnership
This example resynchronizes a partnership.
# geops update paris-newyork-ps |