The focus of this procedure is the illustration of steps required for ZFS-based cluster filesystems in zone clusters replicated with ZFS snapshot replication. As such, it does not show setting up replication users, nor SSH and other steps, such as preparing a zone cluster for the disaster recovery framework. Such procedures are found else where in this guide and other disaster recovery framework guides. For information about the step for creating the replication user and SSH setup, see Use Case: Setting Up Replication User and SSH
The example uses the following names:
Global cluster paris with nodes phys-paris-1 and phys-paris-2
Zone cluster paris-zc paris on with nodes paris-zc-1 and paris-zc-2
Global cluster newyork with nodes phys-newyork-1 and phys-newyork-2
Zone cluster newyork-zc on newyork with nodes newyork-zc-1 and newyork-zc-2
On paris cluster the zpool is named dataz4 with dataz4/fs1 being replicated
On newyork cluster the zpool is named dataz5 with dataz5/fs1 being replicated
paris-rep-lh is replication logical host on paris cluster
newyork-rep-lh is replication logical host on newyork cluster
This example contains the procedure required for this configuration. It shows scalable Apache for illustration purposes only.
From one node of paris global cluster create zpool, filesystem and configure it in HAStoragePlus resource setting globalzpools property.
root@phys-paris-1:~# zpool create dataz4 c0t600144F08010407600005BAEA4BC0004d0 root@phys-paris-1:~# zfs create dataz4/fs1 root@phys-paris-1:~# clrg create -S -p auto_start_on_new_cluster=false gzfsgc-rg root@phys-paris-1:~# cldg list dataz4 cldg: (C531091) Device group "dataz4" does not exist. root@phys-paris-1:~# zpool export dataz4 root@phys-paris-1:~# clrs create -g gzfsgc-rg -t HAStoragePlus -p \ globalzpools=dataz4 hasp4dataz4 root@phys-paris-1:~# cldg status dataz4 === Cluster Device Groups === --- Device Group Status --- Device Group Name Primary Secondary Status ----------------- ------- --------- ------ dataz4 - - Offline root@phys-paris-1:~# cldg show dataz4 === Device Groups === Device Group Name: dataz4 Type: ZPOOL failback: false Node List: phys-paris-2, phys-paris-1 preferenced: false autogen: true numsecondaries: 1 ZFS pool name: dataz4 poolaccess: global readonly: false import-at-boot: false searchpaths: /dev/dsk root@phys-paris-1:~# clrg online -M gzfsgc-rg root@phys-paris-1:~# cldg status dataz4 === Cluster Device Groups === --- Device Group Status --- Device Group Name Primary Secondary Status ----------------- ------- --------- ------ dataz4 phys-paris-2 phys-paris-1 Online root@phys-paris-1:~# df -h | grep dataz4 dataz4 14G 46K 14G 1% /dataz4 dataz4/fs1 14G 40K 14G 1% /dataz4/fs1 root@phys-paris-1:~# zpool list dataz4 cannot open 'dataz4': no such pool root@phys-paris-1:~#
The zpool is imported on one node of the global cluster but the filesystem is mounted on both global cluster nodes.
root@phys-paris-2:~# df -h | grep dataz4 dataz4 20G 46K 14G 1% /dataz4 dataz4/fs1 20G 40K 14G 1% /dataz4/fs1 root@phys-paris-2:~# zpool list dataz4 NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT dataz4 19.9G 158K 19.9G 0% 1.00x ONLINE / root@phys-paris-2:~#
Configure the ZFS cluster file system in the zone cluster and create resource group and HAStoragePlus resource for it setting rg_affinities and resource_dependencies_offline_restart properties.
root@phys-paris-1:~# clzc configure paris-zc clzc:paris-zc> add fs clzc:paris-zc:fs> set dir=/global/apache clzc:paris-zc:fs> set special=/dataz4/fs1 clzc:paris-zc:fs> set type=lofs clzc:paris-zc:fs> end clzc:paris-zc> exit root@phys-paris-1:~# clrg create -Z paris-zc -S -p rg_affinities=++global:gzfsgc-rg \ -p auto_start_on_new_cluster=false apache-rg root@phys-paris-1:~# clrs create -Z paris-zc -t HAStoragePlus -p \ filesystemmountpoints=/global/apache -p \ resource_dependencies_offline_restart=global:hasp4dataz4 -g apache-rg hasp4apache root@phys-paris-1:~# clrg online -M -Z paris-zc apache-rg root@phys-paris-1:~# zlogin paris-zc [Connected to zone 'paris-zc' pts/3] Oracle Corporation SunOS 5.11 11.4 September 2018 You have new mail. root@paris-zc-1:~# df -h | grep apache /global/apache 20G 40K 20G 1% /global/apache root@paris-zc-1:~#
The filesystem is mounted on both zone cluster nodes.
root@phys-paris-2:~# zlogin paris-zc [Connected to zone 'paris-zc' pts/4] Oracle Corporation SunOS 5.11 11.4 September 2018 You have new mail. root@paris-zc-2:~# df -h | grep apache /global/apache 20G 40K 20G 1% /global/apache root@paris-zc-2:~#
From one node of newyork global cluster create zpool, filesystem and configure it in HAStoragePlus resource setting globalzpools property.
root@phys-newyork-1:~# zpool create dataz5 c0t600144F0CD152C9E00005BAE9BB90009d0 root@phys-newyork-1:~# zfs create dataz5/fs1 root@phys-newyork-1:~# clrg create -S -p auto_start_on_new_cluster=false gzfsgc-rg root@phys-newyork-1:~# cldg list dataz5 cldg: (C531091) Device group "dataz5" does not exist. root@phys-newyork-1:~# zpool export dataz5 root@phys-newyork-1:~# clrs create -g gzfsgc-rg -t HAStoragePlus -p \ globalzpools=dataz5 hasp4dataz5 root@phys-newyork-1:~# cldg status dataz5 === Cluster Device Groups === --- Device Group Status --- Device Group Name Primary Secondary Status ----------------- ------- --------- ------ dataz5 - - Offline root@phys-newyork-1:~# cldg show dataz5 === Device Groups === Device Group Name: dataz5 Type: ZPOOL failback: false Node List: phys-newyork-2, phys-newyork-1 preferenced: false autogen: true numsecondaries: 1 ZFS pool name: dataz5 poolaccess: global readonly: false import-at-boot: false searchpaths: /dev/dsk root@phys-newyork-1:~# clrg online -M gzfsgc-rg root@phys-newyork-1:~# cldg status dataz5 === Cluster Device Groups === --- Device Group Status --- Device Group Name Primary Secondary Status ----------------- ------- --------- ------ dataz5 phys-newyork-2 phys-newyork-1 Online root@phys-newyork-1:~# df -h | grep dataz5 dataz5 20G 42K 20G 1% /dataz5 dataz5/fs1 20G 40K 20G 1% /dataz5/fs1 root@phys-newyork-1:~# root@phys-newyork-1:~# zpool list dataz5 cannot open 'dataz5': no such pool root@phys-newyork-1:~# The zpool is imported on one node of the global cluster but the filesystem is mounted on both global cluster nodes. root@phys-newyork-2:~# df -h | grep dataz5 dataz5 20G 42K 20G 1% /dataz5 dataz5/fs1 20G 40K 20G 1% /dataz5/fs1 root@phys-newyork-2:~# zpool list dataz5 NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT dataz5 19.9G 158K 19.9G 0% 1.00x ONLINE / root@phys-newyork-2:~#
Configure the ZFS cluster file system in the zone cluster and create resource group and HAStoragePlus resource for it setting rg_affinities and resource_dependencies_offline_restart properties.
root@phys-newyork-1:~# clzc configure newyork-zc clzc:newyork-zc> add fs clzc:newyork-zc:fs> set dir=/global/apache clzc:newyork-zc:fs> set special=/dataz5/fs1 clzc:newyork-zc:fs> set type=lofs clzc:newyork-zc:fs> end clzc:newyork-zc> exit root@phys-newyork-1:~# clrg create -Z newyork-zc -S -p \ rg_affinities=++global:gzfsgc-rg -p auto_start_on_new_cluster=false apache-rg root@phys-newyork-1:~# clrs create -Z newyork-zc -t HAStoragePlus:12 -p \ filesystemmountpoints=/global/apache -p \ resource_dependencies_offline_restart=global:hasp4dataz5 -g apache-rg \ hasp4apache root@phys-newyork-1:~# clrg online -M -Z newyork-zc apache-rg root@phys-newyork-1:~# zlogin newyork-zc [Connected to zone 'newyork-zc' pts/2] Oracle Corporation SunOS 5.11 st_015.server February 2018 You have new mail. root@newyork-zc-1:~# df -h | grep apache /global/apache 20G 40K 20G 1% /global/apache root@newyork-zc-1:~#
The filesystem is mounted on both zone cluster nodes.
root@phys-newyork-2:~# zlogin newyork-zc [Connected to zone 'newyork-zc' pts/4] Oracle Corporation SunOS 5.11 st_015.server February 2018 You have new mail. root@newyork-zc-2:~# df -h | grep apache /global/apache 20G 40K 20G 1% /global/apache root@newyork-zc-2:~#
Configure apache in paris-zc.
root@phys-paris-1:~# zlogin paris-zc
[Connected to zone 'paris-zc' pts/3]
Oracle Corporation SunOS 5.11 11.4 September 2018
You have new mail.
root@paris-zc-1:~# clrg create -p auto_start_on_new_cluster=false apachesip-rg
root@paris-zc-1:~# clrssa create -g apachesip-rg apache-lh
root@paris-zc-1:~# clrg set -p rg_dependencies=apachesip-rg apache-rg
root@paris-zc-1:~# cd /global/apache
root@paris-zc-1:/global/apache# df -k .
Filesystem 1024-blocks Used Available Capacity Mounted on
/global/apache 15105524 42 15105482 1% /global/apache
root@paris-zc-1:/global/apache# mkdir conf bin docs
root@paris-zc-1:/global/apache# cd /global/apache/conf
root@paris-zc-1:/global/apache/conf#
root@paris-zc-1:/global/apache/conf# cp /etc/apache2/2.4/httpd.conf .
<update httpd.conf>
root@paris-zc-1:/global/apache/conf# cd /global/apache/bin
root@paris-zc-1:/global/apache/bin# cp /usr/apache2/2.4/bin/apachectl .
root@paris-zc-1:/global/apache/bin#
<update apachectl>
root@paris-zc-1:/global/apache/bin# cd
root@paris-zc-1:~# clrg online -M apachesip-rg
root@paris-zc-1:~# clrt register apache
root@paris-zc-1:~# clrs create -g apache-rg -t apache -p bin_dir=/global/apache/bin \
-p resource_dependencies_offline_restart=hasp4apache -p \
resource_dependencies=apache-lh -p scalable=true apache
Oct 3 11:40:55 phys-paris-1 cl_runtime: NOTICE: Load balancer setting distribution on paris-zc:apache:
Oct 3 11:40:55 phys-paris-1 cl_runtime: NOTICE: Node paris-zc-1: weight 1
Oct 3 11:40:55 phys-paris-1 cl_runtime: NOTICE: Node paris-zc-2: weight 1
Oct 3 11:40:55 phys-paris-1 cl_runtime: NOTICE: Load balancer setting distribution on paris-zc:apache:
Oct 3 11:40:55 phys-paris-1 cl_runtime: NOTICE: Node paris-zc-1: weight 1
Oct 3 11:40:55 phys-paris-1 cl_runtime: NOTICE: Node paris-zc-2: weight 1
Oct 3 11:40:55 phys-paris-1 cl_runtime: NOTICE: Scalable service instance [TCP,10.134.28.184,80] registered on node paris-zc-1.
Oct 3 11:41:01 phys-paris-1 cl_runtime: NOTICE: Scalable service instance [TCP,10.134.28.184,80] registered on node paris-zc-2.
root@paris-zc-1:~# clrs status apache
=== Cluster Resources ===
Resource Name Node Name State Status Message
------------- --------- ----- --------------
apache paris-zc-2 Online Online - Completed successfully.
paris-zc-1 Online Online - Completed successfully.
root@paris-zc-1:~#
Set required permissions on the source and target filesystems.
On phys-paris-2, where dataz4 zpool is imported:
root@phys-paris-2:~# zfs allow zfsuser1 create,destroy,\ hold,mount,receive,release,rollback,send,snapshot dataz4/fs1 root@phys-paris-2:~#
On phys-newyork-2, where dataz5 zpool is imported:
root@phys-newyork-2:~# zfs allow zfsuser2 create,destroy,\ hold,mount,receive,release,rollback,send,snapshot dataz5/fs1 root@phys-newyork-2:~#
Update script-based plugins configuration file with the replication component for the ZFS snapshot protection group.
On all nodes of paris:
root@phys-paris-1:~# echo "gzfs-fs1|any|phys-paris-1,phys-paris-2" \ >> /var/tmp/ody/sbp_conf root@phys-paris-1:~#
On all nodes of newyork:
root@phys-newyork-1:~# echo "gzfs-fs1|any|phys-newyork-1,phys-newyork-2" \ >> /var/tmp/ody/sbp_conf root@phys-newyork-1:~#
From one node of paris create the ZFS snapshot protection group.
root@phys-paris-1:~# cat /var/tmp/ody/gzfsgcpg
PS=paris_newyork
PG=gzfspg
REPCOMP=gzfs-fs1
REPRS=gzfs-fs1-rep
REPRG=gzfs-fs1-rep-rg
DESC="GZFS with zfs snapshot replication"
APPRG=gzfsgc-rg
CONFIGFILE=/var/tmp/ody/sbp_conf
LOCAL_REP_USER=zfsuser1
REMOTE_REP_USER=zfsuser2
LOCAL_PRIV_KEY_FILE=/export/home/zfsuser1/.ssh/id_rsa
REMOTE_PRIV_KEY_FILE=/export/home/zfsuser2/.ssh/id_rsa
LOCAL_ZPOOL_RS=hasp4dataz4
REMOTE_ZPOOL_RS=hasp4dataz5
LOCAL_LH=paris-rep-lh
REMOTE_LH=newyork-rep-lh
LOCAL_DATASET=dataz4/fs1
REMOTE_DATASET=dataz5/fs1
REPLICATION_INTERVAL=
NUM_OF_SNAPSHOTS_TO_STORE=
REPLICATION_STREAM_PACKAGE=
SEND_PROPERTIES=
INTERMEDIARY_SNAPSHOTS=
RECURSIVE=
MODIFY_PASSPHRASE=
root@phys-paris-1:~# clrs disable -g gzfsgc-rg +
Oct 3 11:55:18 phys-paris-1 cl_runtime: NOTICE: Scalable service instance [TCP,10.134.28.184,80] deregistered on node paris-zc-2.
root@phys-paris-1:~# Oct 3 11:55:20 phys-paris-1 cl_runtime: NOTICE: Scalable service instance [TCP,10.134.28.184,80] deregistered on node paris-zc-1.
root@phys-paris-1:~# clrg offline gzfsgc-rg
root@phys-paris-1:~# clrg unmanage gzfsgc-rg
root@phys-paris-1:~# clrg status -Z paris-zc apache-rg
=== Cluster Resource Groups ===
Group Name Node Name Suspended Status
---------- --------- --------- ------
paris-zc:apache-rg paris-zc-2 No Offline
paris-zc-1 No Offline
root@phys-paris-1:~# geops status paris_newyork
Partnership "paris_newyork" : OK
Partner clusters : newyork
Synchronization : OK
ICRM Connection : OK
Heartbeat "hb_paris~newyork" monitoring "newyork": OK
Plug-in "ping_plugin" : Inactive
Plug-in "tcp_udp_plugin" : OK
root@phys-paris-1:~# /opt/ORCLscgrepzfssnap/util/zfs_snap_geo_register \
-f /var/tmp/ody/gzfsgcpg
sourcing /var/tmp/ody/gzfsgcpg
Creating protection group "gzfsgcpg"
This operation might take a while...
Protection group "gzfsgcpg" successfully created.
Adding the replication component "gzfs-fs1" to the protection group "gzfsgcpg"
This operation might take a while...
Password for property local_service_password :
Password for property remote_service_password :
Script-based plug-in configuration "gzfs-fs1" successfully added to the protection group "gzfsgcpg".
Wait for 60 seconds to allow potential locks to be released
Adding the application resource group(s) "gzfsgc-rg" to "gzfsgcpg"
Following resource groups successfully added:
"gzfsgc-rg".
Run "geopg get --partnership paris_newyork gzfsgcpg" on the partner cluster if this has not been done.
root@phys-paris-1:~#
Get the protection group from one node of newyork global cluster.
root@phys-newyork-1:~# clrs disable -g gzfsgc-rg +
root@phys-newyork-1:~# clrg offline gzfsgc-rg
root@phys-newyork-1:~# clrg unmanage gzfsgc-rg
root@phys-newyork-1:~# clrg status -Z newyork-zc apache-rg
=== Cluster Resource Groups ===
Group Name Node Name Suspended Status
---------- --------- --------- ------
newyork-zc:apache-rg newyork-zc-2 No Offline
newyork-zc-1 No Offline
root@phys-newyork-1:~# geopg get --partnership paris_newyork gzfsgcpg
Protection group "gzfsgcpg" successfully created.
root@phys-newyork-1:~#
From one node of paris-zc create protection group of type NONE and add application resource groups to it.
root@phys-paris-1:~# zlogin paris-zc
[Connected to zone 'paris-zc' pts/3]
Oracle Corporation SunOS 5.11 11.4 September 2018
You have new mail.
root@paris-zc-1:~# geops status paris-zc_newyork-zc
Partnership "paris-zc_newyork-zc" : OK
Partner clusters : newyork-zc
Synchronization : OK
ICRM Connection : OK
Heartbeat "hb_paris-zc~newyork-zc" monitoring "newyork-zc": OK
Plug-in "ping_plugin" : Inactive
Plug-in "tcp_udp_plugin" : OK
root@paris-zc-1:~# geopg create -s paris-zc_newyork-zc -o primary -d \
none -p External_Dependency_Allowed=true gzfszcpg
Protection group "gzfszcpg" successfully created.
root@paris-zc-1:~# clrg status apachesip-rg apache-rg
=== Cluster Resource Groups ===
Group Name Node Name Suspended Status
---------- --------- --------- ------
apachesip-rg paris-zc-2 No Online
paris-zc-1 No Offline
apache-rg paris-zc-2 No Offline
paris-zc-1 No Offline
root@paris-zc-1:~# clrs disable -g apachesip-rg,apache-rg +
root@paris-zc-1:~# clrg unmanage apachesip-rg apache-rg
clrg: (C605916) Cannot make resource group apachesip-rg UNMANAGED because it is in a non-OFFLINE state on one or more nodes in the cluster
root@paris-zc-1:~# Oct 3 12:17:10 phys-paris-1 cl_runtime: NOTICE: load balancer for group 'paris-zc:apache' released
root@paris-zc-1:~# clrg offline apachesip-rg apache-rg
clrg: (C328207) Skipping resource group apache-rg because it is unmanaged
root@paris-zc-1:~# Oct 3 12:17:30 phys-paris-1 cl_runtime: NOTICE: load balancer deleted
root@paris-zc-1:~# clrg unmanage apachesip-rg apache-rg
root@paris-zc-1:~# geopg add-resource-group apachesip-rg,apache-rg gzfszcpg
Following resource groups successfully added:
"apachesip-rg,apache-rg".
root@paris-zc-1:~#
From one node of newyork-zc, get the protection group from paris-zc.
root@phys-newyork-1:~# zlogin newyork-zc [Connected to zone 'newyork-zc' pts/2] Oracle Corporation SunOS 5.11 st_015.server February 2018 You have new mail. root@newyork-zc-1:~# clrg create -p auto_start_on_new_cluster=false apachesip-rg root@newyork-zc-1:~# clrs disable -g apachesip-rg,apache-rg + root@newyork-zc-1:~# clrg offline apachesip-rg apache-rg clrg: (C328207) Skipping resource group apachesip-rg because it is unmanaged root@newyork-zc-1:~# clrg unmanage apachesip-rg apache-rg root@newyork-zc-1:~# geopg get -s paris-zc_newyork-zc gzfszcpg Protection group "gzfszcpg" successfully created. root@newyork-zc-1:~#
On the paris site, create site, join site, create multigroup, and add protection groups to the multigroup setting dependency from the zone cluster protection group to the global cluster protection group.
root@phys-paris-1:~# geosite create -m paris-zc paris-site
Processing operation...might take several minutes to complete.
Site "paris-site" successfully created.
root@phys-paris-1:~# geomg create -s paris-site parismg
Processing operation...might take several minutes to complete.
Multigroup "parismg" successfully created.
root@phys-paris-1:~# zlogin paris-zc
[Connected to zone 'paris-zc' pts/3]
Oracle Corporation SunOS 5.11 11.4 September 2018
You have new mail.
root@paris-zc-1:~# geosite join paris paris-site
Processing operation...might take several minutes to complete.
Local cluster "paris-zc" successfully joined site "paris-site" on cluster "paris".
root@paris-zc-1:~# exit
logout
[Connection to zone 'paris-zc' pts/3 closed]
root@phys-paris-1:~# geomg add-protection-group \
paris-zc:gzfszcpg/paris:gzfsgcpg parismg
The following protection group list was successfully added to multigroup "parismg":
"paris-zc:gzfszcpg/paris:gzfsgcpg".
root@phys-paris-1:~# geomg status -v parismg
Multigroup: parismg
Description :
Site name : paris-site
Creation signature : paris Oct 3, 2018 12:28:11 PM PDT
Last update : Oct 3, 2018 12:30:19 PM PDT
Configuration status : OK
Synchronization status with :
paris-zc : OK
Timeout : 7320 seconds
Protection group list : paris-zc:gzfszcpg/paris:gzfsgcpg
Status of protection group "paris:gzfsgcpg" :
Global status : Unknown
Partnership : paris_newyork
Synchronization : OK
Cluster paris : Unknown
Role : Primary
Activation state : Deactivated
Configuration : OK
Data replication : Unknown
Resource groups : OK
Cluster newyork : Unknown
Role : Secondary
Activation state : Deactivated
Configuration : OK
Data replication : Unknown
Resource groups : OK
Status of protection group "paris-zc:gzfszcpg" :
Global status : OK
Partnership : paris-zc_newyork-zc
Synchronization : OK
Cluster paris-zc : OK
Role : Primary
Activation state : Deactivated
Configuration : OK
Data replication : None
Resource groups : OK
Cluster newyork-zc : OK
Role : Secondary
Activation state : Deactivated
Configuration : OK
Data replication : None
Resource groups : OK
root@phys-paris-1:~#
On newyork site, create site, join site, create multigroup, and add protection groups to the multigroup setting dependency from the zone cluster protection group to the global cluster protection group.
root@phys-newyork-1:~# geosite create -m newyork-zc newyork-site
Processing operation...might take several minutes to complete.
Site "newyork-site" successfully created.
root@phys-newyork-1:~# geomg create -s newyork-site newyorkmg
Processing operation...might take several minutes to complete.
Multigroup "newyorkmg" successfully created.
root@phys-newyork-1:~# zlogin newyork-zc
[Connected to zone 'newyork-zc' pts/2]
Oracle Corporation SunOS 5.11 st_015.server February 2018
You have new mail.
root@newyork-zc-1:~# geosite join newyork newyork-site
Processing operation...might take several minutes to complete.
Local cluster "newyork-zc" successfully joined site "newyork-site" on cluster "newyork".
root@newyork-zc-1:~# exit
logout
[Connection to zone 'newyork-zc' pts/2 closed]
root@phys-newyork-1:~# geomg add-protection-group \
newyork-zc:gzfszcpg/newyork:gzfsgcpg newyorkmg
The following protection group list was successfully added to multigroup "newyorkmg":
"newyork-zc:gzfszcpg/newyork:gzfsgcpg".
root@phys-newyork-1:~# geomg status -v newyorkmg
Multigroup: newyorkmg
Description :
Site name : newyork-site
Creation signature : newyork Oct 3, 2018 12:34:08 PM PDT
Last update : Oct 3, 2018 12:36:09 PM PDT
Configuration status : OK
Synchronization status with :
newyork-zc : OK
Timeout : 7320 seconds
Protection group list : newyork-zc:gzfszcpg/newyork:gzfsgcpg
Status of protection group "newyork-zc:gzfszcpg" :
Global status : OK
Partnership : paris-zc_newyork-zc
Synchronization : OK
Cluster newyork-zc : OK
Role : Secondary
Activation state : Deactivated
Configuration : OK
Data replication : None
Resource groups : OK
Cluster paris-zc : OK
Role : Primary
Activation state : Deactivated
Configuration : OK
Data replication : None
Resource groups : OK
Status of protection group "newyork:gzfsgcpg" :
Global status : Unknown
Partnership : paris_newyork
Synchronization : OK
Cluster newyork : Unknown
Role : Secondary
Activation state : Deactivated
Configuration : OK
Data replication : Unknown
Resource groups : OK
Cluster paris : Unknown
Role : Primary
Activation state : Deactivated
Configuration : OK
Data replication : Unknown
Resource groups : OK
root@phys-newyork-1:~#
Start the protection group using the geomg command.
root@phys-paris-1:~# geomg start -e global parismg
Processing operation... The timeout period for this multigroup operation is 7320 seconds...
Beginning "Start" operation on protection group "paris:gzfsgcpg"
Oct 3 12:41:05 phys-paris-1 cl_runtime: NOTICE: Load balancer setting distribution on paris-zc:apache:
Oct 3 12:41:05 phys-paris-1 cl_runtime: NOTICE: Node paris-zc-1: weight 1
Oct 3 12:41:05 phys-paris-1 cl_runtime: NOTICE: Node paris-zc-2: weight 1
Oct 3 12:41:05 phys-paris-1 cl_runtime: NOTICE: Scalable service instance [TCP,10.134.28.184,80] registered on node paris-zc-1.
"Start" operation on protection group "paris:gzfsgcpg" completed successfully
Beginning "Start" operation on protection group "paris-zc:gzfszcpg"
Oct 3 12:41:10 phys-paris-1 cl_runtime: NOTICE: Scalable service instance [TCP,10.134.28.184,80] registered on node paris-zc-2.
"Start" operation on protection group "paris-zc:gzfszcpg" completed successfully
Multigroup "parismg" successfully started.
root@phys-paris-1:~# geomg status -v parismg
Multigroup: parismg
Description :
Site name : paris-site
Creation signature : paris Oct 3, 2018 12:28:11 PM PDT
Last update : Oct 3, 2018 12:30:19 PM PDT
Configuration status : OK
Synchronization status with :
paris-zc : OK
Timeout : 7320 seconds
Protection group list : paris-zc:gzfszcpg/paris:gzfsgcpg
Status of protection group "paris:gzfsgcpg" :
Global status : OK
Partnership : paris_newyork
Synchronization : OK
Cluster paris : OK
Role : Primary
Activation state : Activated
Configuration : OK
Data replication : OK
Resource groups : OK
Cluster newyork : OK
Role : Secondary
Activation state : Activated
Configuration : OK
Data replication : OK
Resource groups : OK
Status of protection group "paris-zc:gzfszcpg" :
Global status : OK
Partnership : paris-zc_newyork-zc
Synchronization : OK
Cluster paris-zc : OK
Role : Primary
Activation state : Activated
Configuration : OK
Data replication : None
Resource groups : OK
Cluster newyork-zc : OK
Role : Secondary
Activation state : Activated
Configuration : OK
Data replication : None
Resource groups : OK
root@phys-paris-1:~# clrs status -Z paris-zc apache
=== Cluster Resources ===
Resource Name Node Name State Status Message
------------- --------- ----- --------------
apache paris-zc-2 Online Online - Service is online.
paris-zc-1 Online Online - Service is online.
root@phys-paris-1:~#
In the Zone cluster, add data to the ZFS-based cluster filesystem that is being replicated. This is for illustrating replication functioning after switchover in later step.
root@phys-paris-1:~# zlogin paris-zc [Connected to zone 'paris-zc' pts/3] Oracle Corporation SunOS 5.11 11.4 September 2018 You have new mail. root@paris-zc-1:~# mkdir /global/apache/docs/testdir root@paris-zc-1:~# cp -r /net/howtos/QA_Docs/* /global/apache/docs/testdir root@paris-zc-1:~# ls /global/apache/docs/testdir BestPractice-pplsft.odt howto-2.sxw howto-Template.odt HA-Oracle-on-ARNAS Oracle.Solaris.Cluster.Cheat.Sheet.Template.9.16.10.odt PeopleSoft_and_SolarisCluster_install_and_config_procedure.odt PeopleSoft_and_SolarisCluster_install_and_config_procedure.odt.copy Peoplesoft_scheduler_setup_instructions.txt Peoplesoft_scheduler_setup_instructions.txt.old README--Install-GIB3-in-zone-cluster-v1.odt Solaris_Cluster_3.3_RAC_10g_LDOM.docx Solaris_Cluster_3.3_RAC_10g_LDOM.odt Solaris_Cluster_3.3_RAC_10g_LDOM.pdf Technical.Instruction.Template.ott cheatsheet-HA-PeopleSoft-zone-cluster.txt cheatsheet-setup-HA-PeopleSoft-process-scheduler-ZC.txt cluster-webas-zones-install.pdf geo2sitesHTG.odt how_to_run_scate.odt how_to_run_scate.txt jumbo.odt mpxio.odt oldcli_to_newcli_mapping.odt oldcli_to_newcli_mapping.odt.copy problems_solutions.txt quorum_server_setup.txt rac_upgrade.odt rac_upgrade.txt scate_ha_oracle_load.odt scate_rac_load.odt scate_requirements.html scge-odg-oow.odt setup_avs.odt setup_avs.txt setup_ipv6.odt setup_ipv6.txt setup_ipv6.txt.copy setup_samp_2zc.txt setup_tx.odt setup_tx.txt vxvm_document.odt xxPeoplesoft_scheduler_setup_instructions.txtxx root@paris-zc-1:~#
Switchover multigroup to newyork site, verify that data in the filesystem exists on this cluster, and complete setting up resources for apache on this site.
root@phys-newyork-1:~# geomg status -v newyorkmg
Multigroup: newyorkmg
Description :
Site name : newyork-site
Creation signature : newyork Oct 3, 2018 12:34:08 PM PDT
Last update : Oct 3, 2018 12:36:09 PM PDT
Configuration status : OK
Synchronization status with :
newyork-zc : OK
Timeout : 7320 seconds
Protection group list : newyork-zc:gzfszcpg/newyork:gzfsgcpg
Status of protection group "newyork-zc:gzfszcpg" :
Global status : OK
Partnership : paris-zc_newyork-zc
Synchronization : OK
Cluster newyork-zc : OK
Role : Secondary
Activation state : Activated
Configuration : OK
Data replication : None
Resource groups : OK
Cluster paris-zc : OK
Role : Primary
Activation state : Activated
Configuration : OK
Data replication : None
Resource groups : OK
Status of protection group "newyork:gzfsgcpg" :
Global status : OK
Partnership : paris_newyork
Synchronization : OK
Cluster newyork : OK
Role : Secondary
Activation state : Activated
Configuration : OK
Data replication : OK
Resource groups : OK
Cluster paris : OK
Role : Primary
Activation state : Activated
Configuration : OK
Data replication : OK
Resource groups : OK
root@phys-newyork-1:~# geomg switchover -s newyork-site newyorkmg
Are you sure you want to switch over multigroup 'newyorkmg' to site 'newyork-site'? (yes|no) > yes
Processing operation... The timeout period for this multigroup operation is 7320 seconds...
Beginning "Switchover" operation on protection group "newyork-zc:gzfszcpg"
Beginning "Switchover" operation on protection group "newyork:gzfsgcpg"
"Switchover" operation on protection group "newyork:gzfsgcpg" completed successfully
"Switchover" operation on protection group "newyork-zc:gzfszcpg" completed successfully
"Switchover" operation succeeded for the multigroup "newyorkmg".
root@phys-newyork-1:~# geomg status -v newyorkmg
Multigroup: newyorkmg
Description :
Site name : newyork-site
Creation signature : newyork Oct 3, 2018 12:34:08 PM PDT
Last update : Oct 3, 2018 12:36:09 PM PDT
Configuration status : OK
Synchronization status with :
newyork-zc : OK
Timeout : 7320 seconds
Protection group list : newyork-zc:gzfszcpg/newyork:gzfsgcpg
Status of protection group "newyork-zc:gzfszcpg" :
Global status : OK
Partnership : paris-zc_newyork-zc
Synchronization : OK
Cluster newyork-zc : OK
Role : Primary
Activation state : Activated
Configuration : OK
Data replication : None
Resource groups : OK
Cluster paris-zc : OK
Role : Secondary
Activation state : Activated
Configuration : OK
Data replication : None
Resource groups : OK
Status of protection group "newyork:gzfsgcpg" :
Global status : OK
Partnership : paris_newyork
Synchronization : OK
Cluster newyork : OK
Role : Primary
Activation state : Activated
Configuration : OK
Data replication : OK
Resource groups : OK
Cluster paris : OK
Role : Secondary
Activation state : Activated
Configuration : OK
Data replication : OK
Resource groups : OK
root@phys-newyork-1:~# zlogin newyork-zc
[Connected to zone 'newyork-zc' pts/2]
Oracle Corporation SunOS 5.11 st_015.server February 2018
You have new mail.
root@newyork-zc-1:~# ls /global/apache/docs/testdir
BestPractice-pplsft.odt
howto-2.sxw
howto-Template.odt
HA-Oracle-on-ARNAS
Oracle.Solaris.Cluster.Cheat.Sheet.Template.9.16.10.odt
PeopleSoft_and_SolarisCluster_install_and_config_procedure.odt
PeopleSoft_and_SolarisCluster_install_and_config_procedure.odt.copy
Peoplesoft_scheduler_setup_instructions.txt
Peoplesoft_scheduler_setup_instructions.txt.old
README--Install-GIB3-in-zone-cluster-v1.odt
Solaris_Cluster_3.3_RAC_10g_LDOM.docx
Solaris_Cluster_3.3_RAC_10g_LDOM.odt
Solaris_Cluster_3.3_RAC_10g_LDOM.pdf
Technical.Instruction.Template.ott
cheatsheet-HA-PeopleSoft-zone-cluster.txt
cheatsheet-setup-HA-PeopleSoft-process-scheduler-ZC.txt
cluster-webas-zones-install.pdf
geo2sitesHTG.odt
how_to_run_scate.odt
how_to_run_scate.txt
jumbo.odt
mpxio.odt
oldcli_to_newcli_mapping.odt
oldcli_to_newcli_mapping.odt.copy
problems_solutions.txt
quorum_server_setup.txt
rac_upgrade.odt
rac_upgrade.txt
scate_ha_oracle_load.odt
scate_rac_load.odt
scate_requirements.html
scge-odg-oow.odt
setup_avs.odt
setup_avs.txt
setup_ipv6.odt
setup_ipv6.txt
setup_ipv6.txt.copy
setup_samp_2zc.txt
setup_tx.odt
setup_tx.txt
vxvm_document.odt
xxPeoplesoft_scheduler_setup_instructions.txtxx
root@newyork-zc-1:~# clrg status apachesip-rg apache-rg
=== Cluster Resource Groups ===
Group Name Node Name Suspended Status
---------- --------- --------- ------
apachesip-rg newyork-zc-2 No Online
newyork-zc-1 No Offline
apache-rg newyork-zc-2 No Online
newyork-zc-1 No Online
root@newyork-zc-1:~# clrs list -g apachesip-rg
root@newyork-zc-1:~# clrs list -g apache-rg hasp4apache
root@newyork-zc-1:~# clrssa create -g apachesip-rg apache-lh
root@newyork-zc-1:~# clrg set -p rg_dependencies=apachesip-rg apache-rg
root@newyork-zc-1:~# clrt register apache
root@newyork-zc-1:~# clrs create -g apache-rg -t apache -p bin_dir=/global/apache/bin \
-p resource_dependencies_offline_restart=hasp4apache -p \
resource_dependencies=apache-lh -p scalable=true apache
Oct 3 14:33:16 phys-newyork-1 cl_runtime: NOTICE: Service group 'newyork-zc:apache' created
Oct 3 14:33:16 phys-newyork-1 cl_runtime: NOTICE: Service object [TCP, 10.134.28.184, 80] created in group 'newyork-zc:apache'
Oct 3 14:33:16 phys-newyork-1 cl_runtime: NOTICE: Load balancer setting distribution on newyork-zc:apache:
Oct 3 14:33:16 phys-newyork-1 cl_runtime: NOTICE: Node newyork-zc-1: weight 1
Oct 3 14:33:16 phys-newyork-1 cl_runtime: NOTICE: Node newyork-zc-2: weight 1
Oct 3 14:33:17 phys-newyork-1 cl_runtime: NOTICE: Scalable service instance [TCP,10.134.28.184,80] registered on node newyork-zc-1.
Oct 3 14:33:17 phys-newyork-1 cl_runtime: NOTICE: Scalable service instance [TCP,10.134.28.184,80] registered on node newyork-zc-2.
root@newyork-zc-1:~# clrs status apache
=== Cluster Resources ===
Resource Name Node Name State Status Message
------------- --------- ----- --------------
apache newyork-zc-2 Online Online - Completed successfully.
newyork-zc-1 Online Online - Completed successfully.
root@newyork-zc-1:~# geomg status -v newyorkmg
Multigroup: newyorkmg
Description :
Site name : newyork-site
Creation signature : newyork Oct 3, 2018 12:34:08 PM PDT
Last update : Oct 3, 2018 12:36:09 PM PDT
Configuration status : OK
Synchronization status with :
newyork : OK
Timeout : 7320 seconds
Protection group list : newyork-zc:gzfszcpg/newyork:gzfsgcpg
Status of protection group "newyork-zc:gzfszcpg" :
Global status : OK
Partnership : paris-zc_newyork-zc
Synchronization : OK
Cluster newyork-zc : OK
Role : Primary
Activation state : Activated
Configuration : OK
Data replication : None
Resource groups : OK
Cluster paris-zc : OK
Role : Secondary
Activation state : Activated
Configuration : OK
Data replication : None
Resource groups : OK
Status of protection group "newyork:gzfsgcpg" :
Global status : OK
Partnership : paris_newyork
Synchronization : OK
Cluster newyork : OK
Role : Primary
Activation state : Activated
Configuration : OK
Data replication : OK
Resource groups : OK
Cluster paris : OK
Role : Secondary
Activation state : Activated
Configuration : OK
Data replication : OK
Resource groups : OK
root@newyork-zc-1:~#