Note:

• This tutorial requires access to Oracle Cloud. To sign up for a free account, see Get started with Oracle Cloud Infrastructure Free Tier.
• It uses example values for Oracle Cloud Infrastructure credentials, tenancy, and compartments. When completing your lab, substitute these values with ones specific to your cloud environment.

Invoke custom scripts using the run command with Oracle Cloud Infrastructure Full Stack Disaster Recovery

Introduction

Oracle Cloud Infrastructure (OCI) Full Stack Disaster Recovery orchestrates the transition of compute, database, and applications between OCI regions from around the globe with a single click. Customers can automate the steps needed to recover one or more business systems without redesigning or re-architecting existing infrastructure, databases, or applications and without needing specialized management or conversion servers.

OCI Full Stack Disaster Recovery offers exceptional flexibility. It allows you to utilize user-defined groups within the Disaster Recovery plan, enabling the management of customized steps as part of the overall disaster recovery plan. Once Full Stack Disaster Recovery establishes the initial disaster recovery plan with built-in plan groups and steps, you can incorporate additional user-defined plan groups and steps. You can invoke scripts and OCI Functions with a user-defined plan group.

OCI Compute run command is performed using the Oracle Cloud Agent available in OCI Compute instance. Oracle Cloud Agent is a lightweight process that manages plugins running on the OCI Compute instance. Plugins collect performance metrics, install OS updates, and perform other instance management tasks. To execute scripts using the run command, you must have the compute instances (moving or non-moving) added as members in the disaster recovery protection group.

Full Stack Disaster Recovery creates a VM in the disaster recovery region for moving instances during disaster recovery plan execution. Once the VMs are launched, you can use the run command feature to perform additional steps using scripts. Examples: Load balancer configuration changes, Connect to a database and update table, stop and start application servers. Based on your requirements, you could also use cloud-init or systemd configuration in the compute instance to start your applications or any other operations during the instance launch. The run command option in the disaster recovery plan has better flexibility, like using scripts written in any language, and observability options like stop/restart/log monitoring at a single plane of glass. For non-moving instances, since the VMs are already created in the disaster recovery region, the run command feature would be the ideal option for using scripts to start/stop apps, databases, and so on.

In this tutorial will discuss how to use OCI Compute run Command feature using moving instance and execute scripts as part of a Full Stack Disaster Recovery plan User-defined plan group and steps.

Objectives

To move an instance between two OCI regions and invoke custom scripts using the OCI Compute run Command feature in User-defined Plan Groups and Steps in Full Stack Disaster Recovery Plan. Primary region is Ashburn and Disaster Recovery region is Phoenix.

You can use “administrator” or equivalent privileges user to create Oracle Cloud Infrastructure Identity and Access Management (OCI IAM) resources and perform Full Stack Disaster Recovery operations with a run command. Alternatively, you could create a separate user for Full Stack Disaster Recovery administrator and assign necessary OCI IAM resources to perform Full Stack Disaster Recovery operations with the run command. This article explains how to use “administrator” access to create OCI IAM resources and Full stack administrator user to perform Full Stack Disaster Recovery operations.

The tutorial is intended to show the steps needed to execute a working script from the Full Stack Disaster Recovery user-defined plan group using the run command.

The same tutorial should work for intra regions if you configure block replication across different availability domains.

For more details, see Running Commands on an Instance.

The following steps will be covered in the tutorial

1. Create User, Group and Policy for Full Stack Disaster Recovery
2. Create Dynamic Groups and update policy for Full Stack Disaster Recovery and run Command
3. Configure and validate run Command in the Primary region VM
4. Create volume groups and enable cross region Replication for the Primary region VM
5. Create and associate Disaster Recovery protection groups (DRPG)
6. Create switchover plan in the standby DRPG
7. Customize the switchover plan with user defined plan groups to execute run Command
8. Execute prechecks and run switchover plan

Deployment Architecture

Prerequisites

• Administrator privileges or Configure the required OCI IAM policies for Full Stack Disaster Recovery as outlined here: Configuring Identity and Access Management (IAM) policies to use Full Stack Disaster Recovery and Policies for Full Stack Disaster Recovery

• Running Commands on OCI Compute instance: See Running Commands on OCI Compute instance

• Create Object Storage bucket in the Primary region and Standby region to store Disaster Recovery operation logs: See Object Storage

Implementation

For this tutorial, we have already created the following resources.

Resources	Primary Region - Ashburn(IAD)
Object Storage Bucket	fsdr-bucket-ash
Compartment	cmp-compute-fsdrdemo
Compute VM	demo-fsdr-inst
Script to create a folder and file	/home/opc/scripts/testCommand.sh

#!/bin/bash
# coding: utf-8
echo "Creating Dir testDir under /home/opc"
mkdir -p "/home/opc/testDir"
echo "Creating Testfile"
echo "Contents from `run` Command!!" > /home/opc/testDir/testFile.txt

Resources	Standby Region - Phoenix(PHX)
Object Storage Bucket	fsdr-bucket-phx
Compartment	cmp-compute-fsdrdemo

Task 1: Create Users, Groups and Policies for Full Stack Disaster Recovery

1. Log in to the OCI console with administrator credentials or user who has equivalent privileges to create and modify necessary OCI IAM resources.

2. In the Primary region (Ashburn) from the Hamburger menu, navigate to Identity and Security, Domains, Default,Users. Click Create user, provide First name, Last name, Email id, and then click Create.

   

3. In the Primary region (Ashburn) from the Hamburger menu, go to Identity and Security, Domains, Default, Groups. Click Create group, provide group name as “demo-fsdr-admins”, “description”, select users to assign this group - select the user created in Step 1.

   

4. In the Primary region (Ashburn) from the Hamburger menu, go to Identity and Security, Domains,Policies,Create Policy in root compartment or a compartment where you want to create the policy. disaster-recovery-family is the family type for Full Stack Disaster Recovery.

   

   Allow group demo-fsdr-admins to manage disaster-recovery-family in compartment cmp-compute-fsdrdemo
   

5. To enable disaster recovery service to manage OCI resources that are part of application stack, you must configure policy-based access to allow access to these resources. We added all the OCI service policies which are currently supported as member type. Policies can be modified depending on the resource types which are used with Full Stack Disaster Recovery. Add following policies to demo-fsdr-admin-policies as shown below.

   Allow group demo-fsdr-admins to manage buckets in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to manage objects in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to manage databases in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to manage autonomous-databases in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to manage instance-family in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to manage instance-agent-command-family in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to manage volume-family in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to read virtual-network-family in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to use subnets in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to use vnics in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to use network-security-groups in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to use private-ips in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to read fn-app in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to read fn-function in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins to use fn-invocation in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins use tag-namespaces in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins read vaults in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins read secret-family in compartment cmp-compute-fsdrdemo
   Allow group demo-fsdr-admins read all-resources in compartment cmp-compute-fsdrdemo
   

6. From the menu, navigate to Identity & Security, Policies, select the policy created in Step 4, select Edit Policy Statements, Policy Builder, Advanced.

7. Copy the list of policies as above and click Save changes. Verify whether all the added policies are reflected correctly.

   Note: You should use the correct group name (created in Step 3) and compartment name in the policy syntax.

Task 2: Create Dynamic Groups and Policies for Full Stack Disaster Recovery and run Command

1. Log in to the OCI console with administrator credentials or user who has equivalent privileges to create and modify necessary OCI IAM resources.

2. To allow compute instance to make API Calls against OCI Services, we need to create Dynamic groups and make compute instances as principal actors. Create Dynamic group In the Primary region (Ashburn) from the Hamburger menu, go to Identity & Security, Domains,Dynamic Groups, Create Dynamic Group.

   • If you have multiple instances in the compartment, you can use the Rule All {instance.compartment.id = 'compartment OCID'}. In this example, we have copied OCID of the compartment cmp-compute-fsdrdemo and this allows all instances in this compartment to use run commands.

     

   • You can also use dynamic group to restrict access with instance level by adding rules as Any {instance.id=’ocid….’}.

3. Update the policy demo-fsdr-admins which we have created in Task 1 with the below policy. This is required to use run command functions like execute commands, cancel commands, and view the command output for the instances in a compartment.

   Allow dynamic-group demofsdr-runcmd-dgp to manage instance-agent-command-execution-family in compartment cmp-compute-fsdrdemo
   

   

Note: If your script interacts with other OCI services, you should update the dynamic group policies.

Task 3. Configure and validate run command in the source VM

1. Log in to the OCI console using the Full Stack Disaster Recovery administrator user which we have created in Task 1.

2. Log in to the compute VM (demo-fsdr-inst) as opc user. Make sure to place the script testCommand.sh in /home/opc/scripts/ folder and change the permissions of the file with execute privileges. Contents of scripts are available in the Implementation section at the top of the page.

3. You must grant administrator permissions to the Compute Instance run command plugin to be able to run the command. The plugin runs as the ocarun user.

4. Create a sudoers configuration file for the Compute Instance run Command plugin.

   vi ./101-oracle-cloud-agent-run-command
   

   

5. Allow the ocarun user to run all commands as sudo by adding the following line to the configuration file.

   ocarun ALL=(ALL) NOPASSWD:ALL
   

   

   You can optionally list specific commands. See the Linux man page for sudoers for more information.

6. Validate if the syntax in the configuration file is correct.

   visudo -cf ./101-oracle-cloud-agent-run-command
   

   If the syntax is correct, the following message is returned:

   ./101-oracle-cloud-agent-run-command: parsed OK

   

7. Add the configuration file to /etc/sudoers.d.

   sudo cp ./101-oracle-cloud-agent-run-command /etc/sudoers.d/
   

   

   Note: Steps to grant administrator permissions depend on the operating system. In this example, we have used Linux OS. For more details, see Running Commands on an Instance.

8. Restart Cloud agent manually using the below command. This command is applicable for Oracle Linux 7.x and Oracle Linux 8.x. For other Linux versions and operating system, refer Running Commands on an Instance.

   sudo systemctl restart oracle-cloud-agent
   

   

9. Log in to the OCI console, navigate to the demo-fsdr-inst VM, select Oracle Cloud Agent tab and Compute Instance run Command status should display Running

   

   Note: If the status is not running, refer Troubleshooting Oracle Cloud Agent, to fix the issue. The status of Compute Instance run command should be running.

10. Navigate to Instances, demo-fsdr-inst, run Command, Create run Command.

    

11. Test run command as below.

    

12. Check Status of command by clicking three dots in the right corner.

    

13. You should see the command output and status as shown below.

    

To troubleshoot run command in the VM, you can view the logs that the plugin generates using the following command.

tail -f /var/log/oracle-cloud-agent/plugins/runcommand/runcommand.log

For known errors and resolution, refer Troubleshooting the Compute Instance run Command Plugin section in Running Commands on an Instance.

Task 4: Create Volume Groups in Primary (Ashburn)

1. Log in to the OCI console using the Full Stack Disaster Recovery administrator user which we have created in Task 1.

2. In the Primary region (Ashburn) from the Hamburger menu, go to Migration & Disaster Recovery, Disaster Recovery, Disaster Recovery Protection group, Create Disaster Recovery protection group.

   

3. Add Boot volume of the Compute demo-fsdr-inst.

   

4. Enable cross-region replication from Ashburn to Phoenix.

   

5. Leave Backup policy as None, click Next and review the summary.

   

Task 5: Create Disaster Recovery Protection Groups in Primary and standby

1. Log in to the OCI console using the Full Stack Disaster Recovery administrator user which we have created in Task 1.

2. In the Primary region (Ashburn) from the Hamburger menu, go to Migration & Disaster Recovery, Disaster Recovery, Disaster Recovery Protection group, Create Disaster Recovery protection group.

   

3. In the Standby region (Phoenix) from the Hamburger menu, go to Migration & Disaster Recovery, Disaster Recovery, Disaster Recovery Protection group, Create Disaster Recovery protection group.

   

   Note: For this tutorial example, we choose Ashburn as Primary region and Phoenix as Standby region.

4. In Primary region (Ashburn), select the fsdr-runcmd-drpg-ash DRPG and click Associate. Select the Role as Primary, Peer Region US West (Phoenix), Peer Disaster Recovery Protection group fsdr-runcmd-drpg-phx and click Associate.

   

   Note: Verify the role of the fsdr-runcmd-drpg-ash as Primary and fsdr-runcmd-drpg-phx as Standby.

5. In Primary region (Ashburn), select the fsdr-runcmd-drpg-ash DRPG and click Members and then click Add Members to add Volume Groups and VM.

   

Task 6: Create switchover plan in the standby DRPG (Phoenix)

Switchover Plan – A type of Disaster Recovery plan that performs a planned transition of services from the primary Disaster Recovery protection group to the standby Disaster Recovery protection group.

1. Log in to the OCI console using the Full Stack Disaster Recovery administrator user which we have created in Task 1.

2. Create switchover plan in standby DRPG fsdr-runcmd-drpg-phx.

3. Select Plans and click Create plan.

4. Provide name as switchover-to-phx and choose plan type as Switchover (planned).

5. Click Create.

   

   Note: Make sure to create the switchover plan from the Standby DRPG (Phoenix).

6. Click the Plan and ensure built-in plans have been generated as shown below.

   

Task 7: Customize the switchover plan with user defined plan groups

1. Log in to the OCI console using the Full Stack Disaster Recovery administrator user which we have created in Task 1.

2. Under Plan Groups, click Add group, to create custom plan group to perform switchover. Provide Group name as CustGroup-Demo.

   

3. Click Add Step and add details as below. We will be adding a step in the plan group.

   

   • Make sure to choose region as Ashburn. Select the run local Script option.

   • Considering we don’t have a VM in Phoenix during the user-defined plan group creation,need to choose Region as Ashburn, which is Primary, and Target instance as demo-for-inst VM. Full Stack Disaster Recovery will automatically map those instances, and the script will be executed in the newly launched instance in Phoenix.

   • Select the Target Instance in the compartment where the VM is created, we have created the vm as demo-fsdr-inst.

   • Enter Script Parameter as /home/opc/scripts/testCommand.sh.

   • Enter opc in run as user field.

   • Click Add Step and verify the added step.

     

   • Verify the step is added and click Add.

4. Verify switchover plan groups in the right sequence as shown above.

Task 8: Execute prechecks and run switchover plan

The Run prechecks option performs a quick validation of all steps in a Disaster Recovery (Disaster Recovery) plan.

1. From the switchover plan switchover-to-phx, click Run prechecks to launch the Run prechecks dialog.

   

   

   Verify for the successful completion.

2. From the switchover plan switchover-to-phx, click Execute Disaster Recovery Plan to execute the plan.

   

3. Check the status of the plan and ensure all the steps in the plan succeed.

   

4. After the successful switchover plan execution, log in to the instance in Phoenix and verify the testDir and testFile have been created.

   

5. Automatically the role change will happen in the disaster recovery Protection group. Now, Phoenix will have Primary and Ashburn will have Standby role.

Next Steps

You have seen how to use custom groups to invoke scripts with OCI Full Stack Disaster Recovery. Ideally you will have steps to recover compute, database, block storage along with object storage for your application stack. You can easily use this tutorial to customize your Full Stack Disaster Recovery plan if you want to add any user defined group as part of user defined plan group.

Related Links

• OCI Full Stack Disaster Recovery Product Page
• OCI Full Stack Disaster Recovery Documentation
• Full Stack Disaster Recovery Live Labs
• Full Stack Disaster Recovery for Cold VM Disaster Recovery
• Full Stack Disaster Recovery for WLS
• Full Stack Disaster Recovery - User defined group scripts
• Join #full-stack-dr public slack channel

Acknowledgments

Authors - Vasudeva Manikandan (Implementation Tech Lead - Oracle Lift Services), Suraj Ramesh (Full Stack Disaster Recovery Product Manager)

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.

---

Title and Copyright Information

Invoke custom scripts using the run command with Oracle Cloud Infrastructure Full Stack Disaster Recovery

F86012-01

August 2023

Copyright © 2023, Oracle and/or its affiliates.