Note:

• This tutorial requires access to Oracle Cloud. To sign up for a free account, see Get started with Oracle Cloud Infrastructure Free Tier.
• It uses example values for Oracle Cloud Infrastructure credentials, tenancy, and compartments. When completing your lab, substitute these values with ones specific to your cloud environment.

Automate Switchover and Failover for OCI Object Storage buckets using Functions with OCI Full Stack Disaster Recovery

Introduction

Oracle Cloud Infrastructure (OCI) Full Stack Disaster Recovery orchestrates the transition of compute, database, and applications between OCI regions from around the globe with a single click. Customers can automate the steps needed to recover one or more business systems without redesigning or re-architecting existing infrastructure, databases, or applications and without needing specialized management or conversion servers.

Full Stack Disaster Recovery is highly flexible service, you can take advantage of user defined groups in the Disaster Recovery plan to handle object storage bucket Disaster Recovery operations. Full Stack Disaster Recovery has built-in integration with OCI Functions. OCI Functions is a serverless platform that enables you to create, run, and scale business logic without managing any infrastructure. Full Stack Disaster Recovery allows you to run a function in any region at any step using the user defined groups option available with the Full Stack Disaster Recovery plan.

Objectives

To automate Switchover and Failover for OCI object storage buckets with Oracle Cloud Infrastructure (OCI) Full Stack Disaster Recovery. Primary region is Ashburn and Disaster Recovery region is Phoenix. The following steps will be covered in this tutorial.

1. Create and associate Disaster Recovery protection groups (DRPG)
2. Create switchover plan in the standby DRPG
3. Customize the switchover plan with user defined plan groups
4. Execute prechecks and run switchover plan
5. Create failover plan in the standby DRPG
6. Customize the failover plan with user defined plan groups
7. Execute prechecks and run failover plan

Deployment Architecture

Prerequisites

• Configure Object Storage cross-region replication for Disaster Recovery Cross-region replication for Disaster Recovery
• For OCI functions integration with Object storage, create Oracle Cloud Infrastructure Identity and Access Management (OCI IAM) policies and dynamic groups as outlined here:OCI Function for object storage Disaster Recovery
• Create function for Object storage bucket Disaster Recovery operations using OCI Function and OCI Function for object storage Disaster Recovery
• Administrator privileges or Configure the required OCI IAM policies for Full Stack Disaster Recovery as outlined here: Configuring Identity and Access Management (IAM) policies to use Full Stack Disaster Recovery and Policies for Full Stack Disaster Recovery
• Create Object Storage bucket in the Primary region and Standby region to store Disaster Recovery operation logs: Object Storage

Implementation

For this tutorial, we have already created the below resources.

• Object storage buckets and set up the cross region replication policy. Read Write bucket is Ashburn and Read only bucket is Phoenix regions.

• Create functions in Ashburn and Phoenix region for Object Storage bucket Disaster Recovery.

  Resources	Primary Region - Ashburn(IAD)
  Object Storage Bucket	iad-bucket
  OCI Function	oci-objectstorage-fullstackdr-python

  Resources	Standby Region - Phoenix(PHX)
  Object Storage Bucket	phx-bucket
  OCI Function	oci-objectstorage-fullstackdr-python

• OCI Function which we have created for object storage Disaster Recovery will have these two options:

  • Switchover: Replication policy on the source bucket will be deleted (this will enable the target bucket to become read/write automatically).Reverse replication will be setup on the target bucket (i.e., a new replication policy will be created in the target bucket. The replication will be reversed from target to source)

  • Failover: It is assumed that source region is unreachable during failover.Replication will be stopped on the target bucket (this will enable the bucket to become read/write and disables the replication policy on source when it’s reachable. Policy will display an error).Additional housekeeping needs to be done after failover, refer OCI Function for object storage Disaster Recovery for more details.

Task 1: Create and associate Disaster Recovery protection groups (DRPG)

1. In the Primary region (Ashburn) from the Hamburger menu, go to Migration & Disaster Recovery, Disaster Recovery, Disaster Recovery Protection group, Create Disaster Recovery protection group.

   

2. In the Standby region (Phoenix) from the Hamburger menu, go to Migration & Disaster Recovery, Disaster Recovery, Disaster Recovery Protection group, Create Disaster Recovery protection group.

   

   Note: For this tutorial example, we choose Ashburn as Primary region and Phoenix as Standby region.

3. In Primary region (Ashburn), select the objectstorage-iad DRPG and click Associate. Select the Role as Primary, Peer Region US West (Phoenix), Peer Disaster Recovery Protection group objectstorage-phx and click Associate.

   

Note: Verify the role of the objectstorage-iad as Primary and objectstorage-phx as Standby.

Task 2: Create switchover plan in the standby DRPG (Phoenix)

Switchover Plan – A type of Disaster Recovery plan that performs a planned transition of services from the primary Disaster Recovery protection group to the standby Disaster Recovery protection group.

1. Create switchover plan in standby DRPG objectstorage-phx.

2. Select Plans and click Create plan.

3. Provide name as switchover-to-phx and choose plan type as Switchover (planned).

4. Click Create.

   

   Note: Make sure to create the switchover plan from the Standby DRPG (Phoenix).

Task 3: Customize the switchover plan with user defined plan groups

1. Since we do not have any members added in the DRPG, the switchover plan will not have any built in plan groups and we need to customize the switchover plan with additional used defined plan groups.

   

2. Under Plan Groups click Add group, to create custom plan group to perform object storage bucket Disaster Recovery switchover. Provide Group name as objectstorage-switchover.

   

3. Click Add Step and add details as below. We will be adding a step in the plan group.

   

   • Make sure to choose region as Phoenix. Select the Invoke function option.

   • Select the Target application in the compartment where the application is created, we have created the application as fullstackdr.

   • Select the Target function, we have created the target function as oci-objectstorage-python-storage.

   • Add the payload with the source and target buckets,source and target regions,Disaster Recovery operation (switchover).

     {"sourceBucket": "iad-bucket", "targetBucket": "phx-bucket", "sourceRegion": "us-ashburn-1", "targetRegion": "us-phoenix-1", "operation": "switchover"}

   • Click Add Step and verify the added step.

     

   • Verify the step is added and click Add.

4. Verify switchover plan groups in the right sequence.

   

Task 4: Execute prechecks and run switchover plan

The Run Prechecks option performs a quick validation of all steps in a Disaster Recovery (Disaster Recovery) plan.

1. From the switchover plan switchover-to-phx, click Run Prechecks to launch the Run Prechecks dialog.

   

   

   Verify for the successful completion.

2. From the switchover plan switchover-to-phx, click Execute Disaster Recovery Plan to execute the plan.

   

3. Check the status of the plan and ensure all the steps in the plan succeed.

   

4. After the successful switchover plan execution, replication policy in the iad-bucket will be deleted,phx-bucket will become read-write. Reverse replication will be setup on the phx-bucket with destination as iad-bucket, hence iad-bucket will become read-only.

5. Automatically the role change will happen in the DRPG. Now, Phoenix will have Primary and Ashburn will have Standby role.

6. Verify the phx-bucket.

   

7. Verify the iad-bucket

   

Task 5: Create failover plan in the standby DRPG (Ashburn)

With Task 4, we have completed the switchover of the object storage bucket from Ashburn to Phoenix. Now let’s do failover of the object storage bucket from Phoenix to Ashburn.

Failover Plan – A type of Disaster Recovery plan that performs an unplanned transition of services to the standby Disaster Recovery protection group.

1. Create failover plan in standby DRPG objectstorage-iad.

2. Select Plans and click Create plan.

3. Provide name as failover-to-iad and choose plan type as Failover (unplanned).

4. Click Create.

   

   Note: Make sure to create the failover plan from the Standby DRPG (Ashburn).

Task 6: Customize the failover plan with user defined plan groups

1. Since we do not have any members added in the DRPG, the failover plan will not have any built in plan groups and we need to customize the failover plan with additional used defined plan groups.

   

2. Under Plan Groups click Add group, to create custom plan group to perform object storage bucket Disaster Recovery failover. Provide Group name as objectstorage-failover.

   

3. Click Add Step and add details as below. We will be adding a step in the plan group.

   

   • Make sure to choose region as Ashburn.Select the Invoke function option.

   • Select the Target application in the compartment where the application is created, we have created the application as oss

   • Select the Target function, we have created the target function as oci-objectstorage-python-storage

   • Add the payload with the source and target buckets,source and target regions,Disaster Recovery operation (failover).

     {"sourceBucket": "phx-bucket", "targetBucket": "iad-bucket", "sourceRegion": "us-phoenix-1", "targetRegion": "us-ashburn-1", "operation": "failover"}

   • Click Add Step and verify the added step.

     

   • Verify the step is added and hit Add

4. Verify failover plan groups in the right sequence.

   

Task 7: Execute prechecks and run the failover plan

The Run Prechecks option performs a quick validation of all steps in a Disaster Recovery (Disaster Recovery) plan.

1. From the failover plan failover-to-phx,click Run Prechecks to launch the Run Prechecks dialog.

   

   

   Verify for the successful completion.

2. From the failover plan failover-to-phx,Click Execute Disaster Recovery Plan to execute the plan.

   

3. Check the status of the plan and ensure all the steps in the plan succeed.

   

4. After the successful failover plan execution, replication will be stopped in the iad-bucket bucket and iad-bucket bucket will become read-write. It also disables the replication policy on phx-bucket bucket, assuming the region in reachable. Policy will show error.

5. Automatically the role change will happen in the DRPG. Now, Ashburn will have Primary and Phoenix will have Standby role.

6. Verify the iad-bucket.

   

7. Verify the phx-bucket.

   

8. Assuming if the original bucket (phx-bucket) is available, the following steps needs to be manually performed to clean up replication policies after a failover operation.

   • Delete the replication policy in old source (bucket-phx) bucket (i.e. policy which shows error).
   • Manually setup replication policy in the new source (bucket-iad) bucket to the new target bucket (bucket-phx)

Next Steps

You have seen how to automatically switchover and failover OCI object storage buckets using Functions with OCI Full Stack Disaster Recovery. Ideally you will have steps to recover compute, database, block storage along with object storage for your application stack. You can easily use this tutorial to customize your Full Stack Disaster Recovery plan if you want to add object storage bucket recovery Disaster Recovery process.

Related Links

• OCI Full Stack Disaster Recovery Product Page
• OCI Full Stack Disaster Recovery Documentation
• Full Stack Disaster Recovery Live Labs
• Full Stack Disaster Recovery for Cold VM Disaster Recovery
• Full Stack Disaster Recovery for WLS
• Full Stack Disaster Recovery- User defined group scripts
• Join #full-stack-dr public slack channel

Acknowledgments

Author - Suraj Ramesh (Full Stack Disaster Recovery Product Manager)

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.

---

Title and Copyright Information

Automate Switchover and Failover for OCI Object Storage buckets using Functions with OCI Full Stack Disaster Recovery

F83652-01

June 2023

Copyright © 2023, Oracle and/or its affiliates.