6 Migrate Remote Snapshots and Scheduled Backups

If you have remote snapshots of storage volumes in your Oracle Cloud Infrastructure Compute Classic account, or if you've used backup schedules to back up storage volumes in your Oracle Cloud Infrastructure Compute Classic account, then you can restore these backups as block volumes in your Oracle Cloud Infrastructure tenancy.

In Oracle Cloud Infrastructure Compute Classic, both scheduled backups as well as remote snapshots are stored in the associated Oracle Cloud Infrastructure Object Storage Classic account.

The migration of remote snapshots is important because storage volume backups stored in Oracle Cloud Infrastructure Object Storage Classic are encoded using a different format from objects stored in Oracle Cloud Infrastructure Object Storage. When you migrate object storage, the data stored in these snapshots is migrated over to Oracle Cloud Infrastructure. However, the encoding format used on the data isn't modified, so you can't directly access the data by restoring the snapshots to block volumes in Oracle Cloud Infrastructure. When you migrate remote snapshots, the encoding format is modified so that this data can be accessed by block volumes in Oracle Cloud Infrastructure.

After you restore your backups to block volumes in Oracle Cloud Infrastructure, you can attach these restored block volumes to VMs in Oracle Cloud Infrastructure, if required, or create backups of these volumes in Oracle Cloud Infrastructure Object Storage. Creating a backup of these migrated block volumes in Oracle Cloud Infrastructure allows you to complete the process of migrating your block volume backups from Oracle Cloud Infrastructure Object Storage Classic to Oracle Cloud Infrastructure Object Storage.

Note that, when you migrate storage backups, you can restore those backups only as data volumes. If you had a snapshot of a boot volume in Oracle Cloud Infrastructure Compute Classic, when you migrate that snapshot to Oracle Cloud Infrastructure, you can use that snapshot to create a data volume, but not to create a boot volume.

Workflow

Here's an overview of the high-level steps required to migrate your remote snapshots from Oracle Cloud Infrastructure Compute Classic to Oracle Cloud Infrastructure. You can use Oracle Cloud Infrastructure Classic Block Volume Backup and Restore Tool to automate this process.

  1. Create an instance in Oracle Cloud Infrastructure Compute Classic using the Oracle Cloud Infrastructure Classic Migration Tools image.
  2. Get the snapshot ids of the storage volume snapshots and the scheduled backups that you want to migrate.
    1. Run opcmigrate discover to get a list of resources in your source environment.
    2. Identify the remote snapshots that you want to migrate.

    Note:

    Alternatively, you can use the Oracle Cloud Infrastructure Compute Classic CLI to get a list of storage volume snapshots and to retrieve the snapshot ids. See Storage Volume Snapshot in opc compute CLI Reference for Oracle Compute Cloud Service (IaaS).
  3. Use Oracle Cloud Infrastructure Classic Block Volume Backup and Restore Tool to do the following:
    1. Set up the migration process.
    2. Restore each snapshot as a block volume in Oracle Cloud Infrastructure.
    3. Release each block volume, so that it can be attached to another VM.
  4. Attach each block volume to a VM in Oracle Cloud Infrastructure, if required, and validate that you can access data on the restored volumes.
  5. Create a backup of the block volumes, either manually or by using a backup policy.
  6. When you are done migrating your remote snapshots and scheduled backups, use Oracle Cloud Infrastructure Classic Block Volume Backup and Restore Tool to tear down the resources created for this migration.

Complete the Prerequisites

Before you begin your migration, complete the following prerequisites.

  • To use Oracle Cloud Infrastructure Classic Block Volume Backup and Restore Tool to migrate your storage volume backups, you must create an instance in your Oracle Cloud Infrastructure Compute Classic account using the Oracle Cloud Infrastructure Classic Migration Tools image. Ensure that you have sufficient quota for this instance to be created. For information about creating your Control-S instance, see Complete the Prerequisites and Launch the Migration Controller Instance (Control-S) in the Source Environment. If you've already created this instance earlier in your migration process, you can use the same instance for this procedure. You don't need to create it again.
  • The Oracle Cloud Infrastructure Classic Block Volume Backup and Restore Tool creates a VM in an isolated compartment in your Oracle Cloud Infrastructure tenancy to manage the migration. Ensure that you have sufficient quota for this VM to be launched.
  • The minimum size of a storage volume in Oracle Cloud Infrastructure is 50 GB. Ensure that you have sufficient storage capacity for the number of storage snapshots that you intend to migrate.
  • Ensure that you have a virtual cloud network (VCN) with subnets set up in a compartment that you have access to.

Plan for the Migration

To use Oracle Cloud Infrastructure Classic Block Volume Backup and Restore Tool to migrate your storage volume backups, you must configure the Control-S instance in your Oracle Cloud Infrastructure Compute Classic account. This instance must have already been created using the Oracle Cloud Infrastructure Classic Migration Tools image.

You should also get the snapshot ids of the remote snapshots and scheduled backups that you want to migrate. Do the following:

  1. Create or update the /home/opc/.opc/profiles/default file on Control-S. The opcmigrate migrate rsm commands used to migrate storage volume backups require access to the Oracle Cloud Infrastructure Object Storage Classic account. If you've used other opcmigrate commands earlier in your migration process, this section might already exist in your profile file. If not, add this section now, along with valid credentials.

    Note:

    If you run the Control-S setup command for Oracle Cloud Infrastructure Classic VM and Block Storage Migration Tool, it generates the default profile using information provided by you in the secret.yml file. The Control-S setup command overwrites any existing default profile. Verify and update the existing profile, or create a new profile, if required.
    1. You'll need the user name and API end point for the Oracle Cloud Infrastructure Object Storage Classicservice. Look this up in your Oracle Cloud Dashboard.
    2. Create or locate your profile file. This file contains the credentials and other information required to access your source environment. The default location for this file is ~/.opc/profiles/default. If required, you can create multiple profiles and use the --profile option to specify the profile you want to use when you run the tool. If you create profiles in a location other than ~/.opc/profiles, provide the full path to the profile location by using the --profile-directory option, along with the --profile option to specify the profile file name. If no profile is specified, the ~/.opc/profiles/default profile is used.
    3. Use the following template to add the "object-storage" section to your profile file. Replace the sample values with values specific to your account.
       "object_storage": {
          "auth-endpoint": "uscom-central-1.storage.oraclecloud.com/auth/v1.0",
          "user": "Storage-example:user@example.com",
          "endpoint": "uscom-central-1.storage.oraclecloud.com/v1/Storage-example"
        }

      Passwords aren't specified in the profile file for security reasons. You'll be prompted to provide the password for each service when you run the tool.

  2. Get details of your target environment. Log in to the Oracle Cloud Infrastructure Console and collect the required information. You'll need the following:
    • The user OCID. From the menu, choose Identity and then choose Users.
    • The API PEM key fingerprint. Click the user to view user details. The API Keys section displays the PEM key fingerprint.
    • The compartment OCID. From the menu, choose Identity and then choose Compartments. The VM created by the tool to manage the migration of storage volume backups is created in a compartment under this compartment.
    • The tenancy OCID and the tenancy region. From the menu, choose Administration and then choose Tenancy Details.
    • The Availability Domain. From the menu, choose Compute and then choose Instances. The Instances page displays the availability domain. If you want to attach the restored volumes to existing VMs, ensure that the availability domain that you select has the VMs that you want to use. You can't attach block volumes to VMs in a different availability domain.
  3. Create or edit the /home/opc/.oci/config file on Control-S. This file contains the credentials and other information required to access your target environment. If you've used Oracle Cloud Infrastructure Classic VM and Block Storage Migration Tool earlier in the migration process, this would already have been created. However, the file might not contain all of the information required by the opcmigrate migrate rsm commands. For example, you might have to add information about the availability domain.

    If you run the Control-S setup command for Oracle Cloud Infrastructure Classic VM and Block Storage Migration Tool, it generates the .oci/config file, using information provided by you in the secret.yml file. The Control-S setup command overwrites any existing .oci/config file.

    Use the following template to create or update the .oci/config file. Replace the sample values with values specific to your tenancy.
    [DEFAULT]
    user=ocid1.user.oc1..aaaaa...
    fingerprint=81:45:aa:2...
    key_file=<path to api pem key>
    pass_phrase=
    tenancy=ocid1.tenancy.oc1..aaaaaaa...
    region=us-ashburn-1

    The compartment OCID and availability domain aren't specified in this file. However, make a note of those values. You'll need them later in this migration procedure.

  4. Generate a list of resources in your source environment. If you've already done this earlier in your migration process, you don't need to do this again. However, if your environment has changed since the last time you ran this command, then run the command again to get the latest set of resources. On the Control-S instance, run:
    opcmigrate discover

    You can specify a profile or a profile directory with this command. The output of this command is a file named resources-*.json, where * represents the profile used with this command. This file created in the directory where you run the command.

  5. You can use other opcmigrate commands to filter the output of opcmigrate discover or to view the output in various formats. Use the --help option for help with opcmigrate commands and options. Select a method that works best for you, to identify the remote snapshots and scheduled backups in your source environment. For example, you can search for objects with the following property:
    "property": "/oracle/public/storage/snapshot/default",

    To find out whether a remote snapshot was generated manually or by a scheduled backup, look for the following tags. If there are no tags, no backup schedule is associated with the snapshot.

    "tags": [
            "backupPrivateId=a0f1....",
            "backupConfigurationPrivateId=4a...."
          ],
    For each of the snapshots in the resources file that you want to migrate, make a note of the snapshot id. For example:
    "snapshot_id": "4465........751e-uscom",

Migrate the Specified Storage Volume Backups

Use Oracle Cloud Infrastructure Classic Block Volume Backup and Restore Tool to restore remote snapshots and scheduled backups from Oracle Cloud Infrastructure Compute Classicto block volumes in Oracle Cloud Infrastructure.

You can run this tool on an instance in Oracle Cloud Infrastructure Compute Classic that was created using the Oracle Cloud Infrastructure Classic Migration Tools image.

Note:

If you download and install Oracle Cloud Infrastructure Classic Discovery and Translation Tool on your local system, you won't be able to use Oracle Cloud Infrastructure Classic Block Volume Backup and Restore Tool because this tool isn't included in that distribution of the migration tools. To use Oracle Cloud Infrastructure Classic Block Volume Backup and Restore Tool, you must create an instance using the Oracle Cloud Infrastructure Classic Migration Tools image.
  1. To set up the migration controller instance in the target environment, run:
    opcmigrate migrate rsm setup <compartment_id> <availability_domain>

    This command launches a VM in your Oracle Cloud Infrastructure tenancy, in the specified availability domain. This VM is launched in an isolated subcompartment named rsm, which is created in the specified compartment.

    If you want to migrate backups to different availability domains, you can set up migration controller VMs in other availability domains by running this command again with a different availability domain specified. That way, you can restore backups to different availability domains concurrently.

    For a given availability domain, it is recommended that you run the setup command only once.

    Note that, if you want to attach the restored volumes to existing VMs, those VMs must be in the same availability domain that you specify for restoring volumes. You can't attach block volumes to VMs in a different availability domain.

  2. The opcmigrate migrate rsm setup command returns a stack OCID. Make a note of this value. You'll need this later, when you want to tear down this migration environment.
  3. To restore a storage volume from a snapshot to a block volume in Oracle Cloud Infrastructure, run:
    opcmigrate migrate rsm restore <snapshot_id> <name> <compartment_id> <availability_domain>

    Here, <snapshot_id> is the value of snapshot _id that you obtained from the resources file. You must provide the snapshot ID for each backup that you want to migrate. You can provide only one snapshot ID at a time. To migrate multiple backups, run this command multiple times.

    <name> is the name of the block volume that will be created in your Oracle Cloud Infrastructure tenancy.

    You should have already run the setup command for the same availability domain that you specify here.

    This command creates a target volume with the specified name. The size of the target volume is the same as the size of the backup in the source environment. However, note that the minimum size of a block volume in Oracle Cloud Infrastructure is 50 GB, so even when you migrate smaller backups, the target volume is created with a size of 50 GB.

    The target volume is attached to the migration controller instance in the specified availability domain in Oracle Cloud Infrastructure.

    It is recommended that you run a single restore job at a time in a specified availability domain. Wait for each job to complete before you launch another job in the same availability domain.

  4. The opcmigrate migrate rsm restore command returns a job ID and a volume OCID. Make a note of both values. You'll need the job ID later to monitor the status of the job as well as to release the volume. You can use the volume OCID to attach a restored volume to a VM later on, if required.
  5. To view the status of a migration job, run:
    opcmigrate migrate rsm status <job_id>

    Here, job_id is the value returned by the opcmigrate migrate rsm restore command.

  6. When the restore step completes, you must release the block volume to detach it from the migration controller instance. You can't attach a block volume to another VM until it has been released. To release a block volume, run:
    opcmigrate migrate rsm release <job_id>
  7. For each remote snapshot or scheduled backup that you want to migrate, run the commands opcmigrate migrate rsm restore and opcmigrate migrate rsm release until all backups have been successfully migrated.
  8. Validate the migrated volumes. After each restored volume has been released, attach it to a VM and use the iSCSI commands to mount it on the VM. Verify the data on the volume. If required, create a backup of each volume in your Oracle Cloud Infrastructure tenancy.
  9. When all backups have been successfully migrated and verified, tear down the migration environment. This destroys resources created for the migration including the migration controller VM and releases any migrated volumes created by the specified stack id that haven't yet been released. To tear down the environment, run:
    opcmigrate migrate rsm destroy <stack_id>

    Here, stack_id is the value of the stack OCID returned by the opcmigrate migrate rsm setup command.

  10. The subcompartment rsm that is created for this migration isn't deleted by the opcmigrate migrate rsm destroy command. You can delete this subcompartment by using the Oracle Cloud Infrastructure Console, if required.

Troubleshooting

Here are a few tips for dealing with errors that might occur while migrating remote snapshots and scheduled backups using Oracle Cloud Infrastructure Classic Block Volume Backup and Restore Tool.

  • The opcmigrate migrate rsm setup command errors out.

    Errors in the setup could be due to resource limits in your Oracle Cloud Infrastructure tenancy. Ensure that you have sufficient quota in the specified availability domain to create the migration controller VM and the block volumes that you want to migrate. If the setup command returns an Apply Job OCID, use that value to search in the Oracle Cloud Infrastructure Console for the components that caused the error.

  • The setup command returns the following error: Could not find config file at <home_dir>/.oci/config

    Ensure that you've created the .oci/config file in the appropriate path, or use the -c option to specify a different location for this file.

  • The opcmigrate migrate rsm restore command fails with the following error: Could not retrieve snapshot metadata

    Ensure that the backup corresponding to the specified snapshot ID exists and that it is a remote snapshot or a scheduled backup. This error could indicate that the specified snapshot ID belongs to a colocated snapshot. This tool can't be used to migrate colocated snapshots.

  • The restore command fails to connect to the migration controller VM in Oracle Cloud Infrastructure. Error: No valid service VM found

    Ensure that you've run the setup command to launch the migration controller VM and that the migration environment hasn't subsequently been destroyed. Also check that the migration controller VM is in the same availability domain as the availability domain that you specified for the migration job.

  • The restored volume can't be attached to a VM in Oracle Cloud Infrastructure.

    Ensure that you've run opcmigrate migrate rsm release to release the volume from the migration controller VM. Only after releasing a volume can you attach it to another VM.

  • The rsm subcompartment still exists after cleaning up the target environment using opcmigrate migrate rsm destroy.

    The rsm compartment that is created by opcmigrate migrate rsm setup isn't destroyed by opcmigrate migrate rsm destroy. Verify that all the other resources have been removed by the destroy command, and the delete the rsm compartment manually by using the Oracle Cloud Infrastructure Console.