5 Backup and Recovery of the Exalogic Control Repository and Stack

This chapter provides the steps for backing up and recovering the Exalogic Control repository and stack. It contains the following sections:

5.1 Recovering After Hardware Failure

If the compute node on which a vServer is running crashes, recover the compute node to its previous state, log in to the Exalogic Control BUI, and start the vServer.

If the compute node on which a vServer is running needs to be replaced, follow the steps in Section 3.2.2, "Reimaging and Bare Metal Restore" to recover the compute node. After the compute node is restored to its previous state, log in to the Exalogic Control BUI, and start the vServer.

5.2 Backing Up the Exalogic Control Repository

The Exalogic Control repository contains virtual-machine disk images, templates, and virtual-machine configuration files for all the virtual machines running on the Exalogic machine. The Exalogic Control repository resides in the ZFS storage appliance in the Exalogic machine. It is recommended that you use ZFS snapshots to back up the Exalogic Control repository.

To ensure data consistency, the virtual machines running the Exalogic Control stack must be shut down before taking a ZFS snapshot. Note that the customer virtual machines do not need to be stopped as long as all the applications are running off an NFS mount.

To back up the Exalogic Control repository, do the following:

  1. Stop the components of the Exalogic Control stack as described in Section 5.2.1.

  2. Shut down the Exalogic Control virtual machines as described in Section 5.2.2.

  3. Create a ZFS snapshot of the Exalogic Control repository as described in Section 5.2.3.

  4. Restart the components of the Exalogic Control stack as described in Section 5.2.5.

5.2.1 Stopping the Components of the Exalogic Control Stack

To stop the components of the Exalogic Control stack, do the following:

  1. Log in to the Proxy Controller-2 virtual machine as the root user.

  2. Stop the Proxy Controller process.

    # /opt/sun/xvmoc/bin/proxyadm stop -w
    
  3. Log in to the Proxy Controller-1 virtual machine as the root user.

  4. Stop the Proxy Controller process.

    # /opt/sun/xvmoc/bin/proxyadm stop -w
    
  5. Log in to the Enterprise Controller virtual machine as the root user.

  6. Stop the Enterprise Controller process.

    # /opt/sun/xvmoc/bin/ecadm stop -w
    
  7. Log in to the Oracle VM Manager virtual machine as the root user.

  8. Stop the Oracle VM Manager process.

    # service ovmm stop
    
  9. Log in to the database virtual machine as the root user.

  10. Stop the database process.

    # service oracle-db stop
    

5.2.2 Shutting Down the Exalogic Control Virtual Machines

Gracefully shut down all Exalogic Control virtual machines by doing the following:

  1. Log in to the Proxy Controller-2 virtual machine as the root user.

  2. Shut down the virtual machine, by running the following command:

    # shutdown -h now
    
  3. Log in to the Proxy Controller-1 virtual machine as the root user.

  4. Shut down the virtual machine, by running the following command:

    # shutdown -h now
    
  5. Log in to the Enterprise Controller virtual machine as the root user.

  6. Shut down the virtual machine, by running the following command:

    # shutdown -h now
    
  7. Log in to the Oracle VM Manager virtual machine as the root user.

  8. Shut down the virtual machine, by running the following command:

    # shutdown -h now
    
  9. Log in to the database virtual machine as the root user.

  10. Shut down the virtual machine, by running the following command:

    # shutdown -h now
    

5.2.3 Creating a ZFS Snapshot of the Exalogic Control repository

The snapshot created in this section can be used in the following restoration scenarios:

If you want to use the snapshot to restore the Exalogic Control repository, shutdown customer vServers before creating the snapshot.

To create a ZFS storage appliance snapshot of the Exalogic Control repository, do the following:

  1. Log in to the BUI of the ZFS storage appliance at http://storage_host:215 as the root user.

  2. Navigate to Shares and then to Projects, and select the ExalogicControl project.

  3. Under the ExalogicControl project, select the ExalogicRepo share.

  4. Navigate to Snapshots, and click + to create a snapshot.

  5. In the Create Snapshot dialog box, provide a name, and click Apply.

5.2.4 Creating a Full Backup of the Exalogic Control Artifacts

It is recommended that you create a full backup of the Exalogic Control artifacts regularly.

  1. Stop the components of the Exalogic Control stack as described in Section 5.2.1.

  2. Shut down the Exalogic Control virtual machines as described in Section 5.2.2.

  3. Mount the NFS location defined in Chapter 2, "Backup and Recovery Locations" on one of the compute nodes.

  4. Go to the /OVS/Repositories/*/VirtualMachines directory.

  5. Find the virtual-machine configuration files for the Exalogic Control virtual machines, as shown in the following example:

    grep -i ExalogicControl */vm.cfg
    0004fb000006000038a23e7d5c307e00/vm.cfg:OVM_simple_name =
     'ExalogicControlDB'
    0004fb000006000056cafd150ebc34c0/vm.cfg:OVM_simple_name =
     'ExalogicControlOpsCenterPC2'
    0004fb00000600009c3bf065d07004cb/vm.cfg:OVM_simple_name =
     'ExalogicControlOVMM'
    0004fb0000060000a5143f2ef7d264ac/vm.cfg:OVM_simple_name =
     'ExalogicControlOpsCenterPC1'
    0004fb0000060000f0ccd3391c56a578/vm.cfg:OVM_simple_name =
     'ExalogicControlOpsCenterEC1'
    
  6. Identify the location and name of the virtual-disk image file for each component of the Exalogic Control stack.

    You can identify the location and name of the virtual-disk image file from the disk parameter in the vm.cfg file, as shown in the following example:

    cat 0004fb000006000056cafd150ebc34c0/vm.cfg | grep file
    disk = ['file:/OVS/Repositories/0004fb00000300007d5117500d92ae54/VirtualDisks/0004fb00001200003d3b8b4058ee7682.img,hda,w'
    
  7. Create a tar file with the vm.cfg from step 5 and the virtual-disk image file from step 6, as shown in the following example:

    tar -czvf pc2_oct24.tgz /OVS/Repositories/0004fb00000300007d5117500d92ae54/VirtualDisks/0004fb00001200003d3b8b4058ee7682.img /OVS/Repositories/0004fb00000300007d5117500d92ae54/VirtualMachines/0004fb000006000056cafd150ebc34c0/vm.cfg
    
  8. Copy the tar file to the backup location that you mounted in step 3.

  9. Start the components of the Exalogic Control stack as described in Section 5.2.5.

5.2.5 Starting the Exalogic Control Stack

To start the Exalogic Control stack, do the following:

Note:

If the Exalogic Control virtual machines are not up, follow Step 4: Start the Enterprise Manager Ops Center Control VMs in Section 5.4.

  1. Log in to the database virtual machine as the root user.

  2. Start the database process.

    # service oracle-db start
    
  3. Log in to the Oracle VM Manager virtual machine as the root user.

  4. Start the Oracle VM Manager process.

    # service ovmm start
    
  5. Log in to the Enterprise Controller virtual machine as the root user.

  6. Start the Enterprise Controller process.

    # /opt/sun/xvmoc/bin/ecadm start -w
    
  7. Log in to the Proxy Controller-1 virtual machine as the root user.

  8. Start the Proxy Controller process.

    # /opt/sun/xvmoc/bin/proxyadm start -w
    
  9. Log in to the Proxy Controller-2 virtual machine as the root user.

  10. Start the Proxy Controller process.

    # /opt/sun/xvmoc/bin/proxyadm start -w
    

5.3 Restoring the Exalogic Control Repository

The entire Exalogic Control repository can be recovered to the point in time when the last good ZFS snapshot was taken.

To restore the Exalogic Control repository, do the following:

Caution:

Rolling back to a snapshot reverts all the active data, including customer vServers, to the point in time when the snapshot was created. Any vServers that were created after the snapshot was taken, will no longer be present. Also, if any vServers associated with EoIB networks were deleted after the snapshot was taken, then, when such vServers are restored from the snapshot, the VNICs associated with the vServers will not be re-created

In addition, any recent snapshots and clones will be destroyed. This operation cannot be reversed.

  1. Stop the components of the Exalogic Control stack as described in Section 5.2.1, "Stopping the Components of the Exalogic Control Stack."

  2. Restore the Exalogic Control repository from a ZFS snapshot as follows:

    1. Log in to the BUI of the ZFS storage appliance at http://storage_host:215 as the root user.

    2. Navigate to Shares and then to Projects, and select the ExalogicControl project.

    3. Under the ExalogicControl project, select the ExalogicRepo share.

    4. Navigate to Snapshots, select the required snapshot, and click Rollback data to this snapshot.

    5. In the resulting dialog box, click OK.

  3. Start the components of the Exalogic Control stack as described in Section 5.2.5, "Starting the Exalogic Control Stack."

5.4 Restoring the Exalogic Control Stack

To ensure data consistency, all the Exalogic Control virtual machines must be restored together. The components of the Exalogic Control stack can be recovered from either the ZFS snapshot or the latest full backup.

5.4.1 Restoring the Exalogic Control Stack from a ZFS Snapshot

To restore the Exalogic Control stack from a ZFS snapshot, do the following:

  1. Stop the Exalogic Control stack as described in Section 5.2.1, "Stopping the Components of the Exalogic Control Stack."

  2. Log in to the BUI of the ZFS storage appliance at http://storage_host:215 as the root user.

  3. Navigate to Shares and then to Projects, and select the ExalogicControl project.

  4. Under the ExalogicControl project, select the ExalogicRepo share.

  5. Navigate to Snapshots, select the required snapshot, and clone the snapshot.

  6. Select the location of the Backup project, as defined in Chapter 2, "Backup and Recovery Locations."

  7. Enter a name for the clone (example: RepoClone_date).

  8. Note the mount point for the clone.

  9. Mount the clone on one of the compute nodes, as shown in the following example:

    mount -t storage_host:/export/Exalogic_Backup/RepoClone_Oct24 /backup
    
  10. Go to the /backup/VirtualMachines directory.

  11. Find the virtual-machine configuration files for the Exalogic Control virtual machines, as shown in the following example:

    grep -i ExalogicControl /backup/VirtualMachines/*/vm.cfg
    0004fb000006000038a23e7d5c307e00/vm.cfg:OVM_simple_name =
     'ExalogicControlDB'
    0004fb000006000056cafd150ebc34c0/vm.cfg:OVM_simple_name =
     'ExalogicControlOpsCenterPC2'
    0004fb00000600009c3bf065d07004cb/vm.cfg:OVM_simple_name =
     'ExalogicControlOVMM'
    0004fb0000060000a5143f2ef7d264ac/vm.cfg:OVM_simple_name =
     'ExalogicControlOpsCenterPC1'
    0004fb0000060000f0ccd3391c56a578/vm.cfg:OVM_simple_name =
     'ExalogicControlOpsCenterEC1'
    
  12. Identify the location and name of the virtual-disk image file for each component of the Exalogic Control stack.

    You can identify the location and name of the virtual-disk image file from the disk parameter in the vm.cfg files, as shown in the following example:

    cat 0004fb000006000056cafd150ebc34c0/vm.cfg | grep file
    disk = ['file:/OVS/Repositories/0004fb00000300007d5117500d92ae54/VirtualDisks/0004fb00001200003d3b8b4058ee7682.img,hda,w'
    
  13. Copy the vm.cfg files identified in step 11 to the /OVS/Repositories/*/VirtualMachines/vmGUID directory.

    Note that vmGUID is name of the directory in which the vm.cfg file is located. For example, step 11 shows the following as the location of the ExalogicControlDB virtual machine:

    0004fb000006000038a23e7d5c307e00/vm.cfg:OVM_simple_name = 'ExalogicControlDB'
    

    In this example, 0004fb000006000038a23e7d5c307e00 is the vmGUID.

  14. Go to the /backup/VirtualDisks directory, and copy the virtual-disk image file identified in step 12 to the location identified by the disk parameter in the vm.cfg file, as shown in the following example:

    cp /backup/VirtualDisks/0004fb00001200003d3b8b4058ee7682.img /OVS/Repositories/0004fb00000300007d5117500d92ae54/VirtualDisks/0004fb00001200003d3b8b4058ee7682.img
    
  15. Repeat steps 12 to 14 to restore each component of the Exalogic Control stack.

5.4.2 Restoring the Exalogic Control Stack from a Full Backup

To restore the Exalogic Control stack from a full backup, do the following:

  1. Stop the Exalogic Control stack as described in Section 5.2.1, "Stopping the Components of the Exalogic Control Stack."

  2. Mount the backup location as defined in Section 5.2.4, "Creating a Full Backup of the Exalogic Control Artifacts."

  3. Identify the backup file from which you want to restore the Exalogic Control stack.

  4. Untar the backup files for each of the Exalogic Control components to a directory of your choice.

  5. Copy the vm.cfg files identified earlier to the /OVS/Repositories/*/VirtualMachines/vmGUID directory.

    Note that vmGUID is name of the directory in which the vm.cfg file is located. For example, the following was the previously identified location of the ExalogicControlDB virtual machine:

    0004fb000006000038a23e7d5c307e00/vm.cfg:OVM_simple_name = 'ExalogicControlDB'
    

    In this example, 0004fb000006000038a23e7d5c307e00 is the vmGUID.

  6. Copy the virtual-disk image file identified earlier to the location indicated by the disk parameter in the vm.cfg file

  7. Start the Exalogic Control virtual machines as described in Section 4.1, "Recovering from a Hardware Failure" and the Exalogic Control processes as detailed in Step 4: Start the Enterprise Manager Ops Center Control VMs.