Manually Backing Up and Restoring Cloud Manager Using Block Volume Backups for OCI

Using block volume backup feature in OCI, the Cloud Manager data can be backed up and restored on demand.

To back up the Cloud Manager instance for OCI using block volumes, perform the following:

  1. To create a consistent backup, shut down the database, application server, web server (PIA), and Process Scheduler domains.

    Note: Ensure that there are no provisioning or lifecycle jobs running. If there are any such jobs, they will be abruptly ended and may result in environments in an unstable or unusable state.

  2. Access the Cloud Manager instance with SSH and run the following commands or use the psadmin utility.

     $ sudo systemctl stop psft-pia-peoplesoft.service
     $ sudo systemctl stop psft-prcs-PRCSDOM.service
     $ sudo systemctl stop psft-appserver-APPDOM.service
     $ sudo systemctl stop psft-db-<DBName>.service
  3. Back up the set of files on boot volume that are listed below on to local file system on CM instance or any remote instance.

    Note: Use the environment variable PS_CFG_HOME to determine the exact path. Make a note of this in case the path gets modified during PeopleTools upgrade.

    • All files under PS_APP_HOME/cloud (/opt/oracle/psft/pt/ps_app_home/cloud)

    • All files under PS_CFG_HOME

    • /home/psadm1/.bashrc

    • /home/psadm2/.bashrc

    • /home/psadm3/.bashrc

    • /home/oracle2/.bashrc

    • /home/esadm1/.bashrc

    • /etc/profile

    • /etc/bashrc

    • /usr/lib/systemd/system/psft-db-<DB_NAME>.service

    • /usr/lib/systemd/system/psft-appserver-APPDOM.service

    • /usr/lib/systemd/system/psft-prcs-PRCSDOM.service

    • /usr/lib/systemd/system/psft-pia-peoplesoft.service

  4. On the OCI console, navigate to Compute | Instances | Cloud Manager instance.

  5. Navigate to Cloud Manager Instance Details page.

  6. Scroll down to the Attached Block Volumes section. Click on the attached volume name which will have a name in the format StorageVol_<CMinstance>_<timestamp>. This volume is available as disk /dev/sdb in Cloud Manager instance. It is mounted on /u01/app/oracle/product, where Cloud Manager application is installed.

  7. This will bring up the volume details. On this page, click on 'Create Backup'.

  8. Optionally, create a backup of the boot volume in similar way.

  9. Provide a name for the backup and click 'Create Backup'.

  10. After few minutes a backup is created.

  11. Start the database, pia, app and prcs domains. Use below commands or psadmin utility.

    $ sudo systemctl start psft-db-<DBName>.service
    $ sudo systemctl start psft-appserver-APPDOM.service
    $ sudo systemctl start psft-prcs-PRCSDOM.service
    $ sudo systemctl start psft-pia-peoplesoft.service
    

To restore a backup using block volumes, perform the following:

  1. If restoring to an older version of Cloud Manager from a newer version, then generate the list of IP address of all managed instances that were provisioned by Cloud Manager. Follow step 1 in Restoring from a Backup.

  2. On the OCI console, navigate to Storage | Backups.

  3. Select the backup to restore and click 'Create Block Volume' using menu on the right.

  4. Enter a name for the block volume and choose the Availability Domain in which the volume will be created. Ensure to choose the same Availability Domain where Cloud Manager instance is deployed.

  5. A new volume is created in few seconds.

  6. Access the Cloud Manager instance with SSH and shut down database, pia, app and prcs domains using commands below or psadmin utility.

     $ sudo systemctl stop psft-pia-peoplesoft.service
     $ sudo systemctl stop psft-prcs-PRCSDOM.service
     $ sudo systemctl stop psft-appserver-APPDOM.service
     $ sudo systemctl stop psft-db-<DBName>.service
  7. Clean up any running processes that might be using the data volume that needs to be restored.

     $ ps -ef | grep psadm
     psadm2 2969 1 0 Feb01 ? 00:00:19 rmiregistry 10100
     psadm2 3495 1 0 Feb01 ? 00:00:20 rmiregistry 10200 
     $ sudo kill 2969 3495 
  8. Unmount /dev/sdb which is mounted on /opt/oracle/psft.

     $ sudo umount /opt/oracle/psft 
  9. Navigate to OCI | Compute | Instances | Cloud Manager instance. Scroll down to the Attached Block Volumes. Select the volume to be restored and click Detach.

  10. On the Detach Block Volume page, copy all DETACH COMMANDS.

  11. Run the detach commands on the Cloud Manager instance. 

  12. Click 'Continue Detachment' (from step 9) and confirm detachment.

  13. Verify in OCI UI for the instance that the volume is now removed.

  14. Now restore the volume backup. Click Attach Block Volume. Select ISCSI attachment type. Select the block volume compartment where the backup volume was restored and select the restored volume. Select read-write access mode.

  15. Click Attach to attach the restored volume to Cloud Manager instance. 

  16. After the status shows Attached. Retrieve the iSCSI commands that must be run on the instance to attach the volume in the OS. Click the Actions icon (Actions icon) next to the volume, and then click iSCSI Commands and Information. Copy all ATTACH COMMANDS.

  17. Access the Cloud Manager instance with SSH and run the copied attach commands.

  18. Verify the disk is attached using "sudo fdisk –l" command. There should now be an entry for /dev/sdb.

     Disk /dev/sdb: 107.4 GB, 107374182400 bytes
     255 heads, 63 sectors/track, 13054 cylinders
     Units = cylinders of 16065 * 512 = 8225280 bytes
     Sector size (logical/physical): 512 bytes / 4096 bytes
     I/O size (minimum/optimal): 4096 bytes / 4096 bytes
     Disk identifier: 0x00000000 
  19. Restore below set of files that were backed up from boot volume. Ensure to restore the PS_CFG_HOME files to the right path in case the backup contains files from an older PeopleTools release.

    • Restore /cm_psft_dpks/cloud/ from backup of PS_APP_HOME/cloud

    • All files under PS_CFG_HOME

    • /home/psadm1/.bashrc

    • /home/psadm2/.bashrc

    • /home/psadm3/.bashrc

    • /home/oracle2/.bashrc

    • /home/esadm1/.bashrc

    • /etc/profile

    • /etc/bashrc

    • /usr/lib/systemd/system/psft-db-<DB_NAME>.service

    • /usr/lib/systemd/system/psft-appserver-APPDOM.service

    • /usr/lib/systemd/system/psft-prcs-PRCSDOM.service

    • /usr/lib/systemd/system/psft-pia-peoplesoft.service

  20. Run 'mount -a' command on the CM instance and reboot the instance. Check status of Cloud Manager domains using following commands.

     $ sudo systemctl status psft-db-<DBName>.service
     PeopleSoft Container Database CDBHCM Status is Up
     PeopleSoft Pluggable Database PSPDB Status is Open
     PeopleSoft Database Listener is Up
     $ sudo systemctl status psft-prcs-PRCSDOM.service
     PeopleSoft Process Scheduler Domain PRCSDOM is Up
     $ sudo systemctl status psft-appserver-APPDOM.service
     PeopleSoft Application Server Domain APPDOM is Up
     $ sudo systemctl status psft-pia-peoplesoft.service
     PeopleSoft PIA Domain peoplesoft is Up 

    If database and domains do not come up automatically then start them using the following commands. Reboot only if necessary.

    $ sudo systemctl start psft-db-<DBName>.service
    $ sudo systemctl start psft-appserver-APPDOM.service
    $ sudo systemctl start psft-prcs-PRCSDOM.service
    $ sudo systemctl start psft-pia-peoplesoft.service
    

    If the database or domains don't start successfully, then the restored backup may have issues, In such scenario, there are two options at this point:

    1. Restore the original volume. Follow steps 5 to 18 described under 'How to restore a backup' section.

    2. Troubleshoot the reason for failures and bring up the database or domains manually.

  21. SSH into Cloud Manager and remove the directory /home/psadm2/psft/data/cloud/dm/cache/.

  22. Before accessing the restored Cloud Manager PIA URL, you need to clear the application domain cache. To clear cache:

    1. SSH into Cloud Manager instance.

    2. Switch user to psadm2.

      sudo su - psadm2
    3. Start psadmin.

    4. Select 1) Application Server.

    5. Select 1) Administer a domain.

    6. Select 1) APPDOM.

    7. Select 8) Purge Cache.

  23. If restoring to an older version from a newer version of Cloud Manager, copy the restored /cm_psft_dpks/cloud to all the managed instances. Follow step 10 in Restoring from a Backup.

Note: If you want specific downloads to begin, unsubscribe and then subscribe to the required download channels. Otherwise, the downloads will begin at the next scheduled time.