24 5.1.6 Recovery Scenario 6 (Database Recovery)
5.1.6.1 Recovery Scenario 6: Case 1
For a partial outage with
Cloud Disaster Recovery Guide
- Server having a corrupted database.
- Replication channel from parent is inhibited because of upgrade activity.
- Server is in a different release then that of its Active parent because of upgrade activity.
- Verify that the Server Runtime backup files, performed at the start of the
upgrade, are present in /var/TKLC/db/filemgmt area in the following format
- Backup.DSR.HPC02-NO2.FullDBParts.NETWORK_OAMP.20140524_223507.UPG.tar.bz2
- Backup.DSR.HPC02-NO2.FullRunEnv.NETWORK_OAMP.20140524_223507.UPG.tar.bz2
Note:
During recovery, the corrupted Database will get replaced by the sever Runtime backup. Any configuration done after taking the backup will not be visible post recovery.This procedure performs recovery if database is corrupted in the system.
Check off () each step as it is completed. Boxes have been provided for this purpose under each step number.
If this procedure fails, contact My Oracle Support (MOS), and ask for assistance.
Table 24-1 Procedure 10
| STEP # | Procedure | Description |
|---|---|---|
| 1 | Workarounds |
Refer to Workarounds for Issues not fixed in this Release to understand/apply any workarounds required during this procedure. |
| 2 | NOAM VIP GUI: Set Failed Servers to OOS |
Navigate to Main Menu -> Status & Manage -> HA Figure 24-1 HA  Select Edit Set the Max Allowed HA Role drop down box to OOS for the failed servers. Select Ok Figure 24-2 Cancel  |
| 3 | Server in Question: Login |
Establish an SSH session to the server in question. Login as admusr user. |
| 4 | Server in Question: Change runlevel to 3 |
Execute the following command to bring the system to runlevel 3: Figure 24-3 init3  |
| 5 | Server in Question: Recover System |
Execute the following command and follow the instructions
appearing the console prompt:
Figure 24-4 Backout restore  |
| 6 | Server in Question: Change runlevel to 4 |
Execute the following command to bring the system back to runlevel 4: Figure 24-5 Init 6  |
| 7 | Server in Question: Verify the server |
Execute the following command to verify if the processes are up and running: Figure 24-6 getprocs  |
| 8 | NOAM VIP GUI: Set Failed Servers to Active |
Figure 24-7 HA Click on Edit at the bottom of the screen. For each failed server whose Max Allowed HA Role is set to OOS, set it to Active. Press OK. |
| 9 | Backup and Archive All the Databases from the Recovered System |
Execute DSR Database Backup to back up the Configuration databases: |
5.1.6.2 Recovery Scenario 6: Case 2
For a partial outage with
- Server having a corrupted database.
- Replication channel is not inhibited.
- Server has the same release as that of its Active parent.
This procedure performs recovery if database got corrupted in the system and system is in the state to get replicated
Check off () each step as it is completed. Boxes have been provided for this purpose under each step number.
If this procedure fails, contact My Oracle Support (MOS), and ask for assistance.
Table 24-2 Procedure 11
| STEP # | Procedure | Description |
|---|---|---|
| 1 | Workarounds |
Refer to Release to understand/apply any workarounds required during this procedure. |
| 2 | NOAM VIP GUI: Set Failed Servers to OOS |
Navigate to Main Menu -> Status & Manage -> HA Figure 24-8 HA  Select Edit Set the Max Allowed HA Role drop down box to OOS for the failed servers. Select Ok |
| 3 | Server in Question: Login |
Establish an SSH session to the server in question. Login as admusr user. |
| 4 | Server in Question: Take Server out of Service |
Execute the following command to take the server out of service. Figure 24-9 Clobber  |
| 5 | Server in Question: Take Server to DbUp State and Start the Application |
Execute the following commands to take the server to Dbup and start the DSR application: Figure 24-10 sudo-prod-start  |
| 6 | Server in Question: Verify the Server State |
Execute the following commands to verify the processes are up and running: Figure 24-11 get procs Execute the following command to verify if replication channels are up and running: Figure 24-12 irep stat  Execute the following command to verify if merging channels are up and running: Figure 24-13 inetmstat  |
| 7 | NOAM VIP GUI: Restart DSR application |
Navigate to Main Menu->Status & Manage->Server, Figure 24-14 Server  Select each recovered server and click on Restart. Figure 24-15 Restart  |
| 8 | NOAM VIP GUI: Set Failed Servers to Active |
Figure 24-16 HA Click on Edit at the bottom of the screen For each failed server whose Max Allowed HA Role is set to OOS, set it to Active Press OK |
| 9 | Backup and Archive All the Databases from the Recovered System |
Execute DSR Database Backup to back up the Configuration databases: |