10.2.3 Back Out the Primary SDS NOAM

The following procedure details how to perform software back out for servers in the primary SDS NOAM NE.

Note:

The order of the back out for the primary NOAM NE and DR NOAM NE needs to be followed as shown in Table 3-10. See section Recovery Procedures Overview for more details before proceeding.

1. Access the primary SDS NOAM GUI, use the VIP address to access the primary SDS NOAM GUI as described in Access the OAM GUI Using the VIP (NOAM/SOAM).
2. In the Primary SDS NOAM VIP, expand Status & Manage click HA. Click Filter.

   Figure 10-10 Filter

   
   

   

   
   
3. In the Primary SDS NOAM VIP, locate the primary SDS NOAM NE. Using the information provided in section Logins, Passwords, and Site Information, select the primary SDS Network Element from the Scope field. Click Go.

   Figure 10-11 Scope

   
   

   

   
   
4. Identify each server by Host name, Server Role, and OAM HA Role and record the name of each server.

   Figure 10-12 Identify Servers

   
   

   

   
   

   Record the following information:

   • Active Primary SDS NOAM
   • Standby Primary SDS NOAM
   • Primary SDS Query Server (if equipped)
5. Downgrade Standby Primary SDS NOAM server (as identified and recorded in step 4 of this procedure) using Back Out a Single Server.
6. In the Primary SDS NOAM VIP (CLI), access the active primary SDS NOAM. Use the VIP address to log into the active primary SDS NOAM with the admusr account.

   sds-rlghnc-a login: admusr
   Password: <admusr_password>
   *** TRUNCATED OUTPUT ***
   RELEASE=6.4
   RUNID=00
   VPATH=/var/TKLC/rundb:/usr/TKLC/appworks:/usr/TKLC/awpcommon:/usr/TKLC/comagent-gui:/usr/TKLC/comagent-gui:/usr/TKLC/comagent:/usr/TKLC/sds
   PRODPATH=/opt/comcol/prod
   RUNID=00
   [admusr@sds-rlghnc-a ~]$
   

7. In the Primary SDS NOAM VIP, verify the DbReplication status is Active for the Standby Primary SDS NOAM and Query Server, if equipped.

   [admusr@sds-rlghnc-a ~]$ sudo irepstat -w
   -- Policy 0 ActStb [DbReplication]
   AA To  sds-rlghnc-b      Active  0  0.25 1%R 0.05%cpu 47B/s
   AA To  qs-rlghnc         Active  0  0.25 1%R 0.05%cpu 56B/s
   AA To  sds-mrsvnc-a      Active  0  0.50 1%R 0.04%cpu 47B/s
   AB To  kauai-sds-SO-b    Active  0  0.50 1%R 0.04%cpu 63B/s
   AB To  florence-sds-SO-a Active  0  0.51 1%R 0.03%cpu 65B/s
   AB To  turks-sds-SO-b    Active  0  0.50 1%R 0.04%cpu 65B/s
   irepstat (  8 lines) (h)elp
   

8. If a DbReplication status is Audit is received, then repeat the command until Active is returned.

   Note:

   Do not proceed until the status is Active. Check Replication is showing Active for Standby Primary SDS NOAM, Query Server, Active DR SDS NOAM and Standby DR SDS NOAM (if equipped).
9. Repeat the step until the status is Active for all the mentioned servers.

   Note:

   If a DbReplication status is received as Audit or some other value for these servers, repeat this step until a status of Active is returned. Servers are:

   • Standby Primary SDS NOAM
   • Query Server
   • Active DR SDS NOAM
   • Standby DR SDS NOAM
   Contact My Oracle Support for any assistance.
10. Exit the CLI for the ActivePrimary SDS NOAM.
    [admusr@sds-rlghnc-a filemgmt]$ exit
    logout

    Note:

    The next two steps of this procedure can be run in parallel.
11. Downgrade Primary Query server (as identified and recorded in step 4 of this procedure) using Back Out a Single Server.
12. Downgrade Active Primary SDS NOAM server (as identified and recorded in step 4 of this procedure) using Back Out a Single Server.

    Note:

    This causes an HA activity fail over to the mate primary SDS NOAM server. This occurs within a few minutes of initiating the upgrade.
13. Allow system to auto-clear temporary alarm states. Wait up to ten minutes for Alarms associated with server back out to auto clear.

    Note:

    If PDB Relay was recorded as Enabled in Back Out a Single Server, then Event 14189 (pdbRelay Time Lag) may persist for several hours post upgrade. This alarm can safely be ignored and automatically clears when the PDBI (HLRR) queue catches up with real-time replication.
14. Run Health Check procedures (Post back out) as specified in Health Check Procedures, if downgrade procedures have been completed for all required servers.