Upgrade Single Server

A.3 Upgrade Single Server – DSR 8.x

The following procedures upgrade a single DSR server of any type (For example: NOAM, SOAM, MP) when the active NOAM is on DSR 8.x.

Note:

This procedure may be executed multiple times during the overall upgrade, depending on the number of servers in the DSR and the chosen upgrade methodology. Make multiple copies of this procedure to mark up, or keep another form of written record of the steps performed.

View the preupgrade status of servers in active NOAM VIP
1. Log in to the NOAM GUI using the VIP.
2. Navigate to Administration, then Software Management, and then Upgrade.
3. Select the Network Element of the server to be upgraded (NOAM or site).
  
  The active NOAM server may have some or all of these expected alarms:
  
  Alarm ID = 10008 (Provisioning Manually Disabled)
  
  Alarm ID = 32532 (Server Upgrade Pending Accept/Reject)
Verify Status of Server to be Upgraded
1. Identify the server to be upgraded (NOAM, SOAM, MP, and so on) and record hostname.
2. Verify the Application Version value is the expected source software release version.
3. If the server is in the Backup Needed state, select the server and click Backup.
4. On the Upgrade Backup screen, click OK.
  
  The Upgrade State changes to Backup in Progress.
5. Verify the OAM Max HA Role is the expected condition (either standby or active). This depends on the server being upgraded.
6. When the backup is complete, verify the server state changes to Ready.
Initiate the Server Upgrade
1. From the Upgrade Administration screen, select the server to be upgraded.
2. Click Upgrade Server.
  
  The Initiate Upgrade form appears.
Select Upgrade ISO
1. Initiate the server upgrade. From the Upgrade Settings – Upgrade ISO options, select the ISO to use in the server upgrade.
  
  Note:
  When the active NOAM is the server being upgraded, click OK to initiate an HA switchover and cause the GUI session to log out.
  If the selected server is the active server in an active/standby pair, the OAM Max HA Role column displays Active with a red background. This is NOT an alarm condition. This indicator is to make the user aware the Make Ready action causes an HA switchover.
2. Click OK.
  
  The upgrade begins and control returns to the Upgrade Administration screen.
  
  Note:
  Do not omit this step.
3. Log out of the GUI, clear the browser cache, and log back into the active NOAM via the VIP before continuing. Some GUI forms may exhibit incorrect behaviors if the browser cache is not cleared.
View the Upgrade Administration Form to Monitor Upgrade Progress
1. Observe the upgrade status of the site on the Upgrade Administration screen by selecting the Entire Site link. An upgrade status summary of each server group in the site displays in the Server Upgrade States column.
  
  Servers may have a combination of the following expected alarms. However, not all servers have all alarms.
  
  Alarm ID = 10008 (Provisioning Manually Disabled)
  
  Alarm ID = 10075 (The server is no longer providing services because application processes have been manually stopped)
  
  Alarm ID = 10073 (Server Group Max Allowed HA Role Warning)
  
  Alarm ID = 32515 (Server HA Failover Inhibited)
  
  Alarm ID = 31228 (HA Highly available server failed to receive mate heartbeats) or (Lost Communication with Mate Server)
  
  Alarm ID = 31283 (Highly available server failed to receive mate heartbeats)
  
  Alarm ID = 31106 (DB Merge tTo Parent Failure)
  
  Alarm ID = 31107 (DB Merge fFrom Child Failure)
  
  Alarm ID = 31233 (HA Secondary Path Down)
  
  Alarm ID = 31101 (DB Replication tTo Slave Failure)
  
  Alarm ID = 31104 (DB Replication over SOAP has failed
  
  Alarm ID = 31282 (The HA manager (cmha) is impaired by a s/w fault)
  
  Alarm ID = 31225 (HA Service Start Failure)
  
  Alarm ID = 31226 (HA Availability Status Degraded)
  
  Alarm ID = 31114 (DB Replication over SOAP has failed)
  
  Alarm ID = 31149 (DB Late Write Nonactive)
2. Wait for the upgrade to complete. The Status Message column displays Success. This step takes approximately 20 to 50 minutes.
  
  Note:
  In the unlikely event that after the upgrade, if the Upgrade State of server is Backout Ready or Failed and the Status Message displays Server could not restart the application to complete the upgrade, then perform the stepms mentioned in Manual Completion of Server Upgrade to restore the server to full operational status and return to this step to continue the upgrade.
  
  Note:
  Perform Create a Link for ComAgent to create a link of Comagent. If the upgrade fails, do not proceed. It is recommended to consult with Create a Link for ComAgent on the best course of action. Refer to Recover from a Failed Upgrade for failed server recovery procedures.
  See Server CLI: (Optional) View in progress status from command line of server section for an optional method of monitoring upgrade progress. See Server CLI: If the upgrade fails section for instructions if the upgrade fails.
View In Progress Status from Command Line of Server in Server CLI
Note:
This is an optional method to view the upgrade progress from the command line.

To view the detailed progress of the upgrade, access the server command line (via SSH or Console), and enter:
```
$ tail -f /var/TKLC/log/upgrade/upgrade.log 
```
This command displays the upgrade log entries as the events occur. Once the upgrade is complete, the server reboots. It takes a couple of minutes for the DSR application processes to start up. For example, this command displays the current rev on the server:

[admusr@NO2 ~]$ appRev

Install Time: Thu Dec 15 00:05:46 2016

Product Name: DSR

Product Release: 8.6.0.7.0_96.34.0

Base Distro Product: TPD

Base Distro Release: 7.8.3.0.0-89.21.0

Base Distro ISO: TPD.install-7.8.3.0.0-89.21.0-OracleLinux6.10-x86_64.iso

ISO name: DSR-8.6.0.7.0_96.34.0.iso

OS: OracleLinux 6.10

Note:
If the upgrade fails, do not proceed. It is recommended to consult with on the best course of action. Refer to Recover from a Failed Upgrade for failed server recovery procedures.
If the upgrade of a server fails, access the server command line (through ssh or a console), and collect the following files:

/var/TKLC/log/upgrade/upgrade.log

/var/TKLC/log/upgrade/ugwrap.log

/var/TKLC/log/upgrade/earlyChecks.log

/var/TKLC/log/platcfg/upgrade.log

Note:
It is recommended to contact My Oracle Support by referring to Create a Link for ComAgent of this document and provide these files. Refer to Recover from a Failed Upgrade for failed server recovery procedures.
Verify post upgrade status using active NOAM VIP.
1. Navigate to Administration, then Software Management, and then Upgrade.
2. Select the tab of the NOAM or site being upgraded.
3. Verify the Application Version value for this server has been updated to the target software release version.
4. Verify the Upgrade State of the upgraded server is Accept or Reject.
Verify if the Server was Successfully Upgraded
Navigate to Alarm & Events, then View Active.
The active NOAM or SOAM server may have some or all the following expected alarms:
- Alarm ID = 10008 (Provisioning Manually Disabled)
- Alarm ID = 10010 (Stateful database not yet synchronized with mate database)
- Alarm ID = 10075 (The server is no longer providing services because application processes have been manually stopped)
- Alarm ID = 31000 (Program impaired by S/W Fault)
- Alarm ID = 31201 (Process Not Running) for eclipseHelp process
- Alarm ID = 31282 (The HA manager (cmha) is impaired by a s/w fault)
The active NOAM or SOAM has these expected alarms until both NOAMs/SOAMs are upgraded:
- Alarm ID = 31233 – HA Secondary Path Down
- Alarm ID = 32532 (Server Upgrade Pending Accept/Reject)
Note:
Do not accept upgrade at this time. This alarm is OK.