1. DSR Cloud Software Upgrade Guide
  2. Workarounds

9 Workarounds

9.1 Workaround to Resolve DB Site Replication Alarms

The following procedure resolves DB site replication alarms if encountered during the upgrade. This procedure restarts the inetrep process on the server that has a DB replication failure alarm. Database (DB) replication failure alarms may display during an Auto Site Upgrade or during an event that resets multiple servers in parallel. The DB on the child servers is not updated until resolved.
  1. Server CLI: Log in to the server.
    1. Use the SSH command (on UNIX systems – or putty if running on Windows) to log in to the active NOAM:

      ssh admusr@<server address>

      password: <enter password>

      Answer yes if you are asked to confirm the identity of the server.

  2. Server CLI: Check if the replication links are up.
    1. Execute this command:

      irepstat

      Some of the B-C and C-C replications links may be down.

  3. Server CLI: Resolve replication issue
    1. Execute this command:

      sudo pm.kill inetrep

      Note:

      Repeat this procedure on each affected server.

9.2 Workaround to Resolve Device Deployment Failed Alarm

This procedure resolves the device deployment failed alarm i.e. 10054. If this procedure fails, it is recommended to contact My Oracle Support (MOS) and ask for assistance.

9.2.1 NOAMP VIP GUI: Log In

  1. Open the web browser and enter the following URL:

    http://<Primary_NOAM_VIP_IP_Address>

  2. Log in to the NOAM GUI as the guiadmin user.

9.2.2 NOAMP VIP GUI: Identify Server(s) and Interface(s) with Alarm

Navigate to current alarm details and identify the server and interface where the 10054 - Device Deployment Failed alarm is displayed.
  1. Navigate to Alarms & Events, then View Active.
  2. Look for the 10054 alarm and make a list of the server(s) and interface(s).

9.2.3 NOAMP VIP GUI: Corrective Action for Alarm 10054

Interfaces like XMI and IMI are in locked state and do not allow editing as a corrective action. For XMI and IMI interfaces, first unlock the interface and for other interfaces skip steps 1 to 4 below.
  1. Navigate to Configuration, then Networking, and then Networks, select the respective “Network element” tab used for the server configuration.
  2. Click the Network Name row.
  3. Click Unlock. Click the checkbox to confirm it and click OK.
  4. To unlock the network for the particular device, navigate to Configuration, then Networking, and then Devices.
  5. Click the Server tab from the list in task 2 in this procedure.
  6. Select each interface row one by one for which alarm is showing and click Edit.
  7. Click OK.

    Note:

    Give some time to the system to auto correct the condition to clear the alarm. Once this step is done, lock the network back again which were unlocked above.

    For XMI and IMI interfaces, lock the interface back, for other interfaces skip 1 to 4 below.

  8. To lock the network for a specific device, navigate to Configuration, then Networking, and then Networks, select the respective Network element tab used for the server configuration.
  9. Click the Network Name row.
  10. Click Lock. Click the checkbox to confirm it and click OK.

9.3 Workaround to Resolve syscheck Error for CPU Failure

This procedure details the workaround to resolve syscheck error for CPU failure.

9.3.1 Log in to the server using CLI on which syscheck is failing

  1. Use the SSH command (on UNIX systems – or putty if running on windows) to log in to the server identified:

    ssh admusr@<SERVER_XMI>

    password: <enter password>

    Answer yes if you are asked to confirm the identity of the server.

9.3.2 Server CLI: Execute Workaround

  1. Edit the cpu config file.

    $ sudo vim /usr/TKLC/plat/lib/Syscheck/modules/system/cpu/config

  2. Comment out the all the text that reads: EXPECTED_CPUS= by putting # at the beginning of the line, for example:

    # EXPECTED_CPUS=2

  3. Save the cpu config file.
  4. Reconfig the syscheck by running these commands:

    sudo syscheck --unconfig

    sudo syscheck --reconfig

    sudo syscheck

    CPU related errors do not display.

9.4 Workaround to Resolve the Server HA Switchover Issue

The following procedure resolves the HA switchover issue. It restarts the cmha process on the server that has HA switchover issue.
  1. Server CLI: Log in to the server.
    1. Use the SSH command (on UNIX systems – or putty if running on Windows) to log in to the NOAM server which is experiencing the HA switchover issue:

      ssh admusr@<server address>

      password: <enter password>

      Answer yes if you are asked to confirm the identity of the server.

  2. Server CLI: Resolve HA switchover issue(s).
    1. Execute this command:

      sudo pm.kill cmha

  3. Repeat this procedure on each affected server.

9.5 Resolving Error - CD ROM Invalid

The following CDROM invalid error is displayed at the following path /var/TKLC/appw/logs/Process/upgrade.log 
1595360197::Error: Unable to open file
/var/TKLC/upgrade/DSR-8.5.0.0.0_90.3.0-x86_64.iso: No such file or directory
1595360197::
1595360197::^H|
1595360197::UMVT Validate Utility v2.3.4, (c)Tekelec, May 2014
1595360197::CDROM is Invalid
1595360197::ISO UMVT digest does not match calculated digest!
1595360197::umvtvalidate returned:
1595360197::ERROR:  Backing out changes from VALIDATE_CD on backwards...
1595360197::ERROR:  CD is not valid.
1595360197::upgrade will not be performed!!!
Perform the following procedure to resolve the CDROM invalid error:
  1. As a root user, run the following commands on the server where the upgrade failed because of the CDROM invalid error:
    cd /usr/TKLC/appworks/sbin
    ./backout_restore
    init 6
  2. Before upgrade, verify if /var/TKLC/upgrade has an iso file and if the size of the file is appropriate.
  3. In case of any server issue, instead of ASU, perform upgrade from Platcfg by performing following steps:
    1. Copy the iso file to the following path /var/TKLC/upgrade.
    2. Provide required permission to the iso file.
    3. Perform upgrade from platcfg.
  4. If the upgrade fails, perform the following steps on the server:
    1. Copy the iso file to the following path /var/TKLC/upgrade.
    2. Provide the required permission.
    3. If there is an error, remove the last entry from the revision file.
    4. Skip the early upgrade check by running the following command:
      sudo su -
        sudo su touch /tmp/ignoreDsrEarlyUpgradeChecks echo "IGNORE_EARLY_CHECKS=1" >
        /var/TKLC/log/upgrade/tmp_upgrade.conf chmod 777
        /var/TKLC/log/upgrade/tmp_upgrade.conf
  5. Start the upgrade from platcfg.