Recovery Procedure for DSM with EPAP or ELAP

  1. Use the rep-stat-mps command to obtain MPS status.
    The following is an example of a possible output:
    > rept-stat-mps
    
    Command Accepted - Processing
       peliscaa00w 04-08-13 11:11:04 EDT EAGLE 29.0.2-46.33.1
       rept-stat-mps
       Command entered at terminal #2.
    ;
    
       peliscaa00w 04-08-13 11:11:04 EDT EAGLE 29.0.2-46.33.1
                          VERSION       PST            SST        AST
       ELAP A             002-002-000   IS-NR          Active     -----
          CRITICAL PLATFORM    ALARM DATA = No Alarms
          MAJOR    PLATFORM    ALARM DATA = No Alarms
          MINOR    PLATFORM    ALARM DATA = No Alarms
          CRITICAL APPLICATION ALARM DATA = No Alarms
          MAJOR    APPLICATION ALARM DATA = No Alarms
          MINOR    APPLICATION ALARM DATA = No Alarms
                 ALARM STATUS       = No Alarms.
                          VERSION       PST            SST        AST
       ELAP B             002-002-000   IS-NR          Standby    -----
          CRITICAL PLATFORM    ALARM DATA = No Alarms
          MAJOR    PLATFORM    ALARM DATA = No Alarms
          MINOR    PLATFORM    ALARM DATA = No Alarms
          CRITICAL APPLICATION ALARM DATA = No Alarms
          MAJOR    APPLICATION ALARM DATA = No Alarms
          MINOR    APPLICATION ALARM DATA = No Alarms
                 ALARM STATUS       = No Alarms.
       CARD   PST          SST         LNP STAT
       1101 P IS-NR        Active      ACT
       1103   IS-NR        Active      ACT
       CARD 1101 ALARM STATUS = No Alarms.
         DSM PORT A:      ALARM STATUS       = **  0084 IP Connection Unavailable
         DSM PORT B:      ALARM STATUS       = **  0084 IP Connection Unavailable
       CARD 1103 ALARM STATUS = No Alarms.
         DSM PORT A:      ALARM STATUS       = **  0084 IP Connection Unavailable
         DSM PORT B:      ALARM STATUS       = **  0084 IP Connection Unavailable
       Command Completed.
  2. Use the rept-stat-db:display=all:db=mps to determine the “LEVEL” on the DSM cards are not incrementing.
    The following is an example of a possible output:
    > rept-stat-db:display=all:db=mps
     
      Command Accepted - Processing
       peliscaa00w 04-08-13 11:14:32 EDT EAGLE 29.0.2-46.33.1
       rept-stat-db:display=all:db=mps
       Command entered at terminal #2.
    ;
    
       peliscaa00w 04-08-13 11:14:32 EDT EAGLE 29.0.2-46.33.1
    
                       ELAP A  ( ACTV )
                        C  BIRTHDATE          LEVEL       EXCEPTION
                        -  -----------------  ----------  --------------
       RTDB             Y  04-08-13 02:32:02        1264        -
       RTDB-EAGLE          04-08-13 02:29:22        1264        -
    
                       ELAP B ( STDBY )
                        C  BIRTHDATE          LEVEL       EXCEPTION
                        -  -----------------  ----------  --------------
       RTDB             Y  04-08-13 02:32:02        1264        -
       RTDB-EAGLE          04-08-13 02:29:22        1264        -
     
                       EAGLE RTDB REPORT
       CARD/APPL  LOC   C  BIRTHDATE          LEVEL       EXCEPTION
       ---------  ----  -  -----------------  ----------  --------------
       VSCCP      1101  Y  04-08-13 02:29:22        1108        -
       VSCCP      1103  Y  04-08-13 02:29:22        1108        -
  3. Execute rept-stat-trbl:display=timestamp to see if all of the UAM 0084 occurred at nearly the same time.
    If all of the DSMs are reporting IP connection unavailable and all of the alarms came in at one time, check whether the associated EPAP or ELAP has a problem. If so, go to 7.
  4. Perform the following to verify IP network connectivity.
    1. Use the following command to ping the local host:
      pass:loc=XXXX:cmd=”ping 127.0.0.1”

      This is the loopback address and testing it will indicate if networking support is functioning.

    2. Ping the MPS using:
      pass:loc=XXXX:cmd=”ping 192.168.120.100” for 100 Megabit Network
      pass:loc=XXXX:cmd=”ping 192.168.120.200” for 100 Megabit Network
      pass:loc=XXXX:cmd=”ping 192.168.121.100” for 10 Megabit Network
      pass:loc=XXXX:cmd=”ping 192.168.121.200” for 10 Megabit Network

      Example of a possible output:

      192.168.120.100 for EPAP A, via DSM Port A / 100 Megabit Network
      192.168.120.200 for EPAP B, via DSM Port A / 100 Megabit Network,
      192.168.121.100 for EPAP A, via DSM Port B / 10 Megabit Network,
      192.168.121.200 for EPAP B, via DSM Port B / 10 Megabit Network
    3. If the ping command is not working, verify the IP network cabling.

      Note:

      The network between the MPS and the DSMs is a private network.
    4. Replace the DSM card if you can’t ping the local host and the IP network has been verified.

      See the Maintenance Manual, Appendix A, Card Removal/Replacement Procedures.

      Note:

      This step applies to a single DSM having IP connection unavailable.
  5. If all of the DSMs are reporting IP connection unavailable, reboot the "primary" DSM card. If no primary card is available go to 3. If only one card is reporting IP connection unavailable, use the init command to take down the card and reload it. See the Commands Manual for the INIT-CARD command.
  6. Power cycle the hubs between the MPS and the EAGLE STP.
  7. Task to be completed by either the customer or an Engineer from the LSMS/MPS group: Run a syscheck on the EPAP or ELAP.
  8. Task to be completed by either the customer or an Engineer from the LSMS/MPS group: Switch activity on the EPAP or ELAP.

    If this action does not clear the alarm, continue to 9.

  9. Task to be completed by either the customer or an Engineer from the LSMS/MPS group: Stop and restart the software on the EPAP or ELAP.

    In case of failure of the Active EPAP, the Standby EPAP takes over the role of Active EPAP and continues to provision the database. If the main DSM network fails, the Active EPAP switches to the backup DSM network to continue provisioning the Service Module cards. The switchover to the backup DSM network is possible only when all connectivity is lost on the main DSM network. In case of HUB failure, impact on the provisioning may be partial, so switchover to the backup provisioning network might not be automatically triggered. At any given time, there is only one Active EPAP using one DSM network per EPAP system.

    If this action does not clear the alarm, continue to 10.

  10. If these steps do not clear the fault, contact My Oracle Support (MOS).
When the IP connection recovers you will see an IP Connection Available message.
UAM: 
1481.0085    DSM A   1101          IP Connection Available
1482.0085    DSM B   1101          IP Connection Available