7 Troubleshooting

This chapter offers troubleshooting procedures based on symptoms occurring in the network.

General troubleshooting

1. Work from the bottom of the protocol stack up: first, IP Network; then the SS7 link or connection; then traffic routing.
2. Review provisioning and verify configuration in this order:
   • Card
   • Signaling (SS7) link
   • Linkset
   • IP link or IP network
   • Association or Application Server (IPGWx only)
   • Traffic routing or SS7 route and route key (IPGWx only)

   General troubleshooting tools include the following:

   • Ethereal – PC-based network analyzer (sniffer) – www.ethereal.com/www.wireshark.com
   • netstat/sctp pass commands to display TCP/IP or SCTP/IP network statistics
   • ualog/asplog/linkinfo pass command to retrieve logs of events in stack and control messages transmitted or received
   • msucount pass command to display traffic counts of MSUs that have been transmitted, received, rerouted, or discarded, and the discard reason

Verify UIMs and UAMs

1. If there are any Unsolicited Information Messages (UIMs) or Unsolicited Alarm Messages (UAMs) occurring related to the SIGTRAN configuration, refer to Unsolicited Alarms and Information Messages Reference.

Is the card configured correctly?

1. Card in system?

   rtrv-card

   rept-stat-card

   • IP link configured correctly? (rtrv-ip-lnk; preferred settings are 100/full duplex on card AND switch - no AUTO configure)
   • IP link configured correctly? (rtrv-ip-lnk; preferred settings are 100/full duplex on card AND switch - no AUTO configure)
   • IP host table configured? (rtrv-ip-host; check for local and remote addresses)
   • Signalling links (SLKs) and linksets configured correctly? (rept-stat-slk/rept-stat-ls)
2. IP link configured correctly?

   rtrv-ip-lnk

   Preferred settings are 100/full duplex on card AND switch - no AUTO configure

3. IP routing configured?

   rtrv-ip-rte

   rtrv-ip-card

4. IP host table configured?

   rtrv-ip-host

   Check for local and remote addresses.

5. Signalling links (SLKs) and linksets configured correctly?

   rept-stat-slk

   rept-stat-ls

Connection does not become established

1. Card up and stable?

   rept-stat-card

2. Association status?

   rept-stat-assoc

3. Network connectivity?

   netstat -I

   rept-stat-card:mode=full

4. Errors (collisions, etc.) on the network interface?

   netstat -d 0/1t

5. Far end reachable?

   ping

   traceroute

6. Near end and far end use same SCTP CRC?

   netstat –p

   sctp/rtrv-sg-opts

Connection bounces and is unstable

1. Transport stable?

   netstat –i

   netstat –d

2. RMIN set too low?

   ping

   assocrtt

   rtrv-assoc

   Rule of thumb is above 1.2 * average RTT

AS/PC in route key does not become available or ACTIVE (IPGWx only)

1. Connection in correct AS?

   rtrv-as

2. Routing key provisioned for AS?

   rtrv-appl-rtkey

3. Network appearance/routing context required and matched?

   rtrv-appl-rtkey

   ualog

4. AS/ASP activated at far end?

   aslog

   ualog

5. SS7 APC/SAPC and associated route exists in the same network (and group code) as the PC?

   rtrv-rte

   rtrv-ls

IP destination is not informed of SS7 destination status changes; network management is not working correctly (IPGWx only)

1. Route key is not provisioned for IPGWx linkset virtual APC, but SS7 route is?

   rtrv-rte

   rtrv-appl-rtkey/display-=all

2. AS connections hosted by cards in different linksets/matesets; is the mateset equivalent to linkset?

   rtrv-as

   rtrv-assoc

   rtrv-ls

Traffic not arriving at IP destination or traffic is lost

1. Route to destination’s PC entered and available?

   rept-stat-dstn

2. Traffic being received/discarded on IP card? IPGWx application has numerous discard reasons!

   msucount -l

Are connection(s) congesting?

1. Is SCTP buffering set correctly for network RTT?

   rtrv-assoc

   assocrtt,

   sctp

2. Is IPTPS set correctly for IPGWx?

   rept-stat-iptps

   rtrv-ls

3. Is an interface set to half-duplex somewhere in the path to the far end, causing excessive retransmissions?

   rtrv-ip-lnk

   sctp

Traffic not load-balanced properly

1. Source traffic has uneven SLS distribution?
2. All cards in linkset or mateset do not host a connection to the IP Application Server(IPGWx only)?

   rtrv-assoc

   rtrv-as

3. IPGWx cards in mateset with no established connections have signaling link deactivated to minimize ‘double-hopping’ (IPGWx only)?

   rept-stat-card

   msucount -l

Link Level Events

1. IPLIM pass command

   linkinfo -l

2. IPLIMx linkinfo has other interesting options?

   -c

   -m

3. IPGWx pass command

   ualog

   aslog

4. Both commands have event filtering (link events vs. traffic), so look at options

Association

1. IPLIM/ pass command

   sctp -a