This chapter describes how to identify and correct problems while running your application. Troubleshooting includes the following topics:

• Run-time Errors

• Error Event Logging

• Failure to Connect to the CLS

• Network Errors

• TCP/IP Error Codes

• Tracing PAMS API Activity

• Tracing Client Library Activity

• Recovering from Client Crashes

Before you begin troubleshooting, you can verify the version number of the MessageQ MVS Client library you are running by enabling tracing of Client Library activity, running your application, and checking the error log file, DMQELOG, for the version number.

Run-time Errors

Problems at runtime can arise from many different circumstances. To identify and solve problems with the MessageQ MVS Client, follow these steps:

1. Check the contents of the DMQELOG file to get more information about the problem. Network errors are identified in the error log file.

2. Use the trace output capability on the MVS Client and the Client Library Server (CLS) to get a detailed flow of the activity that leads up to the problem.

3. Log in to the MessageQ Server system and run the MessageQ Monitor Utility. On UNIX systems, use the character-cell program, dmqmonc, to monitor MessageQ groups remotely.

4. Use the netstat TCP/IP Utility to monitor the network connections on the client. Also, use netstat on the server system to monitor the TCP/IP connections on the host system where the CLS is running.

5. Try to repeat the error using one of the test programs included with the MessageQ MVS Client. Reproducing problems with the test programs is an effective way to isolate application programming errors and provide a convenient way to test problems.

Review the following topics in this guide for more information on finding and resolving problems.

Error Logging

Run-time errors detected by the Client library are written to the DMQELOG file. The errors indicate a run-time problem due to a configuration error, an application error, network problem, or unexpected server response.

Error logging can be enabled or disabled by changing the MessageQ MVS Client Configuration Logging option. Refer to Chapter 3 for instruction on how to configure error logging.

Failing to Connect to the CLS

The MessageQ MVS Client attempts to establish a connection to the CLS in response to a call to pams_attach_q. The attempt to establish a connection fails immediately when:

• There is no CLS running on the system identified in the MessageQ MVS Client's default server configuration.

• The CLS is listening for connections on a different endpoint than the endpoint defined in the MessageQ MVS Client's default server configuration.

• The network transport used by the CLS differs from the transport used by the client.

When the connection attempt fails, pams_attach_q returns the following error status:

PAMS__NETNOLINK	

Check the DMQELOG error file for the host name, and the endpoint of the server system with which the MessageQ MVS Client attempted to connect. Either start the CLS on the server system, or reconfigure the MessageQ MVS Client default server.

Network Errors

Network errors result from the Client library receiving an error when attempting to read or write on the network link. Occasional network connection problems can occur due to the state of the TCP/IP protocol stack or the network connection to the host system. Network errors are identified by the return status from the pams_attach_q function, such as the following:

PAMS__NETNOLINK

Network connection errors might also occur when attempting to execute any of the MessageQ API functions. For example, the pams_put_msg and pams_get_msg functions return the following return code when the connection to the server is broken and MRS is not enabled:

PAMS__NETERROR

The specific steps for clearing the network error depend on how the problem developed. The following actions will generally clear the problem:

1. Check the error log file, DMQELOG, for a description of the error event.

2. Stop and restart the application. In some cases, restarting the application or simply retrying the attach operation succeeds.

3. Stop and restart the CLS.

Table 5-1 lists the common TCP/IP error status codes logged to the file DMQELOG when the MVS Client cannot connect to the CLS. These error codes may also be logged when using the DECnet transport.

Table 5-1 Common TCP/IP Connect Errors

TCP/IP Error	Meaning	Recovery
EINVAL	A socket initialization error	Verify the installation of TCP/IP MVS.
ECONNREFUSED	Connection request to the server is refused because a CLS is not listening on the endpoint	Verify that the endpoint defined in the MVS Client Server configuration endpoint matches the endpoint used by the CLS. Verify that the CLS is running.
ENOBUFS	TCP/IP has run out of available network buffers.	Frequent attach/detach operations may result in connections that have not cleared out. Usually, they clear after a 60-90 second time-out. Increase the maximum number of TCP/IP sockets if needed.

Tracing PAMS API Activity

To obtain a time-stamped output file showing the sequence of MessageQ function calls and return status codes, follow the steps outlined in Chapter 3 for Configuring Tracing. The information from the pams_ function call trace is written to the dmqelog file in the default directory for the application. The PAMS tracing option can be useful for observing the sequence of message function calls to determine the run time behavior of the application.

Tracing Client Library Activity

To obtain detailed, time-stamped traces of the client library activity, enable the DmqclTrace option in the client configuration file.

Recovering from Client Crashes

Occasionally, applications crash (particularly during development) and do not have an opportunity to close or return resources in use before terminating. Applications using the MessageQ MVS Client attached to the message queuing bus that terminate without calling pams_exit or pams_detach_q leave many resources allocated but not available for reuse. Resources that are in use after a client application crash include:

• Global memory allocated on behalf of the client application

• Network protocol resources, such as sockets

• Network resources on the server system

• Message queue resources used by the CLS on behalf of the client

After the client crashes, the server system still has an open connection to the client and the CLS remains attached to the primary queue used by the client. The network protocol keep-alive mechanism does not notify the server that the client has gone away for a lengthy time period. Typically, you can reboot the client system and the server still functions as though it has a connection open to the client.

Restarting the client application usually establishes a new connection to CLS. If network connect errors occur, follow the troubleshooting procedure described in the Network Errors topic. The procedure releases and frees all resources used by the client.

If the client application calls pams_attach_q using either ATTACH_BY_NAME or ATTACH_BY_NUMBER to attach to a specific primary queue, the CLS detects a client reconnect attempt and automatically terminates the CLS instance (server process or thread) attached to the same message queue. Reconnecting to the same queue is only accepted if the client application is attempting to reconnect from the same host as the previous connection.

If the client application calls pams_attach_q using the ATTACH_TEMPORARY attach mode, a new instance of the CLS is started to support the client reconnect. The previous instances of the CLS remains active. For information about terminating CLS servers, see the CLS topic in the Installation and Configuration Guide for your MessageQ server platform.