Overview of Logging

Logging is the means by which a system provides you with time-stamped and labeled information about the system’s services. Logging provides both a current snapshot of the system as well as a historical view.

By understanding and using Messaging Server log files, you can:

• Gather message statistics, such as message size, rate of message delivery, and how many messages are passing through the MTA

• Perform trend determination

• Correlate capacity planning

• Troubleshoot problems

For example, if your site needs to add more disk storage due to an increase in the number of users, you can use the Messaging Server log files to see what percentage your system demand has increased by and plan for the amount of new disk storage you need.

You can also use Messaging Server logs to understand what your messaging pattern looks like across one day. Understanding when your daily peak loads occur helps you conduct capacity planning.

Logging is also helpful for troubleshooting user problems. For example, if a user isn’t receiving expected mail messages, you can use the Messaging Server logging facilities to trace the user’s mail messages. In so doing, you might find out that the messages didn’t arrive because they were automatically filtered and sent to a SPAM folder.

Types of Logging Data

In general, logging provides you with two types of information:

• Operational data

• Error conditions, also known as event logging

For the most part, Messaging Server logging provides operational data. This operational data contains information such as: the date and time a message entered the system; the sender and recipient of the message; when the message was written to disk; and at a later point in time, when the message was removed from disk and inserted into user’s mailbox.

However, Messaging Server logging does also provide some event logging data. To obtain event logging data, you need to pull together multiple items from different log files. You could then use a unique constant, such as message ID, to search and correlate the lifecycle of a message as it passed from point to point through the system.

Types of Messaging Server Log Files

Messaging Server logging consists of three types of log files:

1. MTA logs. These logs provide operational data previously described for the Message Transfer Agent.

2. Error logs. These are the MTA debug logs, and the MTA subcomponent logs (that is, job controller, dispatcher and so on).

3. Message Store and Service logs. These logs provide messages from the http server, mshttpd, imap, and pop services, as well as the Admin service. The format of these logs differs from that of the first two types of logs.

The following table lists the different types of log files. By default, log files are located in the msg_svr_base/data/log directory. You can customize and view each type of log file individually.

Table 21–1 Messaging Server Log Files

Type of Log File	Log File description	Default Name
Message Transfer Agent	Show information about message traffic through the MTA including date and time information, enqueue and dequeue information, and so on.	mail.log, mail.log_current, mail.log_yesterday
Connections	Contains remote machines (MTAs) that connect to this system to send email.	connection.log
Counters	Contains message trends in terms of messages sent and received on a per channel basis.	counters
Job Controller	Contains data on the master, job controller, sender, and dequeue channel programs.	job_controller.log
Dispatcher	Contains errors pertaining to the dispatcher. Turning on dispatcher debugging will increase the information.	dispatcher.log
Channel	Records errors pertaining to the channel. Keywords master_debug and slave_debug turns on channel debugging, which increases the verbosity of the channel log files. Level and type of information is controlled with the various *_DEBUG MTA options in option.dat.	channelname_master.log* (example: tcp_local_master.log*) channelname_slave.log* (example, tcp_local_slave.log*)
Admin	Contains logged events related to communication between Console and Messaging Server (mostly through several CGI processes), by way of its Administration Server	admin, admin.sequenceNum.timeStamp
IMAP	Contains logged events related to IMAP4 activity of this server	imap, imap.sequenceNum.timeStamp
POP	Contains logged events related to POP3 activity of this server	pop, pop.sequenceNum.timeStamp
HTTP	Contains logged events related to HTTP activity of this server	http, http.sequenceNum.timeStamp
Default	Contains logged events related to other activity of this server, such as command-line utilities and other processes	default, default.sequenceNum.timeStamp
msgtrace	Contains trace information for the Message Store. File can grow very large very quickly. Monitor accordingly.	msgtrace
watcher	monitor process failures and unresponsive services (see Table 4–4) and will log error messages indicating specific failures.	watcher

where:

sequenceNum - specifies an integer that specifies the order of creation of this log file compared to others in the log-file directory. Log files with higher sequence numbers are more recent than those with lower numbers. Sequence numbers do not roll over; they increase monotonically for the life of the server (beginning at server installation).

timeStamp - Specifies a large integer that specifies the date and time of file creation. (Its value is expressed in standard UNIX time: the number of seconds since midnight January 1, 1970.)

For example, a log file named imap.63.915107696 would be the 63rd log file created in the directory of IMAP log files, created at 12:34:56 PM on December 31, 1998.

The combination of open-ended sequence numbering with a timestamp gives you more flexibility in rotating, expiring, and selecting files for analyzing. For more specific suggestions, see Defining and Setting Service Logging Options

Tracking a Message Across the Various Log Files

The following describes how a message flows through the system, and at what point information gets written to the various log files. This description is meant to aid you in your understanding of how to use Message Server’s log files to troubleshoot and resolve problems. See Figure 8–2 to follow along.

1. A remote host makes a connection to the TCP socket on your messaging host, requesting SMTP service.

2. The MTA dispatcher responds to the request, and hands off the connection to your messaging host’s SMTP service.

   As the MTA is modular in design, it consists of a set of processes, including the job controller and the SMTP service dispatcher. The dispatcher takes the incoming TCP connection and sends it to the SMTP service. The SMTP service writes the message to disk to a channel area. The SMTP service understands the message’s envelope parameters, such as sender and recipient. Configuration entries in the system tell what destination channel it belongs to.

3. The dispatcher writes to the dispatcher.log file that it forked a thread and made the thread available to incoming connection from a certain IP address.

4. The SMTP server writes to its tcp_smtp_server.log file, recording the dialog of what happens when the remote host connected to it and sent a message. This log file gets created when dispatcher hands off to SMTP server on the host’s IP.

5. The SMTP server writes the message to a queue area on disk for a channel program such as tcp_intranet, and informs the job controller.

6. The job controller contacts the channel program.

7. The channel program delivers the message.

   Each channel has its own log file. However, these logs usually show the starting and stopping of the channel. To get more information, you need to enable debug level for the channel. However, as this can slow down your system and actually make problems more obscure if left on, you should only enable debug level when an actual problem is occurring.

   ---

   Note –

   For efficiency, if a channel is already running for an existing process, and a new message comes in, the system does not spawn a new channel process. The currently running process picks up the new message.

   ---

8. The message is delivered to its next hop, which could be another host, another TCP connection, and so forth. This information is written in the connection.log file.

   At the same time that the SMTP server writes the message to a queue area on disk, the channel responsible for the message writes a record in the mail.log_current, or mail.log file. The record shows such information as the date and time the message was enqueued, the sender, the recipient, so forth. See MTA Message Logging Examples for more information. The most useful file for tracing the message is the mail.log_current file.