Sun Java System Messaging Server 6.3 Administration Guide

25.4 Managing Message Store, Admin, and Default Service Logs

This section describes logging for the Message Store (POP, IMAP, and HTTP), Admin, and Default services. (see Table 25–1).

For these services, you specify log settings and to view logs. The settings you specify affect which and how many events are logged. You can use those settings and other characteristics to refine searches for logged events when you are analyzing log files.

This section contains the following subsections:

25.4.1 Understanding Service Log Characteristics

This section describes the following log characteristics for the message store and administration services: logging levels, categories of logged events, filename conventions for logs, and log-file directories.

25.4.1.1 Logging Levels

The level, or priority, of logging defines how detailed, or verbose, the logging activity is to be. A higher priority level means less detail; it means that only events of high priority (high severity) are logged. A lower level means greater detail; it means that more events are recorded in the log file.

You can set the logging level separately for each service—POP, IMAP, HTTP, Admin, and Default by setting the logfile.service.loglevel configuration parameter (see 25.4.3 Defining and Setting Service Logging Options). You can also use logging levels to filter searches for log events. Table 25–6 describes the available levels. These logging levels are a subset of those defined by the UNIX syslog facility.

Table 25–6 Logging Levels for Store and Administration Services

Level  

Description  

Critical 

The minimum logging detail. An event is written to the log whenever a severe problem or critical condition occurs—such as when the server cannot access a mailbox or a library needed for it to run. 

Error 

An event is written to the log whenever an error condition occurs—such as when a connection attempt to a client or another server fails. 

Warning 

An event is written to the log whenever a warning condition occurs—such as when the server cannot understand a communication sent to it by a client. 

Notice 

An event is written to the log whenever a notice (a normal but significant condition) occurs—such as when a user login fails or when a session closes. This is the default log level. 

Information 

An event is written to the log with every significant action that takes place—such as when a user successfully logs on or off or creates or renames a mailbox. 

Debug 

The most verbose logging. Useful only for debugging purposes. Events are written to the log at individual steps within each process or task, to pinpoint problems. 

When you select a particular logging level, events corresponding to that level and to all higher (less verbose) levels are logged. The default level of logging is Notice.


Note –

The more verbose the logging you specify, the more disk space your log files will occupy; for guidelines, see 25.4.3 Defining and Setting Service Logging Options


25.4.1.2 Categories of Logged Events

Within each supported service or protocol, Messaging Server further categorizes logged events by the facility, or functional area, in which they occur. Every logged event contains the name of the facility that generated it. These categories aid in filtering events during searches. Table 25–7 lists the categories that Messaging Server recognizes for logging purposes.

Table 25–7 Categories in Which Log Events Occur

Facility 

Description 

General 

Undifferentiated actions related to this protocol or service 

LDAP 

Actions related to Messaging Server accessing the LDAP directory database 

Network 

Actions related to network connections (socket errors fall into this category) 

Account 

Actions related to user accounts (user logins fall into this category) 

Protocol 

Protocol-level actions related to protocol-specific commands (errors returned by POP, IMAP, or HTTP functions fall into this category) 

Stats 

Actions related to the gathering of server statistics 

Store 

Low-level actions related to accessing the message store (read/write errors fall into this category) 

For examples of using categories as filters in log searches, see 25.4.4 Searching and Viewing Service Logs

25.4.1.3 Service Log File Directories

Every logged service is assigned a single directory, in which its log files are stored. All IMAP log files are stored together, as are all POP log files, and log files of any other service. You define the location of each directory, and you also define how many log files of what maximum size are permitted to exist in the directory.

Make sure that your storage capacity is sufficient for all your log files. Log data can be voluminous, especially at lower (more verbose) logging levels.

It is important also to define your logging level, log rotation, log expiration, and server-backup policies appropriately so that all of your log-file directories are backed up and none of them become overloaded; otherwise, you may lose information. See 25.4.3 Defining and Setting Service Logging Options.

25.4.2 Understanding Service Log File Format

All message store and administration service log files created by Messaging Server have identical content formats. Log files are multiline text files, in which each line describes one logged event. All event descriptions, for each of the supported services, have the general format:

dateTime hostName processName[pid]: category logLevel: eventMessage

Table 25–8 lists the log file components. Note that this format of event descriptions is identical to that defined by the UNIX syslog facility, except that the date/time format is different and the format includes two additional components (category and logLevel).

Table 25–8 Store and Administration Log File Components

Component  

Definition  

dateTime

The date and time at which the event was logged, expressed in dd/mm/yyyy hh:mm:ss format, with a time-zone field expressed as +/-hhmm from GMT. For example:02/Jan/1999:13:08:21 -0700

hostName

The name of the host machine on which the server is running: for example, showshoe.

Note: If there is more than one instance of Messaging Server on the host, you can use the process ID (pid) to separate logged events of one instance from another.

processName

The name of the process that generated the event: for example, cgi_store.

pid

The process ID of the process that generated the event: for example, 18753.

category

The category that the event belongs to: for example, General (see Example 25–5).

logLevel

The level of logging that the event represents: for example, Notice (see Example 25–4).

eventMessage

An event-specific explanatory message that may be of any length: for example, Log created (894305624).

Here are three examples of logged events:

02/May/1998:17:37:32 -0700 showshoe cgi_store[18753]:
 General Notice:
   Log created (894155852)

04/May/1998:11:07:44 -0400 xyzmail cgi_service[343]: General Error:
   function=getserverhello|port=2500|error=failed to connect

03/Dec/1998:06:54:32 +0200 SiroePost imapd[232]: Account Notice:
   close [127.0.0.1] [unauthenticated] 1998/12/3 6:54:32
   0:00:00 0 115 0

IMAP and POP event entries may end with three numbers. The example above has 0 115 0. The first number is bytes sent by client, the second number is the bytes sent by the server, and third number is mailboxes selected (always 1 for POP).

When viewing a log file in the Log Viewer window, you can limit the events displayed by searching for any specific component in an event, such as a specific logging level or category, or a specific process ID. For more information, see 25.4.4 Searching and Viewing Service Logs.

The event message of each log entry is in a format specific to the type of event being logged, that is, each service defines what content appears in any of its event messages. Many event messages are simple and self-evident; others are more complex.

25.4.3 Defining and Setting Service Logging Options

You can define the message store and administration service logging configurations that best serve your administration needs. This section discusses issues that may help you decide on the best configurations and policies, and it explains how to implement them.

25.4.3.1 Flexible Logging Architecture

The naming scheme for log files (service.sequenceNum.timeStamp) helps you to design a flexible log-rotation and backup policy. The fact that events for different services are written to different files makes it easier for you to isolate problems quickly. Also, because the sequence number in a filename is ever-increasing and the timestamp is always unique, later log files do not simply overwrite earlier ones after a limited set of sequence numbers is exhausted. Instead, older log files are overwritten or deleted only when the more flexible limits of age, number of files, or total storage are reached.

Messaging Server supports automatic rotation of log files, which simplifies administration and facilitates backups. You are not required to manually retire the current log file and create a new one to hold subsequent logged events. You can back up all but the current log file in a directory at any time, without stopping the server or manually notifying the server to start a new log file.

In setting up your logging policies, you can set options (for each service) that control limits on total log storage, maximum number of log files, individual file size, maximum file age, and rate of log-file rotation.

25.4.3.2 Planning the Options You Want

Keep in mind that you must set several limits, more than one of which might cause the rotation or deletion of a log file. Whichever limit is reached first is the controlling one. For example, if your maximum log-file size is 3.5 MB, and you specify that a new log be created every day, you may actually get log files created faster than one per day if log data builds up faster than 3.5 MB every 24 hours. Then, if your maximum number of log files is 10 and your maximum age is 8 days, you may never reach the age limit on log files because the faster log rotation may mean that 10 files will have been created in less than 8 days.

The following default values, provided for Messaging Server administration logs, may be a reasonable starting point for planning:

Maximum number of log files in a directory: 10

Maximum log-file size: 2 MB

Total maximum size permitted for all log files: 20 MB

Minimum free disk space permitted: 5 MB

Log rollover time: 1 day

Maximum age before expiration: 7 days

Level of logging: Notice

You can see that this configuration assumes that server-administration log data is predicted to accumulate at about 2 MB per day, backups are weekly, and the total space allotted for storage of admin logs is at least 25 MB. (These settings may be insufficient if the logging level is more verbose.)

For POP, IMAP or HTTP logs, the same values might be a reasonable start. If all services have approximately the same log-storage requirements as the defaults shown here, you might expect to initially plan for about 150 MB of total log-storage capacity. (Note that this is meant only as a general indication of storage requirements; your actual requirements may be significantly different.)

25.4.3.3 Understanding Logging Options

You can set options that control the message store logging configuration by the command line.

The optimal settings for these options depend on the rate at which log data accumulates. It may take between 4,000 and 10,000 log entries to occupy 1 MB of storage. At the more verbose levels of logging (such as Notice), a moderately busy server may generate hundreds of megabytes of log data per week. Here is one approach you can follow:

25.4.4 Searching and Viewing Service Logs

The log files provide the basic interface for viewing message store and administration log data. For a given service, log files are listed in chronological order. Once you have chosen a log file to search, you can narrow the search for individual events by specifying search parameters.

25.4.4.1 Search Parameters

These are useful search parameters you can specify for viewing log data:

* Any set of characters (example: *.com)

? Any single character (example: 199?)

[nnn] Any character in the set nnn (example: [aeiou])

[^nnn] Any character not in the set nnn (example: [^aeiou])

[n-m] Any character in the range n-m (example: [A-Z])

[^n-m] Any character not in the range n-m (example: [^0-9])

\ Escape character: place before *, ?, [, or ] to use them as literals

Note: Searches are case-sensitive.

Examples of combining logging level and facility in viewing logs might include the following:

25.4.5 Working With Service Logs

This section describes how to work with service logs by using the configutil command for searching and viewing logs. It consists of the following sections:

ProcedureTo Send Service Logs to syslog

  1. Run the configutil command with the syslogfacility option:

    configutil -o logfile.service.syslogfacility -v value

    where service is admin, pop, imap, imta, or http and value is user, mail, daemon, local0 to local7, or none.

    Once the value is set, messages are logged to the syslog facility corresponding to the set value and all the other log file service options are ignored. When the option is not set or the value is none, logging uses the Messaging Server log files.

25.4.5.1 To Disable HTTP Logging

If your system does not support HTTP message access, that is, Webmail, you can disable HTTP logging by setting the following variables. Do not set these variables if your system requires Webmail support (for example, Messenger Express).

ProcedureTo Set the Server Log Level

  1. Run the following configutil command:

    configutil -o logfile.service.loglevel -v level

    where service is admin, pop, imap, imta, or http and loglevel is Nolog, Critical, Error, Warning, Notice, Information, or Debug.

ProcedureTo Specify a Directory Path for Server Log Files

  1. Run the following configutil command:


    configutil -o logfile.service.logdir -v dirpath
    

ProcedureTo Specify a Maximum File Size for Each Service Log

  1. Run the following configutil command:


    configutil -o logfile.service.maxlogfilesize -v size
    

    where size specifies a number of bytes.

ProcedureTo Specify a Service Log Rotation Schedule

  1. Run the following configutil command:


    configutil -o logfile.service.rollovertime -v number
    

    where number specifies a number of seconds.

ProcedureTo Specify a Maximum Number of Service Log Files Per Directory

  1. Run the following configutil command:


    configutil -o logfile.service.maxlogfiles -v number
    

    where number specifies the maximum number of log files.

ProcedureTo Specify a Storage Limit

  1. Run the following configutil command:


    configutil -o logfile.service.maxlogsize -v number
    

    where number specifies a number in bytes.

ProcedureTo Specify the Minimum Amount of Free Disk Space to Reserve

  1. Run the following configutil command:


    configutil -o logfile.service.minfreediskspace -v number
    

    where number specifies a number in bytes.

25.4.5.2 To Specify an Age for Logs at Which They Expire


configutil -o logfile.service.expirytime -v number

where number specifies a number in seconds.

25.4.6 Using Message Tracing for Message Store Logging

You can use Message Store logging to trace messages by message ID in a way similar to how the MTA traces messages. Tracing messages in this fashion enables you to track the critical events of a message’s life cycle.

To trace messages in the Message Store logs, you need to configure message tracing in addition to the normal logging configuration. By default, message tracing is not enabled.


Note –

Message tracing will fill up a large amount of disk space. Do not enable this feature unless you have adequate disk space.


Message Store logging can track the following operations:

ProcedureTo Enable Message Tracing

  1. Run either the following configutil commands:


    configutil -o local.msgtrace.active -v “yes”

    Message trace information is written to the default log for each process. The IMAP fetches appear in the imap log file. The ims_master appends appear in the ims_master channel log file.


    configutil -o local.msgtrace.active -v “msgtrace”

    In this command message trace information for all processes is written to the msgtrace log file

ProcedureTo Configure LMTP Logging

  1. If you are using LMTP, and not using a single “msgtrace” log file, then you must also locally configure the tcp_lmtp_server log file. If you are not using LMTP, or are not using message trace, or are using message trace in the “msgtrace” log file, you do not need to initialize the LMTP Message Store side log. (LMTP already logs MTA information separately.) For example:


    configutil -o "local.logfile.tcp_lmtp_server.buffersize" -v "0"
    configutil -o "local.logfile.tcp_lmtp_server.expirytime" -v "604800"
    configutil -o "local.logfile.tcp_lmtp_server.flushinterval" -v "60"
    configutil -o "local.logfile.tcp_lmtp_server.logdir" -v \
       "/opt/SUNWmsgsr/data/log"
    configutil -o "local.logfile.tcp_lmtp_server.loglevel" -v "Information"
    configutil -o "local.logfile.tcp_lmtp_server.logtype" -v "NscpLog"
    configutil -o "local.logfile.tcp_lmtp_server.maxlogfiles" -v "10"
    configutil -o "local.logfile.tcp_lmtp_server.maxlogfilesize" -v "2097152"
    configutil -o "local.logfile.tcp_lmtp_server.maxlogsize" -v "20971520"
    configutil -o "local.logfile.tcp_lmtp_server.minfreediskspace" \
       -v "5242880"
    configutil -o "local.logfile.tcp_lmtp_server.rollovertime" -v "86400"
    
                      

25.4.7 Other Message Store Logging Features

Messaging Server provides a feature called telemetry that can capture a user’s entire IMAP or POP session into a file. This feature is useful for debugging client problems. For example, if a user complains that their message access client is not working as expected, this feature can be used to trace the interaction between the access client and Messaging Server. See 20.14.1.3 Check User IMAP/POP/Webmail Session by Using Telemetry

25.4.8 Message Store Logging Examples

The exact field format and list of fields logged in the Message Store log files vary according to the logging options set. This section shows a few examples of interpreting typical sorts of log entries.

25.4.8.1 Message Store Logging Example: Bad Password

When a user types an invalid password, “authentication” failure is logged, as opposed to a “user not found” message. The message “user not found” is the text passed to the client for security reasons, but the real reason (invalid password) is logged.


Example 25–11 Message Store Logging – Invalid Password


 [30/Aug/2004:16:53:05 -0700] vadar imapd[13027]: Account Notice: badlogin:
[192.18.126.64:40718] plaintext user1 authentication failure

25.4.8.2 Message Store Logging – Account Disabled

The following example shows why a user cannot log in due to a disabled account. Furthermore, the disabled account is clarified as “(inactive)” or “(hold).”


Example 25–12 Message Store Logging – Account Disabled


[30/Aug/2004:16:53:31 -0700] vadar imapd[13027]: Account Notice: badlogin: 
[192.18.126.64:40720] plaintext user3 account disabled (hold)

25.4.8.3 Message Store Logging Example: Message Appended

The following example shows an append message, which occurs when whenever a message is appended to a folder. The Message Store log records all messages entering the Message Store through the ims_master and lmtp channels. Records the “append” of user ID, folder, message size, and message ID.


Example 25–13 Message Store Logging – Append


[31/Aug/2004:16:33:14 -0700] vadar ims_master[13822]: Store Information:append:
user1:user/user1:659:<Roam.SIMC.2.0.6.1093995286.11265.user1@vadar.siroe.com>

25.4.8.4 Message Store Logging Example: Message Retrieved by a Client

The Message Store log writes a “fetch” message when a client retrieves a message. The Message Store log records all client fetches of at least one body part. Records the “fetch” of user ID, folder, and message-ID.


Example 25–14 Message Store Logging – Message Retrieved by a Client


[31/Aug/2004:15:55:26 -0700] vadar imapd[13729]: Store Information:
fetch:user1:user/user1:<Roam.SIMC.2.0.6.1093051161.3655.user1@vad.siroe.com>

25.4.8.5 Message Store Logging Example: Message Removed from a Folder


Example 25–15 Message Store Logging Example: Message Removed from a Folder

The Message Store writes an “expunge” message when an IMAP or POP message is removed from a folder (but not removed from the system). It is logged whether it is expunged by the user or a utility. Records the “expunge” of folder and message ID.


31/Aug/2004:16:57:36 -0700] vadar imexpire[13923]: Store Information:
expunge:user/user1:<Roam.SIMC.2.0.6.1090458838.2929.user1@vadar.siroe.com>

25.4.8.6 Message Store Logging Example: Duplicate Login Messages

If you configure message trace for one msgtrace log file, the normal “login” messages, which appear in the imap and pop log files, are duplicated in the msgtrace file. The following is a normal login message:


Example 25–16 Message Store Logging – Login


[30/Aug/2004:16:53:13 -0700] vadar imapd[13027]: Account Information: login
[192.18.126.64:40718] user1 plaintext