This chapter describes common tools, methods, and procedures for troubleshooting the Message Transfer Agent (MTA). It consists of the following sections:
A related topic, monitoring procedures can be found in Chapter 27, Monitoring Messaging Server
Prior to reading this chapter, you should review Chapters 5 through 10 in this guide and the MTA configuration and command-line utility chapters in the Sun Java System Messaging Server Administration Reference.
One of the first steps in troubleshooting the MTA is to determine where to begin the diagnosis. Depending on the problem, you might look for error messages in log files. In other situations, you might check all of the standard MTA processes, review the MTA configuration, or start and stop individual channels. Whatever approach you use, consider the following questions when troubleshooting the MTA:
Did configuration or environmental problems prevent messages from being accepted (for example, disk space or quota problems)?
Were MTA services such as the Dispatcher and the Job Controller present at the time the message entered the message queue?
Did network connectivity or routing problems cause messages to be stuck or misrouted on a remote system?
Did the problem occur before or after a message entered into the message queue?
This chapter will address these questions in the subsequent sections.
This section outlines standard troubleshooting procedures for the MTA. Follow these procedures if a problem does not generate an error message, if an error message does not provide enough diagnostic information, or if you want to perform general wellness checks, testing, and standard maintenance of the MTA.
Test your address configuration by using the imsimta test -rewrite utility. With this utility, you can test the MTA’s address rewriting and channel mapping without actually having to send a message. Refer to the MTA Command-line Utilities chapter in theChapter 2, Message Transfer Agent Command-line Utilities, in Sun Java System Messaging Server 6.3 Administration Reference for more information.
The utility will normally show address rewriting that will be applied as well as the channel to which messages will be queued. However, syntax errors in the MTA configuration will cause the utility to issue an error message. If the output is not what you expect, you may need to correct your configuration.
Check if messages are present in the MTA message queue directory, typically msg-svr-base/data/queue/. Use command-line utilities like imsimta qm to check for the presence of expected message files under the MTA message queue directory. For more information on imsimta qm, refer to the MTA command-line utilities chapter in theimsimta qm in Sun Java System Messaging Server 6.3 Administration Reference and 27.8.6 imsimta qm counters
If the imsimta test -rewrite output looks correct, check that messages are actually being placed in the MTA message queue subdirectories. To do so, enable message logging (For more information on MTA logging, see 25.3 Managing MTA Message and Connection Logs in the directory /msg-svr-base/log/. You can track a specific message by its message ID to ensure that it is being placed in the MTA message queue subdirectories. If you are unable to find the message, you may have a problem with file disk space or directory permissions.
msg-svr-base/data/queue/ msg-svr-base/data/log msg-svr-base/data/tmp
Commands, like the ones in the following UNIX system example, may be used to check the protection and ownership of these directories:
ls -l -p -d /opt/SUNWmsgsr/data/queue drwxr-x--- 2 mailsrv mail 512 Jan 4 16:09 /opt/SUNWmsgsr/data/queue/ ls -l -p -d /opt/SUNWmsgsr/data/log drwxr-x--- 2 mailsrv mail 3072 Feb 16 12:07 /opt/SUNWmsgsr/data/log/ ls -l -p -d /opt/SUNWmsgsr/data/tmp drwxr-x--- 2 mailsrv mail 512 Feb 16 12:55 /opt/SUNWmsgsr/data/tmp/
Check that the files in msg-svr-base/data/queue are owned by the MTA account by using commands like in the following UNIX system example:
ls -l -p -R /opt/SUNWmsgsr/data/queue
Some MTA channels, such as the MTA’s multi-threaded SMTP channels, include resident server processes that process incoming messages. These servers handle the slave (incoming) direction for the channel. The MTA Dispatcher handles the creation of such MTA servers. Dispatcher configuration options control the availability of the servers, the number of created servers, and how many connections each server can handle.
To check that the Job Controller and Dispatcher are present, and to see if there are MTA servers and processing jobs running, use the command imsimta process. Under idle conditions the command should result in job_controller and dispatcher processes. For example:
# imsimta process USER PID S VSZ RSS STIME TIME COMMAND mailsrv 9567 S 18416 9368 02:00:02 0:00 /opt/SUNWmsgsr/lib/tcp_smtp_server mailsrv 6573 S 18112 5720 Jul_13 0:00 /opt/SUNWmsgsr/lib/job_controller mailsrv 9568 S 18416 9432 02:00:02 0:00 /opt/SUNWmsgsr/lib/tcp_smtp_server mailsrv 6574 S 17848 5328 Jul_13 0:00 /opt/SUNWmsgsr/lib/dispatcher
If the Job Controller is not present, the files in the /msg-svr-base/data/queue directory will get backed up and messages will not be delivered. If you do not have a Dispatcher, then you will be unable to receive any SMTP connections.
For more information on imsimta process, refer to the imsimta process in Sun Java System Messaging Server 6.3 Administration Reference.
You could also use imsimta qm jobs to list, channel by channel, all active and pending delivery processing jobs currently being managed by the Job Controller. Additional cumulative information is provided for each channel such as the number of message files successfully delivered and those requeued for subsequent delivery attempts. The command syntax is as follows:
jobs [-[no]hosts] [-[no]jobs] [-[no]messages] [channel-name]
If neither the Job Controller nor the Dispatcher is present, you should review the dispatcher.log-* or job_controller.log-* file in /msg-svr-base/data/log
If the log files do not exist or do not indicate an error, start the processes by using the start-msg command. For more information, refer to the MTA command-line utilities chapter in the start-msg in Sun Java System Messaging Server 6.3 Administration Reference.
You should not see multiple instances of the Dispatcher or Job Controller when you run imsimta process, unless the system is in the process of forking (fork()) child processes before it executes (exec()) the program that needs to run. However, the time frame during such duplication is very small.
If MTA processing jobs run properly but messages stay in the message queue directory, you can examine the log files to see what is happening. All MTA log files are created in the directory /msg-svr-base/log. Log file name formats for various MTA processing jobs are shown in Table 26–1.Table 26–1 MTA Log Files
Log File Contents
Output of master program (usually client) for channel.
Output of slave program (usually server) for channel.
Dispatcher debugging. This log is created regardless if the Dispatcher DEBUG option is set. However, to get detailed debugging information, you should set the DEBUG option to a non-zero value.
ims-ms channel error messages when there is a problem in delivery.
Job controller logging. This log is created regardless if the Job Controller DEBUG option is set. However, to get detailed debugging information, you should set the DEBUG option to a non-zero value.
Debugging for the tcp_smtp_server. The information in this log is specific to the server, not to messages.
Debug output for the periodic MTA message bouncer job; this log file is created if the return_debug option is used in the option.dat
Each log file is created with a unique ID (uniqueid) to avoid overwriting an earlier log created by the same channel. To find a specific log file, you can use the imsimta view utility. You can also purge older log files by using the imsimta purge command. Note, however, that by default this command is run on a regular basis (see 4.6.2 Pre-defined Automatic Tasks). For more information, see the MTA command-line utilities chapter in theimsimta purge in Sun Java System Messaging Server 6.3 Administration Reference.
The channel_master.log-uniqueid and channel_slave.log-uniqueid log files will be created in any of the following situations:
There are errors in your current configuration.
The master_debug or slave_debug keywords are set on the channel in the imta.cnf file.
For more information on debugging channel master and slave programs, see the Sun Java System Messaging Server Administration Reference.
The command imsimta submit will notify the MTA Job Controller to run the channel. If debugging is enabled for the channel in question, imsimta submit will create a log file in directory /msg-svr-base/log as shown in Table 26–1.
The command imsimta run will perform outbound delivery for the channel under the currently active process, with output directed to your terminal. This may be more convenient than submitting a job, particularly if you suspect problems with job submission itself.
In order to manually run channels, the Job Controller must be running.
For information on syntax, options, parameters, examples of imsimta submit and imsimta run commands, refer to Command Descriptions in Sun Java System Messaging Server 6.3 Administration Reference.
In some cases, stopping and starting individual channels may make message queue problems easier to diagnose and debug. Stopping a message queue allows you to examine queued messages to determine the existence of loops or spam attacks.
Use the imsimta qm stop command to stop a specific channel. Doing so prevents you from having to stop the Job Controller and having to recompile the configuration. In the following example, the conversion channel is stopped:
To resume processing, use the imsimta qm start command to restart the channel. In the following example, the conversion channel is started:
imsimta qm start conversion
For more information on the imsimta qm start and imsimta qm stop commands, see imsimta qm in Sun Java System Messaging Server 6.3 Administration Reference.
The command imsimta qm start/stop channel may fail if run simultaneously for many channels at the same time. The tool might have trouble updating the hold_list and could report: QM-E-NOTSTOPPED, unable to stop the channel; cannot update the hold list. imsimta qm start/stop channel should only be used sequentially with a few seconds interval between each run.
If you only want the channel to run between certain hours, use the following options in the channel definition section in the job controller configuration file:
urgent_delivery=08:00-20:00 normal_delivery=08:00-20:00 nonurgent_delivery=08:00-20:00
You can run one of the following processes if you want to stop inbound message processing for a specific domain or IP address, while returning temporary SMTP errors to client hosts. By doing so, messages will not be held on your system. Refer to the 18.1 PART 1. MAPPING TABLES.
To stop inbound processing for a specific host or domain name, add the following access rule to the ORIG_SEND_ACCESS mapping table in the MTA mappings file (typically /msg-svr-base/config/mappings):
ORIG_SEND_ACCESS *|*@sesta.com|*|* $X4.2.1|$NHost$ temporarily$ blocked
By using this process, the sender’s remote MTA will hold messages on their systems, continuing to resend them periodically until you restart inbound processing.
To stop inbound processing for a specific IP address, add the following access rule to the PORT_ACCESS mapping table in the MTA mappings file (typically /msg-svr-base/config/mappings):
PORT_ACCESS TCP|*|25|IP_address_to_block|* $N500$ can't$ connect$ now
When you want to restart inbound processing from the domain or IP address, be sure to remove these rules from the mapping tables and recompile your configuration. In addition, you may want to create unique error messages for each mapping table. Doing so will enable you to determine which mapping table is being used.
This section explains how to troubleshoot a particular MTA problem step-by-step. In this example, a mail recipient did not receive an attachment to an email message. Note: In keeping with MIME protocol terminology, the “attachment” is referred to as a “message part” in this section. The aforementioned troubleshooting techniques are used to identify where and why the message part disappeared (See 26.2 Standard MTA Troubleshooting Procedures). By using the following steps, you can determine the path the message took through the MTA. In addition, you can determine if the message part disappeared before or after the message entered the message queue. To do so, you will need to manually stop and run channels, capturing the relevant files.
The Job Controller must be running when you manually run messages through the channels.
By identifying which channels are in the message path, you can apply the master_debug and slave_debug keywords to the appropriate channels. These keywords generate debugging output in the channels’ master and slave log files; in turn, the master and slave debugging information will assist in identifying the point where the message part disappeared.
Run imsimta cnbuild to recompile the configuration.
Run imsimta restart dispatcher to restart the SMTP server.
Have the end user resend the message with the message part.
Determine the channels that the message passes through.
While there are different approaches to identifying the channels, the following approach is recommended:
Once you find the message ID: header lines, look for the E (enqueue) and D (dequeue) records to determine the path of the message. Refer to 25.3.1 Understanding the MTA Log Entry Format for more information on logging entry codes. See the following E and D records for this example:
29-Aug-2001 10:39:46.44 tcp_local conversion E 2 ... 29-Aug-2001 10:39:46.44 conversion tcp_intranet E 2 ... 29-Aug-2001 10:39:46.44 tcp_intranet D 2 ...
The channel on the left is the source channel, and the channel on the right is the destination channel. In this example, the E and D records indicate that the message’s path went from the tcp_local channel to the conversion channel and finally to the tcp_intranet channel.
This section describes how to manually start and stop channels. See 26.2.7 Starting and Stopping Individual Channels starting and stopping the channels in the message’s path, you are able to save the message and log files at different stages in the MTA process. These files are later used to To Identify the Point of Message Breakdown.
Add the slave_debug and master_debug keywords to the appropriate channels in the imta.cnf file in directory /msg-svr-base/config.
Use the slave_debug keyword on the inbound channel (or any channel where the message is switched to during the initial dialog) from the remote system that is sending the message with the message part. In this example, the slave_debug keyword is added to the tcp_local channel.
Add the master_debug keyword to the other channels that the message passed through and were identified in 18.104.22.168 Identify the Channels in the Message Path would be added to the conversion and tcp_intranet channels.
Run the command imsimta restart dispatcher to restart the SMTP server.
Use the imsimta qm stop and imsimta qm start commands to manually start and stop specific channels. For more on information by using these keywords, see 26.2.7 Starting and Stopping Individual Channels.
To start the process of capturing the message files, have the end user resend the message with the message part.
When the message enters a channel, the message will stop in the channel if it has been stopped with the imsimta qm stop command. For more information, see Step Step 3.
Copy and rename the message file before you manually run the next channel in the message’s path. See the following UNIX platform example:
# cp ZZ01K7LXW76T7O9TD0TB.00 ZZ01K7LXW76T7O9TD0TB.KEEP1
The message file typically resides in directory similar to /msg-svr-base/data/queue/destination_channel/001. The destination_channel is the next channel that the message passes through (such as: tcp_intranet). If you want to create subdirectories (like 001, 002, and so on) in the destination_channel directory, add the subdirs keyword to the channels.
It is recommended that you number the extensions of the message each time you trap and copy the message in order to identify the order in which the message is processed.
Resume message processing in the channel and enqueue to the next destination channel in the message’s path. To do so, use the imsimta qm start command.
Copy and save the corresponding channel log file (for example: tcp_intranet_master.log-*) located in directory /msg-svr-base/log. Choose the appropriate log file that has the data for the message you are tracking. Make sure that the file you copy matches the timestamp and the subject header for the message as it comes into the channel. In the example of the tcp_intranet_master.log-*, you might save the file as tcp_intranet_master.keep so the file is not deleted.
Repeat steps 5 - 7 until the message has reached its final destination.
The log files you copied in Step Step 7 should correlate to the message files that you copied in Step Step 5. If, for example, you stopped all of the channels in the missing message part scenario, you would save the conversion_master.log-* and the tcp_intranet_master.log-* files. You would also save the source channel log file tcp_local_slave.log-*. In addition, you would save a copy of the corresponding message file from each destination channel: ZZ01K7LXW76T7O9TD0TB.KEEP1 from the conversion channel and ZZ01K7LXW76T7O9TD0TB.KEEP2 from the tcp_intranet channel.
Remove debugging options once the message and log files have been copied.
Remove the slave_debug and the master_debug keywords from the appropriate channels in the imta.cnf file in directory / msg-svr-base /config.
Reset the mm_debug=0, and remove log_message_id=1 in the option.dat file in directory / msg-svr-base /config.
Recompile the configuration by using imsimta cnbuild.
Run the command imsimta restart dispatcher to restart the SMTP server.
A tcp_local_slave.log-* file
A set of channel _master.log-* files for each destination channel
A set of mail.log_current records that show the path of the message
All files should have timestamps and message ID values that match the message ID: header lines in the mail.log_current records. Note that the exception is when messages are bounced back to the sender; these bounced messages will have a different message ID value than the original message.
Examine the tcp_local_slave.log-* file to determine if the message had the message part when it entered the message queue.
Look at the SMTP dialog and data to see what was sent from the client machine.
If the message part did not appear in the tcp_local_slave.log-* file, then the problem occurred before the message entered the MTA. As a result, the message was enqueued without the message part. If this the case, the problem could have occurred on the sender’s remote SMTP server or in the sender’s client machine.
Investigate the copies of the message files to see where the message part was altered or missing.
If any message file showed that the message part was altered or missing, examine the previous channel’s log file. For example, you should look at the conversion_master.log-* file if the message part in the message entering the tcp_intranet channel was altered or missing.
Look at the final destination of the message.
If the message part looks unaltered in the tcp_local_slave.log, the message files (for example: ZZ01K7LXW76T7O9TD0TB.KEEP1), and the channel_master.log-* files, then the MTA did not alter the message and the message part is disappearing at the next step in the path to its final destination.
If the final destination is the ims-ms channel (the Message Store), then you might download the message from the server to a client machine to determine if the message part is being dropped during or after this transfer. If the destination channel is a tcp_* channel, then you need to go to the MTA in the message’s path. Assuming it is an Messaging Server MTA, you will need to repeat the entire troubleshooting process (See 22.214.171.124 Identify the Channels in the Message Path, 126.96.36.199 Manually Start and Stop Channels to Gather Data, and this section). If the other MTA is not under your administration, then the user who reported the problem should contact that particular site.
This sections lists common problems and solutions for MTA configuration and operation.
If, during SMTP dialog, the STARTTLS command returns the following error:
454 4.7.1 TLS library initialization failure
and if you have certificates installed and working for pop/imap access, check the following:
Protections/ownerships of the certificates have to be set so mailsrv account can access the files
The directory where the certificates are stored need to have protections/ownerships set such that the mailsrv account can access the files within that directory.
After changing protections and installing certificates, you must run:
stop-msg dispatcher start-msg dispatcher
Restarting should work, but it is better to shut it down completely, install the certificates, and then start things back up.
Recompile the configuration (by running imsimta cnbuild).
Restart the appropriate processes (like imsimta restart dispatcher).
Re-establish any client connections.
Most MTA channels depend upon a slave or channel program to receive incoming messages. For some transport protocols that are supported by the MTA (like TCP/IP and UUCP), you need to make sure that the transport protocol activates the MTA slave program rather than its standard server. Replacing the native sendmail SMTP server with the MTA SMTP server is performed as a part of the Messaging Server installation.
For the multi-threaded SMTP server, the startup of the SMTP server is controlled by the Dispatcher. If the Dispatcher is configured to use a MIN_PROCS value greater than or equal to one for the SMTP service, then there should always be at least one SMTP server process running (and potentially more, according to the MAX_PROCS value for the SMTP service). The imsimta process command may be used to check for the presence of SMTP server processes. See imsimta process in Sun Java System Messaging Server 6.3 Administration Reference for more information.
If the dispatcher won’t start up, first check the dispatcher.log-* for relevant error messages. If the log indicates problems creating or accessing the /tmp/.SUNWmsgsr.dispatcher.socket file, then verify that the /tmp protections are set to 1777. This would show up in the permissions as follows:
drwxrwxrwt 8 root sys 734 Sep 17 12:14 tmp/ .
Also do an ls -l of the .SUNWmsgsr.dispatcher.socket file and confirm the proper ownership. For example, if this is created by root, then it is inaccessible by inetmail.
Do not remove the .SUNWmsgsr.dispatcher.file and do not create it if it’s missing. The dispatcher will create the file. If protections are not set to 1777, the dispatcher will not start or restart because it won’t be able to create/access the socket file. In addition, there may be other problems occurring not related to the Messaging Server.
Timeouts on incoming SMTP connections are most often related to system resources and their allocation. The following techniques can be used to identify the causes of timeouts on incoming SMTP connections:
Check how many simultaneous incoming SMTP connections you allow. This is controlled by the MAX_PROCS and MAX_CONNS Dispatcher settings for the SMTP service; the number of simultaneous connections allowed is MAX_PROCS*MAX_CONNS. If you can afford the system resources, consider raising this number if it is too low for your usage.
Another technique you can use is to open a TELNET session.
In the following example, the user connects to 127.0.0.1 port 25. Once connected, 220 banner is returned. For example:
telnet 127.0.0.1 25 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is ’^]’. 220 budgie.sesta.com --Server ESMTP (Sun Java System Messaging Server 6.1 (built May 7 2001))
If you are connected and receive a 220 banner, but additional commands (like ehlo and mail from) do not illicit a response, then you should run imsimta test -rewrite to ensure that the configuration is correct.
If the response time of the 220 banner is slow, and if running the pstack command on the SMTP server shows the following iii_res* functions (these functions indicate that a name resolution lookup is being performed):
febe2c04 iii_res_send (fb7f4564, 28, fb7f4de0, 400, fb7f458c, fb7f4564) + 42c febdfdcc iii_res_query (0, fb7f4564, c, fb7f4de0, 400, 7f) + 254
then it is likely that the host has to do reverse name resolution lookups, even on a common pair like localhost/127.0.0.1. To prevent such a performance slowdown, you should reorder your host’s lookups in the /etc/nsswitch.conf file. To do so, change the following line in the /etc/nsswitch.conf file from:
hosts: dns nis [NOTFOUND=return] files
hosts: files dns nis [NOTFOUND=return]
Making this change in the /etc/nsswitch.conf file can improve performance as fewer SMTP servers have to handle messages instead of multiple SMTP servers having to perform unnecessary lookups.
You can also put the slave_debug keyword on the channels handling incoming SMTP over TCP/IP mail, usually tcp_local and tcp_intranet. After doing so, review the most recent tcp_local_slave.log-uniqueid files to identify any particular characteristics of the messages that time out. For example, if incoming messages with large numbers of recipients are timing out, consider using the expandlimit keyword on the channel.
Remember that if your system is overloaded and overextended, timeouts will be difficult to avoid entirely.
Errors encountered during TCP/IP delivery are often transient; the MTA will generally retain messages when problems are encountered and retry them periodically. It is normal on large networks to experience periodic outages on certain hosts while other host connections work fine. To verify the problem, examine the log files for errors relating to delivery attempts. You may see error messages such as, “Fatal error from smtp_open.” Such errors are not uncommon and are usually associated with a transient network problem. To debug TCP/IP network problems, use utilities like PING, TRACEROUTE, and NSLOOKUP.
The following example shows the steps you might use to see why a message is sitting in the queue awaiting delivery to xtel.co.uk. To determine why the message is not being dequeued, you can recreate the steps the MTA uses to deliver SMTP mail on TCP/IP.
% nslookup -query=mx xtel.co.uk (Step 1) Server: LOCALHOST Address: 127.0.0.1 Non-authoritative answer: XTEL.CO.UK preference = 10, mail exchanger = nsfnet-relay.ac.uk (Step 2) % telnet nsfnet-relay.ac.uk 25 (Step 3) Trying... [188.8.131.52] telnet: Unable to connect to remote host: Connection refused
Use the NSLOOKUP utility to see what MX records, if any, exist for this host. If no MX records exist, then you should try connecting directly to the host. If MX records do exist, then you must connect to the designated MX relays. The MTA honors MX information preferentially, unless explicitly configured not to do so. See also 184.108.40.206 TCP/IP MX Record Support.
In this example, the DNS (Domain Name Service) returned the name of the designated MX relay for xtel.co.uk. This is the host to which the MTA will actually connect. If more than one MX relay is listed, the MTA will try each MX record in succession, with the lowest preference value tried first.
If you do have connectivity to the remote host, you should check if it is accepting inbound SMTP connections by using TELNET to the SMTP server port 25.
If you use TELNET without specifying the port, you will discover that the remote host accepts normal TELNET connections. This does not indicate that it accepts SMTP connections; many systems accept regular TELNET connections but refuse SMTP connections and vice versa. Consequently, you should always do your testing against the SMTP port.
In the previous example, the remote host is refusing connections to the SMTP port. This is why the MTA fails to deliver the message. The connection may be refused due to a misconfiguration of the remote host or some sort of resource exhaustion on the remote host. In this case, nothing can be done to locally to resolve the problem. Typically, you should let the MTA continue to retry the message.
If you are running Messaging Server on a TCP/IP network that does not use DNS, you can skip the first two steps. Instead, you can use TELNET to directly access the host in question. Be careful to use the same host name that the MTA would use. Look at the relevant log file from the MTA’s last attempt to determine the host name. If you are using host files, you should make sure that the host name information is correct. It is strongly recommended that you use DNS instead of host names.
Note that if you test connectivity to a TCP/IP host and encounter no problems using interactive tests, it is quite likely that the problem has simply been resolved since the MTA last tried to deliver the message. You can re-run the imsimta submit tcp_channel on the appropriate channel to see if messages are being dequeued.
In certain circumstances, a remote domain can break down and the volume of mail addressed to this server can be so great that the outgoing channel queue will fill up with messages that cannot be delivered. The MTA tries to redeliver these messages periodically (the frequency and number of the retries is configurable using the backoff keywords) and under normal circumstances, no action is needed. However, if too many messages get stuck in the queue, other messages may not get delivered in a timely manner because all the channel jobs are working to process the backlog of messages that cannot be delivered.
In this situation, you can reroute these messages to a new channel running in its own job controller pool. This will avoid contention for processing and allow the other channels to deliver their messages. This procedure is described below. We assume a domain called siroe.com
Create a new channel called tcp_siroe-daemon and add a new value for the pool keyword.
Channels are created in the channel block section of /msg-svr-base/config/imta.cnf. The channel should have the same channel keywords on your regular outgoing tcp_* channel. Typically, this is the tcp_local channel, which handles all outbound (internet) traffic. Since siroe.com is out on the internet, this is the channel to emulate. The new channel may look something like this:
tcp_siroe smtp nomx single_sys remotehost inner allowswitchchannel \ dentnonenumeric subdirs 20 maxjobs 7 pool SMTP_SIROE maytlsserver \ maysaslserver saslswitchchannel tcp_auth missingrecipientpolicy 0 \ tcp_siroe-daemon
Note the new keyword-value pair pool SMTP_SIROE. This specifies that messages to this channel will only use computer resources from the SMTP_SIROE pool. Note also that a blank line is required before and after the new channel.
Add two rewrite rules to the rewrite rule section of the imta.cnf file to direct email destined for siroe.com to the new channel.
The new rewrite rules look like this:
siroe.com $U%$D@tcp_siroe-daemon .siroe.com $U%$H$D@tcp_siroe-daemon
These rewrite rules will direct messages to siroe.com (including addresses like host1.siroe.com or hostA.host1.siroe.com) to the new channel whose official host name is tcp_siroe-daemon. The rewriting part of these rules, $U%$D and $U%$H$D, retain the original addresses of the messages. $U copies the user name from original address. % is the separator—the @ between the username and domain. $H copies the unmatched portion of host/domain specification at the left of dot in pattern. $D copies the portion of domain specification that matched.
Define a new job controller pool called SMTP_SIROE.
In /msg-svr-base/config/job_controller.cnf add the following:
This creates a message resource pool called SMTP_SIROE that allows up to 10 jobs to be simultaneously run. Be sure not to leave any blank lines between this pool definition and the others. See 8.7 The Job Controller for details on jobs and pools.
Restart the MTA.
Issue the commands: imsimta cnbuild;imsimta restart
This recompiles the configuration and restarts the job controller and dispatcher.
In this example, a large quantity of email from your internal users is destined for a particular remote site called siroe.com. For some reason, siroe.com, is temporarily unable to accept incoming SMTP connections and thus cannot deliver email. (This type of situation is not a rare occurence.)
As email destined for siroe.com comes in, the outgoing channel queue, typically tcp_local, will fill up with messages that cannot be delivered. The MTA tries to redeliver these messages periodically (the frequency and number of the retries is configurable using the backoff keywords) and under normal circumstances, no action is needed.
However, if too many messages get stuck in the queue, other messages may not get delivered in a timely manner because all the channel jobs are working to process the backlog of siroe.com messages. In this situation, you may wish reroute siroe.com messages to a new channel running in its own job controller pool (see 8.7 The Job Controller). This will allow the other channels to deliver their messages without having to contend for processing resources used by siroe.com messages. Creating a new channel to address this situation is described below.
The queue cache is not synchronized with the messages in the queue directories. Message files in the MTA queue subdirectories that are awaiting delivery are entered into an in-memory queue cache. When channel programs run, they consult this queue cache to determine which messages to deliver in their queues. There are circumstances where there are message files in the queue, but there is no corresponding queue cache entry.
The queue cache is normally synchronized every four hours. If required, you can manually resynchronize the cache by using the command imsimta cache -sync. Once synchronized, the channel programs will process the originally unprocessed messages after new messages are processed. If you want to change the default (4 hours), you should modify the job_controller.cnf file in directory msg-svr-base/config by adding sync_time=timeperiod where timeperiod reflects how often the queue cache is synchronized. Note that the timeperiod must be greater than 30 minutes. In the following example, the queue cache synchronization is modified to 2 hours by adding the sync_time=02:00 to the global defaults section of the job_controller.cnf:
! VERSION=5.0 !IMTA job controller configuration file ! !Global defaults tcp_port=27442 secret=N1Y9[HzQKW slave_command=NULL sync_time=02:00
You can run imsimta submit channel to clear out the backlog of messages after running imsimta cache -sync. It is important to note that clearing out the channel may take a long time if the backlog of messages is large (greater than 1000).
For summarized queue cache information, run imsimta qm -maint dir -database -total.
If after synchronizing the queue cache, messages are still not being delivered, you should restart the Job Controller. To do so, use the imsimta restart job_controller command.
Restarting the Job Controller will cause the message data structure to be rebuilt from the message queues on disk.
Restarting the Job Controller is a drastic step and should only be performed after all other avenues have been thoroughly exhausted.
Refer 8.7 The Job Controller for more information on the Job Controller.
Channel processing programs fail to run because they cannot create their processing log file. Check the access permissions, disk space and quotas.
If the MTA detects that a message is looping, that message will be sidelined as a .HELD file. See 220.127.116.11 Diagnosing and Cleaning up .HELD Messages. Certain cases can lead to message loops which the MTA can not detect.
The first step is to determine why the messages are looping. You should look at a copy of the problem message file while it is in the MTA queue area, MTA mail log entries (if you have the logging channel keyword enabled in your MTA configuration file for the channels in question) relating to the problem message, and MTA channel debug log files for the channels in question. Determining the From: and To: addresses for the problem message, seeing the Received: header lines, and seeing the message structure (type of encapsulation of the message contents), can all help pinpoint which sort of message loop case you are encountering.
Some of the more common cases include:
The MTA requires that the postmaster address be a functioning address that can receive email. If a message to the postmaster is looping, check that your configuration has a proper postmaster address pointing to an account that can receive messages.
Stripping of Received: header lines is preventing the MTA from detecting the message loop.
Normal detection of message loops is based on Received: header lines. If Received: header lines are being stripped (either explicitly on the MTA system itself, or on another system like a firewall), it can interfere with proper detection of message loops. In these scenarios, check that no undesired stripping of Received: header lines is occurring. Also, check for the underlying reason why the messages are looping. Possible reasons include: a problem in the assignment of system names or a system not configured to recognize a variant of its own name, a DNS problem, a lack of authoritative addressing information on the system in question, or a user address forwarding error.
Incorrect handling of notification messages by other messaging systems are generating reencapsulated messages in response to notification messages.
Internet standards require that notification messages (reports of messages being delivered, or messages bouncing) have an empty envelope From: address to prevent message loops. However, some messaging systems do not correctly handle such notification messages. When forwarding or bouncing notification messages, these messaging systems may insert a new envelope From: address. This can then lead to message loops. The solution is to fix the messaging system that is incorrectly handling the notification messages.
If the MTA detects a serious problem having to do with delivery of a message, the message is stored in a file with the suffix .HELD in /msg-svr-base/data/queue/channel. For example:
% ls ZZ0HXZ00G0EBRBCP.HELD ZZ0HY200C0O6LGHU.HELD ZZ0HYA006LP66O3H.HELD ZZ0HZ7003EOQSE37.HELD
.HELD files can occur due to three major reasons:
Looping messages. The MTA detected that the messages were looping via build-up of one or another sort of Received: header lines).
User or domain status set to hold. These are messages that are, by intent of the MTA administrator, intentionally being side-lined, typically while some maintenance procedure is being performed, (for example, while moving user mailboxes).
Suspicious messages. Messages that met some suspicion thresh hold and were held for later manual inspection by the MTA administrator. Messages can be .HELD due to exceeding a configured maximum number of envelope recipients (see the holdlimit channel keyword in 12.5.9 Expansion of Multiple Addresses), due to running the imsimta qclean in Sun Java System Messaging Server 6.3 Administration Reference, clean in Sun Java System Messaging Server 6.3 Administration Reference or hold in Sun Java System Messaging Server 6.3 Administration Reference commands based on some suspicion of the message(s) in question, or due to use of a hold action in a Sieve script.
Messages bouncing between servers or channels are said to be looping. Typically, a message loop occurs because each server or channel thinks the other is responsible for delivery of the message. Looping messages usually have a great many *Received: header lines. The Received: header lines will illustrate the exact path of the message loop. Look carefully at the host names and any recipient address information (for example, for recipientclauses or (ORCPT recipient)comments) appearing in such header lines. One cause of such message loops is user error.
For example, an end user may set an option to forward messages on two separate mail hosts to one another. On his sesta.com account, the end-user enables mail forwarding to his varrius.com account. And, forgetting that he has enabled this setting, he sets mail forwarding on his varrius.com account to his sesta.com account.
A loop can also occur with a faulty MTA configuration. For example, MTA Host X thinks that messages for mail.sesta.com go to Host Y. However, Host Y thinks that Host X should handle messages for mail.sesta.com; as a result, Host Y returns the mail to Host X.
In these cases, the message is ignored by the MTA and no further delivery is attempted. When such a problem occurs, look at the header lines in the message to determine which server or channel is bouncing the message. Fix the entry as needed.
Another common cause of message loops is the MTA receiving a message that was addressed to the MTA host using a network name that the MTA does not recognize (has not been configured to recognize) as one of its own names. The solution is to add the additional name to the list of names that your MTA recognizes as its own. Note that the MTA's thresh holds for determining that a message is looping are configurable; see the MAX_*RECEIVED_LINES option.dat options (Option File Format and Available Options in Sun Java System Messaging Server 6.3 Administration Reference). Also note that the MTA may optionally be configured--see the HELD_SNDOPR global MTA option--to generate a syslog notice whenever a message is forced into .HELD state due to exceeding such a thresh hold. If syslog messages of Received count exceeded; message held.are present, then you know that this is occurring.
You can resend the .HELD message by running release in Sun Java System Messaging Server 6.3 Administration Reference or following these steps:
Rename the .HELD extension to any 2 digit number other than 00. For example, .HELD to .06.
Before renaming the .HELD file, be sure that the message has stopped looping.
Run imsimta cache -sync. Running this command will update the cache.
Run imsimta submit channel or imsimta run channel.
It may be necessary to perform these steps multiple times, since the message may again be marked as .HELD, because the Received: header lines accumulate. If the problem still exists, the *.HELD file will be recreated under the same channel with as before. If the problem has been addressed, the messages will be dequeued and delivered.
If you determine that the messages can simply be deleted with no attempt to deliver them, see clean in Sun Java System Messaging Server 6.3 Administration Reference in the Sun Java System Messaging Server 6.3 Administration Reference.
Messages .HELD due to a user or domain status of hold--and only messages .HELD for such a reason--will normally be stored in the hold channel's queue area. That is, .HELD message files in the hold channel's queue area can be assumed to be .HELD due to user or domain status.
Messages .HELD due to some suspicious characteristic will of course exhibit that characteristic. The characteristic could be anything which the site has chosen to characterize as suspicious. MTA Administrators should stay aware of these configuration choices and actions. However, if you are not the only or original administrator of this MTA, then check the MTA configuration for any configured use of the holdlimit channel keyword (12.5.9 Expansion of Multiple Addresses), any use of the $H flag in address-based *_ACCESS mapping tables in the MTA mappings file, or any use of the hold action in any system Sieve file (the system level imta.filter file, or any channel level Sieve filters configured and named via use of sourcefilter or destinationfilter channel keywords; see 12.12.4 Specifying Mailbox Filter File Location); and ask any fellow MTA administrators about any manual command line message holds (through, for instance, an imsimta qm clean command) they might have recently performed. Note also that application of a Sieve filter hold action, whether from a system Sieve filter or from users' personal Sieve filters, may optionally be logged; see the LOG_FILTER global MTA option (Option File Format and Available Options in Sun Java System Messaging Server 6.3 Administration Reference) for more information.
Messages sent by the MTA are received in an encoded format. For example:
Date: Wed, 04 Jul 2001 11:59:56 -0700 (PDT) From: "Desdemona Vilalobos" <Desdemona@sesta.com> To: email@example.com Subject: test message with 8bit data MIME-Version: 1.0 Content-type: TEXT/PLAIN; CHARSET=ISO-8859-1 Content-transfer-encoding: QUOTED-PRINTABLE 2=00So are the Bo=F6tes Void and the Coal Sack the same?=
These messages appear unencoded when read with the MTA decoder command imsimta decode. Refer to the Sun Java System Messaging Server Administration Reference for more information.
The SMTP protocol only allows the transmission of ASCII characters (a seven-bit character set) as set forth by RFC 821. In fact, the unnegotiated transmission of eight-bit characters is illegal via SMTP, and it is known to cause a variety of problems with some SMTP servers. For example, SMTP servers can go into compute bound loops. Messages are sent over and over again. Eight-bit characters can crash SMTP servers. Finally, eight-bit character sets can wreak havoc with browsers and mailboxes that cannot handle eight-bit data.
An SMTP client used to only have three options when handling a message containing eight-bit data: return the message to the sender as undeliverable, encode the message, or send it in direct violation of RFC 821. But with the advent of MIME and the SMTP extensions, there are now standard encodings which may be used to encode eight-bit data by using the ASCII character set.
In the previous example, the recipient received an encoded message with a MIME content type of TEXT/PLAIN. The remote SMTP server (to which the MTA SMTP client transferred the message) did not support the transfer of eight-bit data. Since the original message contained eight-bit characters, the MTA had to encode the message.
This section includes information on the following SSR topics:
See also 18.15 To Debug User-level Filters.
To check the MTA’s user filters, use the command:
# imsimta test -rewrite -debug -filter user@domain
In the output, look for the following information:
mmc_open_url called to open ssrf:user@ims-ms URL with quotes stripped: ssrd: user@ims-ms Determined to be a SSRD URL. Identifier: user@ims-ms-daemon Filter successfully obtained.
In addition, you can add the slave_debug keyword to the tcp_local channel to see how a filter is applied. The results are displayed in the tcp_local_slave.log file. Be sure to add mm_debug=5 in the option.dat file in directory /msg-svr-base/config in order to get sufficient debugging information.
Error parsing filter expression:...
If the filter is good, then filter information will be at the end of the output.
If the filter is bad, then the following error will be at the end of the output:Address list error -- 4.7.1 Filter syntax error: firstname.lastname@example.org
Also, if the filter is bad, then the SMTP RCPT TO command will return a temporary error response code:
RCPT TO: user@domain 452 4.7.1 Filter syntax error
If users are experiencing delays when they send messages, it may be because disk input/output is reduced due to insufficiently sized message queue disks. When users press the SEND button on their email client, the MTA will not fully accept receipt of the message until the message has been committed to the message queue. Information on message queue sizing can be found
When the MTA fails to start, general error messages appear at the command line. In this section, common general error messages will be described and diagnosed.
To diagnose your own MTA configuration, use the imsimta test -rewrite -debug utility to examine your MTA’s address rewriting and channel mapping process. By using this utility allows you to check the configuration without actually sending a message. See 26.2.1 Check the MTA Configuration.
MTA subcomponents might also issue other error messages that are not described in this chapter. You should refer to the chapters on MTA command-line utilities and configuration in the Sun Java System Messaging Server Administration Reference and chapters 5 through 10 for more information on each subcomponent. This section includes the following types of errors:
An error in mm_init generally indicates an MTA configuration problem. If you run the imsimta test -rewrite utility, these errors will be displayed. Other utilities like imsimta cnbuild, a channel, a server, or a browser might also return such an error.
Commonly encountered mm_init errors include:
Two alias file entries have the same left hand side. You will need to find and eliminate the duplication. Look for an error message that says error line #XXX where XXX is a line number. You can fix the duplicated alias on the line.
Note that an extraneous blank line in the rewrite rules (upper portion) of your MTA configuration file (imta.cnf) causes the MTA to interpret the remainder of the configuration file as channel definitions. Make sure that the very first line of the file is not a blank. Since there are often multiple rewrite rules with the same pattern (left-hand side), this then causes MTA to interpret them as channel definitions with non-unique official host names. Check your MTA configuration for any channel definitions with duplicate official host names and for any improper blank lines in the upper (rewrite rules) portion of the file.
This message indicates that two mapping tables have the same name, and one of the duplicate mapping tables needs to be removed. However, formatting errors in the mapping file may cause the MTA to wrongly interpret something as a mapping table name. For example, failure to properly indent a mapping table entry will cause the MTA to think that the left hand side of the entry is actually a mapping table name. Check your mapping file for general form and check the mapping table names.
A blank line should precede and follow any line with a mapping table name. However, no blank lines should be interspersed among the entries of a mapping table.
This error means that a mapping table name is too long and needs to be shortened. Formatting errors in the mapping file may cause the MTA to wrongly interpret something as a mapping table name. For example, failure to properly indent a mapping table entry will cause the MTA to think that the left hand side of the entry is actually a mapping table name. Check your mapping file and mapping table names.
If you see this message, you need to recompile and reinstall your compiled character set tables through the command imsimta chbuild. See the imsimta chbuild in Sun Java System Messaging Server 6.3 Administration Reference for more information.
This error message generally means that you need to resize your MTA character set internal tables and then rebuild the compiled character set tables with the following commands:
imsimta chbuild -noimage -maximum -option imsimta chbuild
Verify that nothing else needs to be recompiled or restarted before making this change. Refer to imsimta chbuild in Sun Java System Messaging Server 6.3 Administration Reference for more information on imsimta chbuild.
This error indicates that a local host alias or proper name is too long (the optional right hand side in the second or subsequent names in a channel block). However, certain syntax errors earlier in the MTA configuration file (an extraneous blank line in the rewrite rules, for instance) may cause MTA to wrongly interpret something as a channel definition. Aside from checking the indicated line of the configuration file, also check above that line for other syntax errors. In particular, if the line in which MTA issues this error is intended as a rewrite rule, then be sure to check for extraneous blank lines above it.
This error indicates that a channel definition block is missing the required second line (the official host name line). See the chapters on MTA configuration and command-line utilities in the Sun Java System Messaging Server Administration Reference and Chapter 12, Configuring Channel Definitions for more information on channel definition blocks. A blank line is required before and after each channel definition block, but a blank line must not be present between the channel name and official host name lines of the channel definition. Also note that blank lines are not permitted in the rewrite rules portion of the MTA configuration file.
The official host name for a channel (second line of the channel definition block) is limited to 128 octets in length. If you are trying to use a longer official host name on a channel, shorten it to a place holder name, and then use a rewrite rule to match the longer name to the short official host name. You may see this scenario if you work with the l (local) channel host name. For example:
Original l Channel: !delivery channel to local /var/mail store l subdirs 20 viaaliasrequired maxjobs 7 pool LOCAL_POOL walleroo.pocofronitas.thisnameismuchtoolongandreallymakesnosensebutitisan example.monkey.gorilla.orangutan.antidisestablismentarianism.newt.salaman der.lizard.gecko.komododragon.com Create Place Holder: !delivery channel to local /var/mail store l subdirs 20 viaaliasrequired maxjobs 7 pool LOCAL_POOL newt Create Rewrite Rule: newt.salamander.lizard.gecko.komododragon.com $U%$D@newt
Note that when using the l (local) channel, you will need to use a REVERSE mapping table. Refer to the MTA configuration chapter in the Sun Java System Messaging Server Administration Reference for information on usage and syntax.
Certain syntax errors earlier in the MTA configuration file (for example, an extraneous blank line in the rewrite rules) may cause the MTA to wrongly interpret something as a channel definition. This could result in an intended rewrite rule being interpreted as an official host name. Besides checking the indicated line of the configuration file, also check above that line for other syntax errors. In particular, if the line on which the MTA issues this error is intended as a rewrite rule, be sure to check for extraneous blank lines above it.
One of the functions of the imsimta cnbuild utility is to compile MTA configuration information into an image that can be quickly loaded. The compiled format is quite rigidly defined and often changes substantially between different versions of the MTA. Minor changes might occur as part of patch releases.
When such changes occur, an internal version field is also changed so that incompatible formats can be detected. The MTA components will halt with the above error when an incompatible format is detected. The solution to this problem is to generate a new, compiled configuration with the command imsimta cnbuild.
It is also a good idea to use the imsimta restart command to restart any resident MTA server processes, so they can obtain updated configuration information.
To ensure proper operation, it is important to configure enough swap space on your messaging system. The amount of required swap space will vary depending on your configuration. A general tuning recommendation is that the amount of swap space should be at least three times the amount of main memory.
An error message such as the following indicates a lack of swap space:
jbc_channels: chan_execute : fork failed: Not enough space
You might see this error in the Job Controller log file. Other swap space errors will vary depending on your configuration.
Solaris systems: swap -s (at the time MTA processes are busy), ps -elf, or tail /var/adm/messages
HP-UX systems: swapinfo or tail /var/adm/syslog/syslog.log
In order to send a message, the MTA reads configuration files and creates message files in the MTA message queue directories. Configuration files must be readable by the MTA or any program written against the MTA’s SDKs. During installation, proper permissions are assigned to these files. The MTA utilities and procedures which create configuration files also assign permissions. If the files are protected by the system manager, other privileged user, or through some site-specific procedure, the MTA may not be able to read configuration information. This will result in “File open” errors or unpredictable behavior. The imsimta test -rewrite utility reports additional information when it encounters problems reading configuration files. See imsimta test in Sun Java System Messaging Server 6.3 Administration Reference.
If the MTA appears to function from privileged accounts but not from unprivileged accounts, then file permissions in the MTA table directory are likely the cause of the problem. Check the permissions on configuration files and their directories. See 26.2.3 Check the Ownership of Critical Files.
“File create” errors usually indicate a problem while creating a message file in an MTA message queue directory. See 26.2.2 Check the Message Queue Directories to diagnose file creation problems.
You may see this error when an address is provided to the MTA through a browser. Or, the error may be deferred and returned as part of an error return mail message. In both cases, this error message indicates that the MTA is not able to deliver mail to the specified host. To determine why the mail is not being sent to the specified host, you should follow these troubleshooting procedures:
Verify that the address in question is not misspelled, is not transcribed incorrectly, or does not use the name of a host or domain that no longer exists.
Run the address in question through the imsimta test -rewrite utility. If this utility also returns an “illegal host/domain” error on the address, then MTA has no rules in the imta.cnf file and related files to handle the address. Verify that you have configured MTA correctly, that you answered all configuration questions appropriately, and that you have kept your configuration information up to date.
If imsimta test -rewrite does not encounter an error on the address, then MTA is able to determine how to handle the address, but the network transport will not accept it. You can examine the appropriate log files from the delivery attempt for additional details. Transient network routing or name service errors should not result in returned error messages, though it is possible for badly misconfigured domain name servers to cause these problems.
If you are on the Internet, check that you have properly configured your TCP/IP channel to support MX record lookups. Many domain addresses are not directly accessible on the Internet and require that your mail system correctly resolve MX entries. If you are on the Internet and your TCP/IP is configured to support MX records, you should have configured the MTA to enable MX support; see TCP/IP Connection and DNS Lookup Support 12.4.3 TCP/IP Connection and DNS Lookup Support for more information. If your TCP/IP package is not configured to support MX record lookups, then you will not be able to reach MX-only domains.
Errors such as the following are not necessarily MTA errors: os_smtp_* errors like os_smtp_open, os_smtp_read, and os_smtp_write errors. These errors are generated when the MTA reports a problem encountered at the network layer. For example, an os_smtp_open error means that the network connection to the remote side could not be opened. The MTA may be configured to connect to an invalid system because of addressing errors or channel configuration errors. The os_smtp_* errors are commonly due to DNS or network connectivity problems, particularly if this was a previously working channel or address. os_smtp_read or os_smtp_write errors are usually an indication that the connection was aborted by the other side or due to network problems.
Network and DNS problems are often transient in nature. The occasional os_smtp_* error is usually nothing to be concerned about. However, if you are consistently seeing these errors, it may be an indication of an underlying network problem.
To obtain more information about a particular os_smtp_* error, enable debugging on the channel in question. Investigate the debug channel log file that will show details of the attempted SMTP dialogue. In particular, look at the timing of when a network problem occurred during the SMTP dialogue. The timing may suggest the type of network or remote side issue. In some cases, you may also want to perform network level debugging (for example, TCP/IP packet tracing) to determine what was sent or received.