What's New in This Release of Messaging Server

The imsimta cache -change command allows certain job controller parameter changes to immediately take effect. The allowed formats of this command are:

• imsimta cache -cache -global -debug= integer

• imsimta cache -change -global -max_messages= integer

• imsimta cache -change -channel_template = name master_job = command

• imsimta cache -change -channel_template= name slave_job=command

• imsimta cache -change -channel= name master_job=command

• imsimta cache -change -channel= name slave_job=command

• imsimta cache -change -channel= name thread_depth=integer

• imsimta cache -change -channel= name job_limit=integer

Changing parameters for a channel template (such as tcp_* ) changes that parameter for all channels derived from that template.

The imsimta qm jobs command displays what messages are being processed by what jobs for what channels. Output might be in the following format:

channel <channel name>

job <pid>

host <host name>

host <host name>

<count of hosts> HOST BEING PROCESSED BY JOB <pid>

message <subdir/message name>

message <subdir/message name>

processed messages: <# messages successfully dequeued>

failed processing attempts: <# messages reenqueued>

<count of messages> MESSAGES BEING PROCESSES BY JOB <pid>

<count of jobs> JOBS ACTIVE FOR CHANNEL foo

<count of active channels> ACTIVE CHANNELS

The following input flags are now available in the FORWARD mapping. In the past they were only available to the various *_ACCESS mappings.

E - Incoming connection used ESMTP/EHLO.

L - Incoming connection used LMTP/LHLO.

F - NOTIFY=FAILURES active for this recipient.

S - NOTIFY=SUCCESSES active for this recipient.

D - NOTIFY=DELAYS active for this recipient.

A - SASL used to authenticate connection.

T - SSL/TLS used to secure connection.

The buffer used for spamfilter verdict destination strings has been increased in size from 256 to 1024 characters. This was done to accommodate the much longer verdict destination strings that Brightmail 6.0 can return.

Two new values now have meaning for the various SPAMFILTERx_OPTIONAL MTA options: 3 and 4. A value of 3 causes spam filter failures to accept the message but queue it to the reprocess channel for later processing. A value of 4 does the same thing but also logs the spam filter temporary failure to syslog.

The ability to log the amount of time a message has spent in the queue has been added to the MTA logging facility. A new option, LOG_QUEUE_TIME , enables this capability. Setting the option to 1 enables queue time logging, while the default value of 0 disables it. The queue time is logged as an integer value in seconds. It appears immediately after the application information string in non-XML format logs. The attribute name in XML formatted logs for this value is qt.

Source channel switching based on user or domain settings is now possible. There are three new settings: 

• A new channel keyword userswitchchannel must be present on the initial source channel for user channel switching to occur.

• A new MTA option LDAP_DOMAIN_ATTR_SOURCE_CHANNEL specifies the name of a domain-level attribute containing the name of the channel to switch to.

• A new MTA option LDAP_SOURCE_CHANNEL is a user-level attribute containing the name of the channel to switch to. Additionally, the channel being switched to must be set to allow channel switches, that is, it cannot be marked with the noswitchchannel keyword. Switching is done based on information returned by rewriting the MAIL FROM address. Note that MAIL FROM addresses are easily forged so this functionality should be used with extreme care.

List expansion in the context of the mgrpallowedbroadcaster LDAP attribute now includes all the attributes used to store email addresses (normally mail, mailAlternateAddress, and mailEquivalentAddress). Previously, only mail attributes were returned, making it impossible to send to lists restricted to their own members using alternate addresses.

The default for the GROUP_DN_TEMPLATE MTA option has been changed to "ldap:///$A??sub?mail=*". It used to be "ldap:///$A?mail?sub?mail=*".

The new MTA option LDAP_DOMAIN_ATTR_DEFAULT_MAILHOST specifies a domain-level attribute containing the default mail host for the domain. If set, and the attribute is present on the domain, the mailhost attribute is no longer required on user entries in the domain. This option currently has no default, but preferredmailhost is the logical attribute to use as long as some other, conflicting usage doesn't exist.

New channel keywords generatemessagehash, keepmessagehash, and deletemessagehash. The keyword generatemessagehash , if specified on a destination channel, inserts a Message-hash: header field into the message. The keyword keepmessagehash retains any existing Message-hash: field. The keyword deletemessagehash deletes any existing Message-hash: field. The keyword deletemessagehash is the default. The value placed in Message-Hash: fields is a hash of the message.

New MTA options control how the hash is generated: 

• MESSAGE_HASH_ALGORITHM - The hash algorithm. Can be any of "md2","md4", "md5" (the default), "sha1", "md128" (for RIPE-MD128), or "md160" (for RIPE-MD160).

• MESSAGE_HASH_FIELDS - Comma-separated list of fields from the header to hash (in order). Any known header field can be specified. If this option is not specified it defaults to "message-id,from,to,cc,resent-message-id,resent-from,resent-to,resent-cc,resent-bcc,subject,content-id,content-type,content-description."

The new MTA option UNIQUE_ID_TEMPLATE specifies a template used to convert an address into a unique identifier. The template's substitution vocabulary is the same as that for delivery options. The resulting unique identifier is intended for use by message archiving tools.

Per-user aliasdetourhost is now possible through the following set of features:

• aliasoptindetourhost channel keyword where detouring only occurs if the user has opted in via the following attribute.

• LDAP_DETOURHOST_OPTIN MTA option specifies the name of an attribute whose presence opts the user in to the detour (assuming of course the source channel has aliasoptindetourhost set).

• ALIASDETOURHOST_NULL_OPTIN MTA option is similar to SPAMFILTERx_NULL_OPTIN in that it specifies a special value which if used in the optin attribute is treated as the same as the attribute being omitted. The default value is "", which means that an empty attribute value is ignored.

Support for a new IP_ACCESS table has been added. This access mapping is consulted during SMTP client operations just prior to attempting to open connections to a remote server. The mapping probe has the following format: source-channel|address-count|address-current|ip-current|hostname

• source-channel is the channel the message is being dequeued from, address-count is the total number of IP addresses for the remote server, address-current is the index of the current IP address being tried.

• ip-current is the current IP address, and hostname is the symbolic name of the remote server.

The mapping can set the following flags: 

• $N - Immediately reject the message with an "invalid host/domain error." Any supplied text will be logged as the reason for rejection but will not be included in the DSN.

• $I - Skip the current IP without attempting to connect.

• $A - Replace the current IP address with the mapping result.

The ACCESS_ORCPT MTA option has been changed from a simple boolean (0 or 1) to a bit-encoded value. Bit 0 enables the addition of the ORCPT to all the various access mappings. Bits 1-4 (values 2-16), if set, selectively enable the addition to the ORIG_SEND_ACCESS, SEND_ACCESS, ORIG_MAIL_ACCESS, and MAIL_ACCESS mappings respectively.

The new ACCESS_COUNTS MTA option finds various types of recipient count information in the various recipient *_ACCESS mappings. ACCESS_COUNTS is bit-encoded in the same way as ACCESS_ORCPT . If set, it enables the addition of a set of counts to the end of the access mapping probe string. Currently, the format of the count addition is: RCPT-TO-count/total-recipient-count/ (Note the trailing slash.) All mappings using this information should be coded to ignore anything following the last slash or they may break without warning.

Support has been added for a new caption channel keyword. The caption channel keyword is similar to the existing description channel keyword in that it takes a quoted string as an argument that is intended for use in channel displays. The difference is presumably that a caption is shorter than a description. The Java Enterprise System Monitoring Framework needs both the caption as well as the description.

A new utility routine verifies domain-level Schema 1 and 2 information in the directory. This utility routine is accessible to users through the verify command in imsimta test -domain:

% imsimta test -domain

DOMAIN_MAP> verify

This utility verifies canonical domain settings for domains with overlapping user entries. For more information, see imsimta test -domain in Sun Java System Messaging Server 6.3 Administration Reference

Support for SMTP chunking (RFC 3030) has been added to both the SMTP client and server. This support is enabled by default. Four new channel keywords can be used to control whether or not chunking is allowed. 

They are: 

chunkingclient - enables client chunking support (default)

chunkingserver - enables server chunking support (default)

nochunkingclient - disables client chunking support

nochunkingserver - disables server chunking support

The log file action field has been extended to indicate whether or not chunking was used to transfer a given message. Specifically, a C will be appended if chunking is used. Note that ESMTP has to be used for chunking to work, so you'll typically see field values like EEC or DEC.

The ability to generate :addresses arguments to sieve vacation via an LDAP autoreply attribute has been added to Messaging Server. The new MTA option LDAP_AUTOREPLY_ADDRESSES provides the name of the attribute to use. This option has no value by default. The attribute can be multi-valued, with each value specifying a separate address to pass to the :addresses vacation parameter.

The new LDAP_DOMAIN_ATTR_CATCHALL_MAPPING can now be used to specify the name of a LDAP domain attribute. This option is not set by default. If set the option specifies the name of a mapping which is consulted when an address associated with the domain fails to match any user entries. The format of the mapping probe is the same as that of the forward mapping, and the USE_FORWARD_DATABASE MTA option controls the format of the probe of this mapping in the same way as the forward mapping. If the mapping sets the $Y metacharacter the resulting string will replace the address being processed.

The MTA now fetches the block limit associated with the envelope return address and will set RET=HDRS if no return policy is specified and the message size exceeds the block limit. This prevents nondelivery reports for large messages from being undeliverable themselves. No new options or settings are associated with this change.

The $E metacharacter in a mapping template indicates an exit after processing the current template. There are cases where it is desirable to exit immediately without interpreting the rest of the template. The $+1E metacharacter sequence now produces this behavior.

The restriction that the same attribute cannot be assigned to multiple slots and hence can have multiple semantics during alias expansion and address reversal. 

The internal separator character used to delimit multiple subject line tag additions has been changed from space to vertical bar. This makes it possible to add a tag containing spaces, as some spam filters want to do. This change effectively prevents vertical bars from being used in tags, but such usage is almost certainly nonexistent. 

The MIME specification prohibits the use of a content-transfer-encoding other than 7–bit, 8–bit, and binary on multipart or message/rfc822 parts. It has long been the case that some agents violate the specification and encode multi-parts and message/rfc822 objects. Accordingly, the MTA has code to accept such encodings and remove them. However, recently a different standards violation has shown up, one where a CTE field is present with a value of quoted-printable or base63 but the part isn't actually encoded. If the MTA tries to decode such a message the result is typically a blank message.

Messages with this problem have become sufficiently prevalent that two new pairs of channel keywords have been added to deal with the problem: interpretation of content-transfer-encoding fields on multiparts and message/rfc822 parts can be enabled or disabled. The first pair is interpretmultipartencoding and ignoremultipartencoding and the second is interpretmessageencoding and ignoremessageencoding. The defaults are interpretmultipartencoding and interpretmessageencoding.

Several additional error messages the SMTP server either returns or places in DSNs have been made configurable. The new options and their default values are: 

ERROR_TEXT_MAILFROMDNSVERIFY invalid/host-not-in-DNS return address not allowed ERROR_TEXT_INVALID_RETURN_ADDRESS invalid/unroutable return address not allowed ERROR_TEXT_UNKNOWN_RETURN_ADDRESS invalid/no-such-user return address ERROR_TEXT_ACCEPTED_RETURN_ADDRESS return address invalid/unroutable but accepted anyway ERROR_TEXT_SOURCE_SIEVE_ACCESS source channel sieve filter access error ERROR_TEXT_SOURCE_SIEVE_SYNTAX source channel sieve filter syntax error: ERROR_TEXT_SOURCE_SIEVE_AUTHORIZATION source channel sieve filter authorization error ERROR_TEXT_TRANSACTION_LIMIT_EXCEEDED number of transactions exceeds allowed maximum ERROR_TEXT_INSUFFICIENT_QUEUE_SPACE insufficient free queue space available ERROR_TEXT_TEMPORARY_WRITE_ERROR error writing message temporary file ERROR_TEXT_SMTP_LINES_TOO_LONG lines longer than SMTP allows encountered; message rejected ERROR_TEXT_UNNEGOTIATED_EIGHTBIT message contains unnegotiated 8–bit

Overly aggressive SMTP servers might issue a "5xy bad recipient" response to the first RCPT TO and disconnect immediately, a standards violation. Messaging Server treats the response as a temporary error and tries later, only to get the same result. To work around this server bug, Messaging Server will handle the one recipient as bad and requeue any remaining recipients for a later retry.

Two new actions are available to system sieves: addconversiontag and setconversiontag. Both accept a single argument: A string or list of conversion tags. The addconversiontag action adds the conversion tag(s) to the current list of tags while setconversiontag empties the existing list before adding the new ones. Note that these actions are performed late in sieve processing so setconversiontag can be used to undo all other conversion tag setting mechanisms.

The MTA option, INCLUDE_CONVERSIONTAG, has been added to selectively enable the inclusion of conversion tag information in various mapping probes. This is a bit-encoded value. The bits are assigned as follows: Bit Value Mapping

0 1 CHARSET_CONVERSION - added as ;TAG= field before ;CONVERT

1 2 CONVERSION - added as ;TAG= field before ;CONVERT

2 4 FORWARD - added just before current address (| delim)

3 8 ORIG_SEND_ACCESS - added at end of probe (| delim)

4 16 SEND_ACCESS - added at end of probe (| delim)

5 32 ORIG_MAIL_ACCESS - added at end of probe (| delim)

6 64 MAIL_ACCESS - added at end of probe (| delim)

These tags appear in the probe as a comma-separated list. 

The sieve envelope test now accepts "conversiontag" as an envelope field specifier value. The test checks the current list of tags, one at a time. Note that the :count modifier, if specified, allows checking of the number of active conversion tags. This type of envelope test is restricted to system sieves. Also note that this test only "sees" the set of tags that were present prior to sieve processing — the effects of setconversiontag and addconversiontag actions are not visible.

Metacharacter substitutions can now be specified in mgrpModerator , mgrpAllowedBroadcaster and mgrpDisallowedBroadcaster attributes. In particular, the various address-related metacharacter sequences ($A for the entire address, $U for the mailbox part, $D for the domain part) refer to the current envelope from address and can in some cases be used to limit the results returned by the URL to entries that are likely (or guaranteed) to match. This may make authorization checks much more efficient. The new MTA option PROCESS_SUBSTITUTIONS controls whether or not substitutions are performed. This is a bit-encoded value, with the bits defined as follows:

Bit Value 

0 1 Enables substitutions in mgrpDisallowedBroadcaster if set

1 2 Enables substitutions in mgrpAllowedBroadcaster if set

2 4 Enables substitutions in mgrpModerator if set

The PROCESS_SUBSTITUTIONS MTA option defaults to 0, meaning that all of these substitutions are disabled by default.

New MTA option LDAP_DOMAIN_ATTR_UPLEVEL. This option specifies the name of a domain-level attribute used to store a domain-specific uplevel value which overrides the value of the DOMAIN_UPLEVEL MTA option for this one domain. Currently only bits 0 and 2 (values 1 and 4) are used from this value; the other bits of DOMAIN_UPLEVEL remain in effect. Note that this attribute is only consulted if the domain is looked up. This means that setting bit 0 of this value to 1 for a domain won't make subdomains of the domain match unless bit 0 of DOMAIN_UPLEVEL is also set. As such, the way to get subdomain matching for some domains but not others is to set bit 0 of DOMAIN_UPLEVEL (this enabling subdomain matches for all domains) then clear bit 0 of the attribute for the domains where you don't want uplevel matching to occur.

Rewrite rules can now be used to override the default ALIAS_MAGIC setting. Specifically, a construct in the form $nT , where n is an appropriate value for the ALIAS_MAGIC MTA option, overrides the setting for the domain when the rule matches during alias expansion.