Many of the new MTA features described in this section have been incorporated into the Messaging Server documentation. The features are listed here for completeness and to announce the new features.
(54) A new facility has been added to store information that previously would have gone in the general, forward, and reverse databases in the compiled configuration instead. A new MTA option, USE_TEXT_DATABASES, has been added to control this capability. This option is bit encoded. If bit 0 (value 1) is set the file IMTA_TABLE:general.txt is read as the MTA configuration is initialized and the information from that file replaces all uses of the general database. If bit 1 (value 2) is set the file IMTA_TABLE:reverse.txt is read and used in instead of the reverse database. Finally, if bit 2 (value 4) is set the file IMTA_TABLE:forward.txt is read and used instead of the forward database. The default value for this option is 0, which disables all use of text databases. Note that use of the text database option means that changes to the underlying files will only be seen after a cnbuild, and in the case of running processes, after a reload. Several additional MTA options can be used to set the initial size of the various text database tables: GENERAL_DATA_SIZE - Initial number of entries in the general text database. REVERSE_DATA_SIZE - Initial number of entries in the reverse text database. FORWARD_DATA_SIZE - Initial number of entries in the forward text database. The MTA stores the database template strings in string pool 3, so the STRING_POOL_SIZE_3 MTA option controls the initial allocation of space for this purpose. Note that these various options only control initial sizes; the various tables and arrays will resize automatically up to the maximum allowed size. The maximum string pool size in 6.2P8 and earlier is 10Mb, after 6.2P8 is has been increased to 50Mb. Up to 1 million entries are allowed in 6.2P8 and earlier, this has been increased to 2 million entries in later releases. (144) A new MTA option, USE_CANONICAL_RETURN, has been added. This option is bit-encoded with the various bits matching those of the USE_ORIG_RETURN option. Each place where the MTA performs a comparison operation against the envelope from (MAIL FROM) address has an assigned bit. If the bit in USE_CANONICAL_RETURN is clear normal rewriting is applied to the envelope from address prior to use. In particular rewriting from mailAlternateAddress attributes to mail attributes will be performed; mailEqvuialentAddress attributes won't be rewritten to the corresponding mail attribute. If, however, the bit is set, the corresponding address will be rewritten if it appears in a mailEquivalentAddress attribute. It should be noted that the bit USE_ORIG_RETURN will, if set, disable rewriting entirely. So setting a bit in USE_ORIG_RETURN makes the corresponding bit in USE_CANONICAL_RETURN a noop. Note that the various bits of USE_ORIG_RETURN don't appear to be documented at this time, so here's a list of them: Bit Value Usage 0 1 When set, use the original envelope From: address in ORIG_SEND_ACCESS mapping table probes 1 2 When set, use the original envelope From: address in SEND_ACCESS mapping table probes 2 4 When set, use the original envelope From: address in ORIG_MAIL_ACCESS mapping table probes 3 8 When set, use the original envelope From: address in MAIL_ACCESS mapping table probes 4 16 When set, use the original envelope From: address in mailing list [AUTH_LIST], [MODERATOR_LIST], [SASL_AUTH_LIST], and [SASL_MODERATOR_LIST] checks 5 32 When set, use the original envelope From: address in mailing list [CANT_LIST] and [SASL_CANT_LIST] checks 6 64 When set, use the original envelope From: address in mailing list [AUTH_MAPPING], [MODERATOR_MAPPING], [SASL_AUTH_MAPPING], and [SASL_MODERATOR_MAPPING] checks 7 128 When set, use the original envelope From: address in mailing list [CANT_MAPPING] and [SASL_CANT_MAPPING] checks 8 256 When set, use the original envelope From: address in mailing list [ORIGINATOR_REPLY] comparisons 9 512 When set, use the original envelope From: address in mailing list [DEFERRED_LIST], [DIRECT_LIST], [HOLD_LIST], and [NOHOLD_LIST] checks 10 1024 When set, use the original envelope From: address in mailing list [DEFERRED_MAPPING], [DIRECT_MAPPING], [HOLD_MAPPINGS], and [NOHOLD_MAPPING] checks 11 2048 When set, use the original envelope From: address in mailing list checks for whether the sender is the list moderator 12 4096 When set, use the original envelope From: address in mailing list LDAP_AUTH_DOMAIN LDAP attribute (e.g., mgrpAllowedDomain) checks 13 8192 When set, use the original envelope From: address in mailing list LDAP_CANT_DOMAIN LDAP attribute (e.g., mgrpDisallowedDomain) checks 14 16384 When set, use the original envelope From: address in mailing list LDAP_AUTH_URL LDAP attribute (e.g., mgrpAllowedBroadcaster) checks 15 32768 When set, use the original envelope From: address in mailing list LDAP_CANT_URL LDAP attribute (e.g., mgrpDisallowedBroadcaster) checks 16 65536 OBSOLETE. In Messaging Server 5.0 and Messaging Server 5.1, when set use the original envelope From: address in mailing list LDAP_MODERATOR_RFC822 comparisons; since as of Messaging Server 5.2 there is no longer any such global MTA option nor need for such an attribute (since the LDAP_MODERATOR_URL attribute value can, in fact, specify a mailto: URL pointing to an RFC 822 address), this bit no longer has any meaning. 17 131072 When set, use the original envelope From: address in mailing list LDAP_MODERATOR_URL LDAP attribute (e.g., mgrpModerator) comparisons 18 262144 When set, use the original envelope From: address in any source-specific FORWARD mapping tables probes 19 524288 When set, use the original envelope From: address in any source-specific FORWARD database probes Bit 0 is the least significant bit. (145) The SPAMFILTERn_OPTIONAL MTA options now accept two additional values: -2 and 2. -2 and 2 are the same as 0 and 1 respectively except that they also cause a syslog message to be sent in the event of a problem reported by the spam filter plugin. (146) Old-style mailing lists defined in the aliases file or aliases database now accept a nonpositional [capture] parameter. If used the [capture] parameter specifies a capture address with the same semantics as capture addresses specified by the LDAP_CAPTURE attribute applied to a user or group in LDAP. (147) The default value for the MISSING_RECIPIENT_POLICY MTA option has been changed from 2 (add envelope recipient list as a To: field) to 1 (ignore missing recipient condition). This brings Messaging Server in line with what RFC 2822 recommends. (148) Although it will rarely make sense to do so, the x_env_to keyword can now be used without also setting single on a channel. (149) The MTA now has the ability to process multiple different LDAP attributes with the same semantics. Note that this is not the same as processing of multiple values for the same attribute, which has always been supported. The handling attributes receive depends on the semantics of the attribute. The possible options are: (a) Multiple different attributes don't make sense and render the user entry invalid. In 6.2 and later this handling is the default for all attributes unless otherwise specified. (b) If multiple different attribute are specified one is chosen at random and used. LDAP_AUTOREPLY_SUBJECT, LDAP_AUTOREPLY_TEXT, and LDAP_AUTOREPLY_TEXT_INT all receive this handling in 6.2 only; in 6.3 and later they receive the handling described in item 153 below. 6.3 adds the LDAP_SPARE_3 and LDAP_PERSONAL_NAME attribute to this category. Note that this was how all attributes were handled prior to 6.2. (c) Multiple different attributes do make sense and should all be acted on. This handling is currently in effect for LDAP_CAPTURE, LDAP_ALIAS_ADDRESSES, LDAP_EQUIVALENCE_ADDRESSES and LDAP_DETOURHOST_OPTIN. Note that LDAP_DETOURHOST_OPTIN attribute was first added to Messaging Server in 6.3. (150) The MTA now has the ability to chose between multiple LDAP attributes and attribute values with different language tags and determine the correct value to use. The language tags in effect are compared against the preferred language information associated with the envelope from address. Currently the only attributes receiving this treatment are LDAP_AUTOREPLY_SUBJECT (normally mailAutoReplySubject), LDAP_AUTOREPLY_TEXT (normally mailAutoReplyText), LDAP_AUTOREPLY_TEXT_INT (normally mailAutoReplyTextInternal), LDAP_SPARE_4, LDAP_SPARE_5, LDAP_PREFIX_TEXT and LDAP_SUFFIX_TEXT. It is expected that each attribute value will have a different language tag value; if different values have the same tag value the choice between them will be essentially random. 151) The length of URLs that can be specified in a mapping URL lookup has been increased from 256 to 1024. The same increase also applies to expressions evaluated by mappings and mapping calls to other mappings. (152) A new MTA option, LOG_REASON, controls storage of error reason information in log records. Setting the option to 1 enables this storage, 0 (the default) disables it. This information, if present, appears just before diagnostic information in log records. (153) A :percent argument has been added to spamtest. If present it changes the range of the spamtest result from 0-10 to 0-100. See the Internet Draft draft-ietf-sieve-spamtestbis-05.txt for additional information on this change. (154) The SpamAssassin spam filter plugin's DEBUG option setting now accepts an integer value instead of a boolean 0 or 1. The larger the value the more debugging will be generated. In particular, a setting of 2 or greater reports exactly what was received from spamd. (155) The conversion mapping now allows a new "PREPROCESS" directive. If specified it allows charset conversions to be done on messages prior to sending them to the conversion channel. (156) The $. metacharacter sequence can now be used in a mapping or rewrite rule to establish a string which will be processed as the mapping entry result in the event of a temporary LDAP lookup failure. By default temporary LDAP failures cause the current mapping entry to fail. This is problematic in cases where different actions need to be taken depending on whether the LDAP lookup failed to find anything versus the directory server being unavailable or misconfigured. The temporary failure string is terminated by an unescaped ".". In the case of mappings once a failure string has been set using this construct it will remain set until current mapping processing is completed. Rewrite rules behave differently; a temporary failure string remains set only for the duration of the current rule. "$.." can be used to return to the default state where no temporary failure string is set and temporary LDAP failures cause mapping entry or rewrite rule failure. Note that all errors other than failure to match an entry in the directory are considered to be temporary errors; in general it isn't possible to distinguish between errors caused by incorrect LDAP URLs and errors caused by directory server configuration problems. (157) Setting the LOG_FORMAT MTA option to 4 now causes log entries to be written in an XML-compatible format. Entry log entry appears as a single XML element containing multiple attributes and no subelements. Three elements are currently defined, en for enqueue/dequeue entries, co for connection entries, and he for header entries. Enqueue/dequeue (en) elements can have the following attributes: ts - time stamp (always present) no - node name (present if LOG_NODE=1) pi - process id (present if LOG_PROCESS=1) sc - source channel (always present) dc - destination channel (always present) ac - action (always present) sz - size (always present) so - source address (always present) od - original destination address (always present) de - destination address (always present) de - destination address (always present) rf - recipient flags (present if LOG_NOTARY=1) fi - filename (present if LOG_FILENAME=1) ei - envelope id (present if LOG_ENVELOPE_ID=1) mi - message id (present if LOG_MESSAGE_ID=1) us - username (present if LOG_USERNAME=1) ss - source system (present if bit 0 of LOG_CONNECTION is set and source system information is available) se - sensitivity (present if LOG_SENSITIVITY=1) pr - priority (present if LOG_PRIORITY=1) in - intermediate address (present if LOG_INTERMEDIATE=1) ia - initial address (present if bit 0 of LOG_INTERMEDIATE is set and intermediate address information is available) fl - filter (present if LOG_FILTER=1 and filter information is available) re - reason (present if LOG_REASON=1 and reason string is set) di - diagnostic (present if diagnostic info available) tr - transport information (present if bit 5 of LOG_CONNECTION is set and transport information is available) ap - application information (present if bit 6 of LOG_CONNECTION is set and application information is available) Here is a sample en entry: en ts="2004-12-08T00:40:26.70" pi="0d3730.10.43" sc="tcp_local" dc="l" ac="E" sz="12" so="info-E8944AE8D033CB92C2241E@whittlesong.com" od="rfc822;ned+2Bcharsets@mauve.sun.com" de="email@example.com" rf="22" fi="/path/ZZ01LI4XPX0DTM00IKA8.00" ei="01LI4XPQR2EU00IKA8@mauve.sun.com" mi="<11a3b401c4dd01$7c1c1ee0$1906fad0@elara>" us="" ss="elara.whittlesong.com ([126.96.36.199])" in="firstname.lastname@example.org" ia="email@example.com" fl="spamfilter1:rvLiXh158xWdQKa9iJ0d7Q==, addheader, keep" Here is a sample co entry: co ts="2004-12-08T00:38:28.41" pi="1074b3.61.281" sc="tcp_local" dr="+" ac="O" tr="TCP|188.8.131.52|25|184.108.40.206|33469" ap="SMTP"/ Header (he) entries have the following attributes: ts - time stamp (always present, also used in en entries) no - node name (present if LOG_NODE=1, also used in en entries) pi - process id (present if LOG_PROCESS=1, also used in en entries) va - header line value (always present) Here is a sample he entry: he ts="2004-12-08T00:38:31.41" pi="1074b3.61.281" va="Subject: foo"/ (158b) Added list authorization policy values SMTP_AUTH_USED and AUTH_USED. These are similar in effect to the old SMTP_AUTH_REQUIRED and AUTH_REQ but unlike the old values do not require posters to authenticate. (159) Sieve errors are now logged as such in mail.log when LOG_FILTER is enabled. (160) The ALLOW_TRANSACTION_PER_SESSION limit kicked in one transaction too early; it now allows the specified number of transaction instead of one less. (161) The type of transport protocol in use (SMTP/ESMTP/LMTP) is now logged and made available to the various access mappings. In particular, two new modifier characters have been added to the set that can appear after an action indicator in the mail.log* files: E - An EHLO command was issued/accepted and therefore ESMTP was used L - LMTP was used Previously the only modifier characters that would appears were A (SASL authentication used) and S (TLS/SSL used). Additionally, the $E and $L flags respectively will be set as appropriate for the various *_ACCESS mappings. (162) Wildcards are now allowed in the strings used to match verdicts returned by spam filters. (163) imsimta encode now supports three new switches: -disposition=VALUE Sets the content-disposition to the specified VALUE -parameters=NAME=VALUE Specifies one or more additional content-type parameters and their values -dparameters=NAME=VALUE Specifies one or more additional content-disposition parameters and their values (164) Bit 4 (value 16) of the DOMAIN_UPLEVEL MTA option is now used to control whether address reversal rewriting is: (1) Skipped if the address is a mailEquivalentAddress (bit clear) (2) Performed only if the address is a mailAlternateAddress (bit set) (165) A value "/" given as an [envelope_from] nonpositional alias parameter, as an errors to positional alias parameter, or as a value of the mgrpErrorsTo LDAP attribute is now interpreted as a request to revert to using the original envelope from address for the incoming message while retaining mailing list semantics. This can be useful for setting up mailing lists that report all forms of list errors to the original sender. (166) The Job controller directory sweep is now more sophisticated. Instead of reading all the files in the queue directory in the order in which they are found, it reads several channel queue directories at once. This makes for much more reasonable behaviour on startup, restart, and after max_messages has been exceeded. The number of directories to be read at once is controlled by the job controller option Rebuild_Parallel_Channel. This can take any value between 1 and 100. The default is 12. (167) The sieve interpreter now keeps track of whether a response message was generated by a notify or vacation action and logs this information as needed. (168) Add the option Rebuild_In_Order parameter to the job_controller. If this is set to a non zero value, then on startup the job controller adds previously untried (ZZ*) messages to the delivery queue in creation order. Previous (and default) behavior is to add the messages in the order in which they are found on disk. There is a cost associated with recreating the queues in order. (169) Some additional reasons why a requested vacation response isn't sent are now logged. (170) Add the command imsimta cache -change command. This command allows certain job controller parameters to be changed on the fly. The allowed formats of this command are: imsimta cache -change -global -debug=<integer> imsimta cache -change -global -max_messages=<integer> imsimta cache -change -channel_template=<name> master_job=<command> imsimta cache -change -channel_template=<name> slave_job=<command> imsimta cache -change -channel=<name> master_job=<command> imsimta cache -change -channel=<name> slave_job=<command> imsimta cache -change -channel=<name> thread_depth=<integer> imsimta cache -change -channel=<name> job_limit=<integer> Changing parameters for a channel template (e.g. tcp_*) changes that parameter for all channels derived from that template. (171) Add the command imsimta qm jobs. This command displays what messages are being processed by what jobs for what channels. Typical output might be: channel <channel name> job <pid> host <host name> host <host name> <count of hosts> HOSTS BEING PROCESSED BY JOB <pid> message <subdir/message name> message <subdir/message name> processed messages: <# messages sucessfully dequeued> failed processing attempts: <# messages reenqueued> <count of messages> MESSAGES BEING PROCESSED BY JOB <pid> <count of jobs> JOBS ACTIVE FOR CHANNEL foo <count of active channels> ACTIVE CHANNELS In the past they were only available to the various *_ACCESS mappings. E - Incoming connection used ESMTP/EHLO. L - Incoming connection used LMTP/LHLO. F - NOTIFY=FAILURES active for this recipient. S - NOTIFY=SUCCESSES active for this recipient. D - NOTIFY=DELAYS active for this recipient. A - SASL used to authenticate connection. T - SSL/TLS used to secure connection. (174) The buffer used for spamfilter verdict destination strings has been increased in size from 256 to 1024 characters. This was done to accomodate the much longer verdict destination strings that Brightmail 6.0 can return. (175) Two new values now have meaning for the various SPAMFILTERx_OPTIONAL MTA options: 3 and 4. A value of 3 causes spamfilter failures to accept the message but queue it to the reprocess chanel for later processing. A value of 4 does the same thing but also logs the spam filter temporary failure to syslog. (176) The ability to log the amouint of time a message has spent in the queue has been added to the MTA logging facility. A new option, LOG_QUEUE_TIME, enables this capability. Setting the option to 1 enables queue time logging, while the default value of 0 disables it. The queue time is logged as an integer value in seconds. It appears immediately after the application information string in non-XML format logs. The attribute name in XML formatted logs for this value is "qt". (177) Source channel switching based on user or domain settings is now possible. There are three new settings involved: (a) A new channel keyword userswitchchannel. This keyword must be present on the initial source channel for user channel switching to occur. (b) A new MTA option LDAP_DOMAIN_ATTR_SOURCE_CHANNEL that specifies the name of a domain-level attribute containing the name of the channel to switch to. (c) A new MTA option LDAP_SOURCE_CHANNEL that specified is the name of a user-level attribute containing the name of the channel to switch to. Additionally, the channel being switched to must be set to allow channel switches, that is, it cannot be marked with the noswitchchannel keyword. Switching is done based on information returned by rewriting the MAIL FROM address. Note that MAIL FROM addresses are easily forged so this functionality should be used with extreme care. (178) List expansion in the context of the mgrpallowedbroadcaster LDAP attribute now includes all the attributes used to store email addresses (normally mail, mailAlternateAddress, and mailEquivalentAddress). Previously only mail attributes were returned, making it impossible to send to lists restricted to their own members using alternate addresses. (179) The default for the GROUP_DN_TEMPLATE MTA option has been changed to ""ldap:///$A??sub?mail=*". It used to be ""ldap:///$A?mail?sub?mail=*". This change makes the change described in item 178 work correctly in the case of lists defined using DNs. a domain-level attribute containing the default mailhost for the domain. If set and the attribute is present on the domain the mailhost attribute is no longer required on user entries in the domain. This option currently has no default, but preferredmailhost is the logical attribute to use as long as some other, conflicting usage doesn't exist. (181) New channel keywords generatemessagehash, keepmessagehash, and deletemessagehash. Generatemessage will, if specified on a destination channel, cause a Message-hash: header field to be inserted into the message. Keepmessagehash will cause any existing Message-hash: field to be retained. Deletemessagehash will delete any existing Message-hash: field. Deletemessagehash is the default. The value placed in Message-Hash: fields is (obviously) a hash of the message. Several new MTA options control how the hash is generated: MESSAGE_HASH_ALGORITHM - The hash algorithm. Can be any of "md2", "md4", "md5" (the default), "sha1", "md128" (for RIPE-MD128), or "md160" (for RIPE-MD160). MESSAGE_HASH_FIELDS - Comma separated list of fields from the header to hash (in order). Any known header field can be specified. If this option is not specified it defaults to "message-id,from,to,cc,bcc, resent-message-id,resent-from,resent-to,resent-cc,resent-bcc, subject,content-id,content-type,content-description". (182) New MTA option UNIQUE_ID_TEMPLATE. This option specifies a template used to convert an address into a unique identifier. The template's substitution vocabulary is the same as that for delivery options. The resulting unique identifier is intended for use by message archiving tools. (183) Per-user aliasdetourhost is now possible through the following set of features: (a) Added a aliasoptindetourhost channel keyword. This is similar in function to aliasdetourhost except detouring only occurs if the user has opted in via the following attribute. The keyword's value is a comma-separated list of potential detour hosts. (b) Added a LDAP_DETOURHOST_OPTIN MTA option, which specifies the name of an attribute used to opt the user in to the detour (assuming of course the source channel has aliasoptindetourhost set). If the values of this attribute contain periods they will be compared against the list of potential detour hosts and the first host on the list that matches will be the chosen detour. If the value doesn't contain a period the first detour host will be used unconditionally. (c) Added a ALIASDETOURHOST_NULL_OPTIN MTA option. This is similar to SPAMFILTERx_NULL_OPTIN - it specifies a "special" value which if used in the optin attribute is treated as the same as the attribute being omitted. The default valueis "", which means that an empty attribute value is ignored. (184) Support for a new IP_ACCESS table has been added. This access mapping is consulted during SMTP client operations just prior to attempting to open connections to a remote server. The mapping probe has the following format: source-channel|address-count|address-current|ip-current|hostname source-channel is the channel the message is being dequeued from, address-count is the total number of IP addresses for the remote server, address-current is the index of the current ip address being tried, ip-current is the current IP address, and hostname is the symbolic name of the remote server. The mapping can set the following flags: $N - Immediately reject the message with an "invalid host/domain error" Any supplied text will be logged as the reason for rejection but will not be included in the DSN. $I - Skip the current IP without attempting to connect. $A - Replace the current IP address with the mapping result. (185) The ACCESS_ORCPT MTA option has been changed from a simple boolean (0 or 1) to a bit-encoded value. Bit 0 (value 1) has the same effect it always had: It enables the addition of the ORCPT to all the various access mappings. Bits 1-4 (values 2-16), if set, selectivey enable the addition to the ORIG_SEND_ACCESS, SEND_ACCESS, ORIG_MAIL_ACCESS, and MAIL_ACCESS mappings respectively. (186) The new ACCESS_COUNTS MTA option provides a way to get at various types of recipient count information in the various recipient *_ACCESS mappings. ACCESS_COUNTS is bit-encoded in the same way as ACCESS_ORCPT now is (see the previous item for specifics) and if set enables the addition of a set of counts to the end of the access mapping probe string. Currently the format of the count addition is: RCPT-TO-count/total-recipient-count/ Note the trailing slash. It is expected that additional counter information will be added to this field in the future; all mappings making use of this information should be coded to ignore anything following the (current) last slash or they may break without warning. (187) Support for SMTP chunking (RFC 3030) has been added to both the SMTP client and server. This support is enabled by default. Four new channel keywords can be used to control whether or not chunking is allowed. They are chunkingclient - Enable client chunking support (default) chunkingserver - Enable server chunking support (default) nochunkingclient - Disable client chunking support nochunkingserver - DIsable server chunking support The log file action field has been extended to indicate whether or not chunking was used to transfer a given message. Specifically, a C will be appended if chunking is used. Note that ESMTP has to be used for chunking to work, so you'll typically see field values like "EEC" or "DEC". (188) Support has been added for a new caption channel keyword. This keyword is similar to the existing description channel keyword in that it takes a quoted string as an argument that is intended for use in channel displays. The difference is presumably that a "caption" is short than a "description". JES MF appears to need both. (189) A new utility routine has been written to verify domain-level Schema 1 and 2 information in the directory. This utilty routine is accessible to user through a new verify command in the imsimta test -domain program: % imsimta test -domain DOMAIN_MAP> verify Various checks are done by this utility, but the most important by far is verification of canonical domain settings for domains with overlapping user entries. The verification utility can return the following fatal errors: %DMAP-F-CANTGETDN, Cannot obtain DN of domain entry, directory error %DMAP-F-INTDEFERROR, Internal defined flag error on domain '%.*s', aborting %DMAP-F-INTHASHERROR, Internal hash error, aborting %DMAP-F-INTTREESTRUCTERROR, Internal tree structure error, aborting These are all indicative of an internal error in the verification code and should never occur. The following domain errors can be reported: %DMAP-E-ALIASTOOLONG, Domain alias '%s' in entry with DN '%s' is too long %DMAP-E-BASEDNTOOLONG, Base DN pointer '%s' in entry for domain '%.*s' is too long %DMAP-E-CANONICAL, Overlapping domains '%.*s' and '%.*s' defined by entries '%.*s' and '%.*s' have different canonical domains '%.*s' and '%.*s' %DMAP-E-CANONICALINVALID, Canonical domain '%.*s' defined/referenced by domain entry with DN '%.*s' is syntactically invalid %DMAP-E-CANONICALTOOLONG, Canonical name '%s' in entry for domain '%.*s' is too long %DMAP-E-CANTCONVDCDN, Cannot convert DN '%s' in DC tree to domain name %DMAP-E-CANTEXTALIAS, Empty alias pointer attribute in '%.*s' domain alias entry %DMAP-E-DOMAININVALID, Domain name '%.*s' defined/referenced by domain entry with DN '%.*s' is syntactically invalid %DMAP-E-DOMAINMULTDEF, Domain '%s' multiply defined by entries with DNs '%s' and '%s' %DMAP-E-DOMAINTOOLONG, Domain '%s' in entry with DN '%s' is too long %DMAP-E-DOMAINUNDEF, Domain name '%.*s' referenced by domain entry with DN '%.*s' never defined %DMAP-E-EMPTYCANONICAL, Domain '%.*s' has an empty canonical name %DMAP-E-INVALIDBASEDN, Base DN pointer '%.*s' in entry for domain '%.*s' is not a valid DN %DMAP-E-MULTICANONICAL, Multivalued canonical name in entry for domain '%.*s', used value '%s' ignored '%s' %DMAP-E-NOBASEDN, Domain '%.*s' has no base DN %DMAP-E-EMPTYBASEDN, Domain '%.*s' has an empty base DN %DMAP-E-NODOMAINNAME, Domain entry with DN '%s' does not have a domain name The following warnings can be reported: %DMAP-W-DISALLLOWEDATTR, Domain '%.*s' has a disallowed attribute '%s' with value '%s' %DMAP-W-DNTOOLONG, Domain entry DN '%s' is too long %DMAP-W-EMPAPPSTAT, Domain '%.*s' has an empty application status %DMAP-W-EMPDISALLLOWED, Domain '%.*s' has an empty disallowed attribute '%s' %DMAP-W-EMPDOMSTAT, Domain '%.*s' has an empty domain status %DMAP-W-EMPUIDSEP, Domain '%.*s' has an empty UID separator %DMAP-W-INVALIDAPPSTAT, Application status '%s' for domain '%.*s' is invalid %DMAP-W-INVALIDDOMSTAT, Domain status '%s' for domain '%.*s' is invalid %DMAP-W-INVALIDUIDSEP, UID separator '%s' for domain '%.*s' is invalid %DMAP-W-MULTDOMAINNAMES, Domain entry with DN '%s' has multiple domain names, used value '%s' ignored '%s' %DMAP-W-MULTIAPPSTAT, Multivalued application status in entry for domain '%.*s', used value '%s' ignored '%s' %DMAP-W-MULTIBASEDN, Multivalued base DN pointer in entry for domain '%.*s', used value '%s' ignored '%s' %DMAP-W-MULTIDOMSTAT, Multivalued domain status in entry for domain '%.*s', used value '%s' ignored '%s' %DMAP-W-MULTIUIDSEP, Multivalued UID separator in entry for domain '%.*s', used value '%s' ignored '%s' %DMAP-W-MULTIVALIAS, Multivalued alias pointer in entry for domain alias '%.*s', used value '%s' ignored '%s' %DMAP-W-NOBASEDNNODE, Base DN pointer '%.*s' in entry for domain '%.*s' doesn't point at anything %DMAP-W-NODOMAINNAME, Domain entry with DN '%s' has a blank domain alias %DMAP-W-NOENTRIES, No domain entries found, aborting Additional messages will undoubtedly be added to this list over time. (190) The ability to generate :addresses arguments to sieve vacation via an LDAP autoeply attribute has been added to Messaging Server. The new MTA option LDAP_AUTOREPLY_ADDRESSES provides the name of the attribute to use. This option has no value by default. The attribute can be multivalued, with each value specifying a separate address to pass to the :addresses vacation parameter. (191) The new LDAP_DOMAIN_ATTR_CATCHALL_MAPPING can now be used to specify the name of a LDAP domain attribute. This option is not set by default. If set the option specifies the name of a mapping which is consulted when an address associated with the domain fails to match any user entries. The format of the mapping probe is the same as that of the forward mapping, and the USE_FORWARD_DATABASE MTA option controls the format of the probe of this mapping in the same way as the forward mapping. If the mapping sets the $Y metacharacter the resulting string will replace the address being processed. (192) The MTA now fetches the block limit associated with the envelope return address and will set RET=HDRS if no return policy is specified and the message size exceeds the block limit. This prevents nondelivery reports for large messages from being undeliverable themselves. No new options or settings are associated with this change. (193) The $E metacharacter in a mapping template means "exit after processing the current template". There are cases where it is desireable to exit immediately without interpreting the rest of the template. The $+1E metacharacter sequence now produces this behavior. (194) Use of POP-before-SMTP via the MMP is now indicated in mail.log E records by the addition of a "P" to the action code. (195) Use of POP-before-SMTP can now be checked in the various *_ACCESS mappings (except PORT_ACCESS, which occurs before the necessary information has been communicated to the server), the FORWARD mapping, and any domain catchall mapping. The $P metacharacter flag is set if POP-before-SMTP is used. (196) The restriction that the same attribute cannot be assigned to multiple "slots" and hence can have multiple semantics during alias expansion and address reversal. (197) The internal separator character used to delimit multiple subject line tag additions has been changed from space to vertical bar. This makes it possible to add a tag containing spaces, as some spam filters want to do. This change effectively prevents vertical bars from being used in tags, but such usage is almost certainly nonexistant. (198) The MIME specification prohibits the use of a content-transfer-encoding other than 7bit, 8bit, and binary on multipart or message/rfc822 parts. It has long been the case that some agents violate the specification and encode multiparts and message/rfc822 objects. Accordingly, the Messaging Server MTA has code to accept such encodings and remove them. However, recently a different standards violation has shown up, one where a CTE field is present with a value of quoted-printable or base63 but the part isn't actually encoded! If the MTA tries to decode such a message the result is typically a blank messages, which is pretty much what you'd expect. Messages with this problem have become sufficiently prevalent that two new pairs of channel keywords have been added to deal with the problem - interpretation of content-transfer-encoding fields on multiparts and message/rfc822 parts can be enabled or disabled. The first pair is interpretmultipartencoding and ignoremultipartencoding and the second is interpretmessageencoding and ignoremessageencoding. The defaults are interpretmultipartencoding and interpretmessageencoding. (199) Several additional error messages the SMTP server either returns or places in DSNs have been made configurable. The new options and their default values are: ERROR_TEXT_MAILFROMDNSVERIFY invalid/host-not-in-DNS return address not allowed ERROR_TEXT_INVALID_RETURN_ADDRESS invalid/unroutable return address not allowed" ERROR_TEXT_UNKNOWN_RETURN_ADDRESS invalid/no-such-user return address ERROR_TEXT_ACCEPTED_RETURN_ADDRESS return address invalid/unroutable but accepted anyway ERROR_TEXT_SOURCE_SIEVE_ACCESS source channel sieve filter access error ERROR_TEXT_SOURCE_SIEVE_SYNTAX source channel sieve filter syntax error: ERROR_TEXT_SOURCE_SIEVE_AUTHORIZATION source channel sieve filter authorization error ERROR_TEXT_TRANSACTION_LIMIT_EXCEEDED number of transactions exceeds allowed maximum" ERROR_TEXT_INSUFFICIENT_QUEUE_SPACE insufficient free queue space available ERROR_TEXT_TEMPORARY_WRITE_ERROR error writing message temporary file ERROR_TEXT_SMTP_LINES_TOO_LONG lines longer than SMTP allows encountered; message rejected ERROR_TEXT_UNNEGOTIATED_EIGHTBIT message contains unnegotiated 8bit (200) We're seeing cases of overly agressive SMTP servers which will issue a "5xy bad recipient" response to the first RCPT TO and then disconnect immediately. (This is of course a flagrant standards violation.) The problem is Messaging Server treats this as a temporary error (which of course it is) and tries later, only to get the same result. A better thing to do which works around this server bug is to handle the one recipient as bad and requeue any remaining recipients for a later retry. (201) Two new actions are availabile to system sieves: addconversiontag and setconversiontag. Both accept a single argument: A string or list of conversion tags. Addconversiontag adds the conversion tag(s) to the current list of tags while setconversiontag empties the existing list before adding the new ones. Note that these actions are performed very late in the game so setconversiontag can be used to undo all other conversion tag setting mechanisms. (202) A new MTA option, INCLUDE_CONVERSIONTAG, has been added to selectively enable the inclusion of conversion tag information in various mapping probes. This is a bit-encoded value. The bits are assigned as follows: pos value mapping 0 1 CHARSET_CONVERSIOn - added as ;TAG= field before ;CONVERT 1 2 CONVERSION - added as ;TAG= field before ;CONVERT 2 4 FORWARD - added just before current address (| delim) 3 8 ORIG_SEND_ACCESS - added at end of probe (| delim) 4 16 SEND_ACCESS - added at end of probe (| delim) 5 32 ORIG_MAIL_ACCESS - added at end of probe (| delim) 6 64 MAIL_ACCESS - added at end of probe (| delim) In all cases the current set of tags appears in the probe as a comma separated list. (203) The sieve envelope test now accepts "conversiontag" as an envelope field specifier value. The test checks the current list of tags, one at a time. Note that the :count modifier, if specified, allows checking of the number of active conversion tags. This type of envelope test is restricted to system sieves. Also note that this test only "sees" the set of tags that were present prior to sieve processing - the effects of setconversiontag and addconversiontag actions are not visible. (204) Trailing dots on domains, e.g. "foo@bar.", are illegal in email but have been tolerated in some contexts by Messaging Server for a long time. RFC 1123 points out that trailing dots are syntactically illegal in email but notes that some convention needs to exist in user interfaces where short form names can be used. Accordingly, it may be handy in contexts like SMTP submission to be able to accept addresses with trailing dots, remove the dot while attaching special semantics to its presence. Accordingly, Messaging Server has modified in two ways: (1) Trailing dots are now accepted by the low-level address parser, making it possible to use them in context where they could not previously be used, like addresses inside of group constructs. (2) Trailing dots, when specified will cause a rewrite of the address with a trailing dot. If the rewrite with a trailing dot isn't found or otherwise fails rewriting will continue as before without the trailing dot. (205) Metacharacter substitutions can now be specified in mgrpModerator, mgrpAllowedBroadcaster and mgrpDisallowedBroadcaster attributes. In particular, the various address-related metacharacter sequences ($A for the entire address, $U for the mailbox part, $D for the domain part) refer to the current envelope from address and can in some cases be used to limit the results returned by the URL to entries that are likely (or guaranteed) to match. This may make authorization checks much more efficient. The new MTA option PROCESS_SUBSTITUTIONS controls whether or not substitutions are performed in various LDAP attributes that specify a URL. This is a bit-encoded value, with the bits defined as follows: Bit Value 0 1 Enables substitutions in mgrpDisallowedBroadcaster if set 1 2 Enables substitutions in mgrpAllowedBroadcaster if set 2 4 Enables substitutions in mgrpModerator if set 3 8 Enables substitutions in mgrpDeliverTo if set 4 16 Enables substitutions in memberURL The PROCESS_SUBSTITUTIONS MTA option defaults to 0, meaning that all of these substitutions are disabled by default. Note that the information available for substitution varies depending on whether the attribute is used for authorization checks or for actual list expansion. For authorization attributes the whole address ($A), domain ($D), host ($H), and local-part ($L) are all derived from the authenticated sender address. In the case of list expansion attributes all of these substitution values are derived from the envelope recipient address that specified the list. In both cases, however, the subaddress substitution ($S) is derived from the current envelope recipient address. The ability to access subaddress information in list expansion URLs makes it possible to define "metagroups", that is, a single group entry that in effect creates an entire collection of different groups. For example, a group with a mgrpDeliverTo value of: ldap:///o=usergroup?mail?sub?(department=$S) would make it possible to send mail to every member of a given department with an address of the form firstname.lastname@example.org. Note that a mechanism like a forward mapping could be used to alter the syntax if subaddresses are seen as too difficult. 206) New MTA option LDAP_DOMAIN_ATTR_UPLEVEL. This option specifies the name of a domain-level attribute used to store a domain-specific uplevel value which overrides the value of the DOMAIN_UPLEVEL MTA option for this one domain. Note that this attribute is only consulted if the domain is looked up. This means that setting bit 0 of this value to 1 for a domain won't make subdomains of the domain match unless bit 0 of DOMAIN_UPLEVEL is also set. As such, the way to get subdomain matching for some domains but not others is to set bit 0 of DOMAIN_UPLEVEL (this enabling subdomain matches for all domains) then clear bit 0 of the attribute for the domains where you don't want uplevel matching to occur. (207) Rewrite rules can now be used to override the default ALIAS_MAGIC setting. Specifically, a construct of the form $nT, where n is an appropriate value for the ALIAS_MAGIC MTA option, overrides the setting for the domain when the rule matches during alias expansion. ((208) $U in a PORT_ACCESS mapping template can now be used to selectively enable channel level debugging. (209) In 6.2 and earlier the PORT_ACCESS mapping was only reevaluated by the SMTP server (as opposed to the dispatcher) when bit 4 (value 16) of the LOG_CONNECTION MTA option is set, SMTP auth is enabled, or both. Additionally, evaluation only occurred when an AUTH, EHLO, or HELO command was issued. This has now been changed; PORT_ACCESS is now evaluated unconditionally as soon as the SMTP server thread starts, before the banner is sent. PORT_ACCESS may be reevaluated with different transport information when proxying from the MMP is used. (210) A useful spam-fighting strategy is to delay sending the SMTP banner for a brief time (half a second, say), then clear the input buffer, and finally send the banner. The reason this works is that many spam clients are not standards-compliant and start blasting SMTP commands as soon as the connection is open. Spam clients that do this when this capability is enabled will lose the first few commands in the SMTP dialogue, rendering the remainder of the dialogue invalid. This feature has now been implemented in Messaging Server. It can be enabled unconditionally by setting the BANNER_PURGE_DELAY SMTP channel option to the number of centiseconds to delay before purging and sending the banner. A value of 0 disabled both the delay and purge. The PORT_ACCESS mapping can also be used to control this capability. Specifying $D in the template causes an additional argument to be read from the template result, after the mandatory SMTP auth rulset and realm and optional application info addition. This value must be an integer with the same semantics as the BANNER_PURGE_DELAY value. Note that any PORT_ACCESS mapping setting overrides the BANNER_PURGE_DELAY SMTP channel option. (211) Added channel keywords acceptalladdresses and acceptvalidaddresses. Keyword acceptvalidaddresses is the default and corresponds to the MTA's standard behavior where any recipient errors are reported immediately during the SMTP dialogue. If the keyword acceptalladdresses is specified on a channel, then all recipient addresses are accepted during the SMTP dialogue. Any invalid addresses will have a DSN sent later. (212) Support has been added for postprocessing LDAP expansion results with a mapping. The new LDAP_URL_RESULT_MAPPING MTA option can be used to specify the name of a group attribute which in turn specifies the name of a mapping. This mapping will be applied to any results returned by expanding either a mgrpDeliverTo or memberURL attribute. The mapping probe will be of the form: LDAP-URL|LDAP-result If the mapping returns with $Y set the mapping result string will replace the LDAP result for alias processing purposes. If the mapping returns with $N set the result will be skipped. This mechanism can be used to define groups based on attributes that don't contain proper email address. For example, suppose a company has placed pager numbers in all their user entries. Messages can be sent to these numbers via email by suffixing them with a particular domain. A group could then be defined as follows: (a) Define a new mgrpURLResultMapping attribute in the directory and set the LDAP_URL_RESULT_MAPPING MTA option to this attribute's name. (b) Define a page-all group with the following attributes: mgrpDeliverto: ldap:///o=usergroup?pagerTelephoneNumber?sub mgrpURLResultMapping: PAGER-NUMBER-TO-ADDRESS (c) Define the mapping: PAGER-NUMBER-TO-ADDRESS *|* "$1"@pagerdomain.com$Y Even more interesting effects can be acheived by combining this mechanism with the PROCESS_SUBSTITUTION mechanism described in item 205 above. For example, it would be easy to create a metagroup where sending to an address of the form email@example.com sends a page to the user named "user". (213) Setting the LOG_QUEUE_TIME MTA option to 1 now causes an additional field to be selectively written to connection log records. This new field appears immediately after any diagnostic information and is labelled as "ct" in the XML-based log format. The value of this field is an integer count of the number of seconds that elapsed when performing the operation. So, for connection open ("O") records, the time shown is the number of seconds needed to open the connection. For connection close ("C") records it indicates the number of seconds the connection was open. For connection failure records ("Y") the value indicates the amount of time that was spent attempting to open the connection. (214) "S" transaction log entries now increment the various submitted message counters associated with the channel. (215) The $( metacharacter in a FROM_ACCESS specifies that an address should be read from the result string and used to replace the current overriding postmaster address. $) has the same effect with the added constraint that the overriding postmaster address must not be set prior to invoking the mapping. This allows for specific postmaster addresses to be used with addresses in nonlocal domains - domain postmaster addresses by definition only work with locally defined domains. The override address is (currently) the last string read from the FROM_ACCESS result prior to reading any $N/$F failure result. (216) The capture sieve action now has two optional nonpositional parameter: :dsn and :message. Only one of these can be specified in a single capture action. :dsn is the default, and encapsulates the captured message inside a special type of DSN. :message eliminates the enacapsulation and behaves more like a redirect. But unlike redirect, capture :message is only available to system sieves, always takes effect even when a more specific sieve specifies some other sort of action, and the envelope from address will be overridden with the address of the sieve owner. (217) The MTA now checks to make sure the UID attribute has a single value and reports an alias expansion error if it does not. The UID attribute is required to be single-valued in order to insure the user has a single, unique mailbox. (218) Two additional MTA options have been added to support more efficient domain lookups from user base DNs. They are: LDAP_BASEDN_FILTER_SCHEMA1 String specifying filter used to identify Schema 1 domains when performing baseDN searches. Default is the value of LDAP_DOMAIN_FILTER_SCHEMA1 if that MTA option is specified. If neither option is specified the default is "(objectclass=inetDomain)". LDAP_BASEDN_FILTER_SCHEMA2 String specifying additional filter elements used to identify Schema 2 domains when performing baseDN searches. Default is the value of LDAP_DOMAIN_FILTER_SCHEMA2 if that MTA option is specified. If neither option is specified the default is an empty string. (219) A new MTA option MESSAGE_SAVE_COPY_FLAGS has been added to control how the probes are constructed for the MESSAGE-SAVE-COPY mapping. If bit 0 (value 1) is set it adds the transport and application information to the beginning of the probe, if bit 1 (value 2) is set the original source channel is added, if bit 2 (value 4) is set the most recent conversion tag string is added. If all three bits are set the overall probe format is: transport|orig-source-channel|conversion-tags|queue-channel|return-address|D|filename (220) The LDAP_OPTIN1 through LDAP_OPTIN8 MTA options specify attributes for per-user optins to spam filtering based on destination addresses. There are now 8 new MTA options, LDAP_SOURCE_OPTIN1 through LDAP_SOURCE_OPTIN8, that provide comparable originator-address-based per-user spam filter optins. (221) Some additional switches have been added to imsimta test -rewrite: -saslused - Set internal flag indicating SASL authentication was used -tlsused - Set internal flag indication TLS is in use -esmtpused - Set internal flag indicating ESMTP is in use -lmtpused - Set internal flag indicating LMTP is in use -proxyused - Set internal flag indicating proxy authentication was used Only -saslused and -tlsused are available in 6.2; the other depend on other changes made in 6.3 and hence cannot be implemented in earlier versions. -lmtpused and -esmtpused cannot be set at the same time. -proxyused requires that -esmtpused or -lmtpused also be set. (222) New LMTP channel option MAILBOX_BUSY_FAST_RETRY. If set to 1 (the default) a 4.2.1 Mailbox busy error in response to LMTP message data is handled by retrying the message after a random but short interval; normal message backoff values do not apply. Setting the option to 0 disables this behavior.