Chapter 7 Address Translation and Routing Using Direct LDAP

Prior to release 6.0, Messaging Server used to access all user, domain, and group data from a database that was compiled from information stored in an LDAP server. When directory information was updated in the LDAP server, the database information was synchronized with a program called dirsync. The Sun™ ONE Messaging Server MTA now accesses the LDAP directory directly. This chapter describes the flow of data through the MTA using direct LDAP data access. It contains the following sections:

The Direct LDAP Algorithm and Implementation

Domain Locality Determination

Starting with an address of the form user@domain, the address translation and routing process first checks to see if “domain” is local.

Rewrite Rule Machinery

A facility has been added to the MTA rewrite rule machinery to check a given string to see if it is a domain we need to handle locally. This new facility is activated by a $V or $Z metacharacter. These new metacharacters are syntactically similar to the existing $N, $M, $Q, and $C metacharacters, that is, they are followed by a pattern string. In the case of $N, $M, $Q, and $C the pattern is matched against either the source or destination channel. In the case of $V and $Z the pattern is a domain and the check is to see whether or not it is local. $V causes a rule failure for a nonlocal domain and $Z causes a rule failure for a local domain.

Call dmap_locate_domain with the current domain as an argument. The domain is local if this call succeeds. Note that the actual result of the call doesn't matter; all we care about is that the domain is known to us.

If the base DN determination succeeds the attribute specified by the LDAP_DOMAIN_ATTR_ROUTING_HOSTS MTA option (default mailRoutingHosts) is retrieved. If this attribute is present, it lists the set of hosts able to handle users in this domain. This list is compared against the host specified by the local.hostname configutil parameter and the list of hosts specified by the local.imta.hostnamealiases configutil parameter. These options can be overridden by the LDAP_LOCAL_HOST and LDAP_HOST_ALIAS_LIST MTA options, respectively. If there is a match or the attribute is not present on the domain, the domain is local. If no match occurs, the domain is nonlocal.

The handling of domains considered to be nonlocal because of the mailRoutingHosts attribute depends on the setting of the ROUTE_TO_ROUTING_HOST MTA option. If the option is set to 0 (the default), the address is simply treated as nonlocal and MTA rewrite rules are used to determine routing. If the option is set to 1, a source route consisting of the first value listed in the LDAP_DOMAIN_ATTR_ROUTING_HOSTS MTA option is prepended to the address.

If the base DN determination fails, remove a component from the left hand side of the domain and go to Step 1. If no components remain continue to Step 4.

A consequence of this backtracking up the domain tree is that if domain.com is recognized as local, any subdomain of domain.com will be recognized as local. Situations may arise where this is undesirable, so an MTA option, DOMAIN_UPLEVEL, is provided to control this behavior. Specifically, bit 0 (value = 1) of DOMAIN_UPLEVEL, if clear, disables retries with domain components removed. The default value of DOMAIN_UPLEVEL is 0.

Vanity domain checking now needs to be performed. This is done by instituting an LDAP search using the LDAP URL specified by the DOMAIN_MATCH_URL MTA option. The value of this option should be set to:

Domain Map Determination of Domain Locality

It is also instructive to note what specific steps are performed by the domainMap_get_namespace or dmap_locate_domain call. The steps are schema-level-specific. In the case of Sun ONE LDAP Schema, v.1, they are:

The dmap_locate_domain call also supports Sun ONE LDAP Schema, v.2. In Sun ONE LDAP Schema, v.2, the action taken is much simpler: The directory is searched for an entry with the object class sunManagedOrganization where the domain appears as a value of either the sunPreferredDomain or associatedDomain attribute. If need be the use of the sunPreferredDomain and associatedDomain attributes for this purpose can be overridden with the respective MTA options LDAP_ATTR_DOMAIN1_SCHEMA2 and LDAP_ATTR_DOMAIN2_SCHEMA2. The search is done under the root specified by the service.dcroot configutil parameter. The service.dcroot configutil parameter can be overridden by setting the LDAP_DOMAIN_ROOT MTA option.

Caching Of Domain Locality Information

Due to the frequency with which domain rewrite operations are performed and the expense of the directory queries (especially the vanity domain check) both negative and positive indications about domains need to be cached. This is implemented with an in-memory open-chained dynamically-expanded hash table. The maximum size of the cache is set by the DOMAIN_MATCH_CACHE_SIZE MTA option (default 100000) and the timeout for entries in the cache is set by the DOMAIN_MATCH_CACHE_TIMEOUT MTA option (default 600 seconds).

Error Handling

Temporary server failures during this process have to be handled carefully, since when they occur, it is impossible to know whether or not a given domain is local. Basically two outcomes are possible in such a case:

Neither of these options is appropriate in all cases. For example, outcome 1 is appropriate when talking to a remote SMTP relay. But outcome 2 is appropriate when dealing with an SMTP submission from a local user.

While it would be possible in theory to handle temporary failures by using multiple rules with the same pattern, the overhead of repeating such queries, even with a cache in place, is unacceptable. For these reasons, the simple success/fall-through-to-the-next-rule matching model of domain rewriting is inadequate. Instead, a special template, specified by the MTA option DOMAIN_FAILURE, is used in the event of a domain lookup failure. When a $V operation fails, this template replaces the remainder of the current rewrite rule template being processed.

In addition to $V and $Z, several other new metacharacters have been added to the rewrite rule facility:

Pattern for Domain Check Rewrite Rule

This domain check needs to be performed before other rewrite rules have a chance to operate. This ordering is insured by using the special $* on the left hand side in the rule. The $* pattern is checked prior to any other rules.

Putting It All Together

Taking all the machinery described so far into account, the new rewrite rule we need in the imta.cnf is:

and the value of the DOMAIN_FAILURE MTA option in the option.dat file needs to be:

In this rewrite rule, localhost is the host name associated with the local channel. The value of the DOMAIN_FAILURE option shown here is the default value so it does not need to appear in option.dat under normal circumstances.

The ordering here is especially tricky. The MTA checks $V after the address is rebuilt but before the route is added. This lets the MTA to change the route in the event of a temporary lookup failure. Pending channel match checks are applied any time the insertion point changes, so the @ after the second $H invokes the check. If the check succeeds, the remainder of the template applies and rewrite processing concludes. If the check fails, the rewrite fails and rewriting continues with the next applicable rewrite rule. If the check cannot be performed due to a temporary failure, template processing continues with the value specified by the DOMAIN_FAILURE MTA option. The value of this template first sets the routing host to reprocess-daemon. Then the template checks to see whether or not the MTA is dealing with a reprocessing channel of some sort or tcp_local. If the MTA is dealing with such a channel, the rule continues, making the routing host illegal and specifying a temporary failure as the outcome. If the MTA is not dealing with such a channel, the rule is truncated and successfully terminated, thereby rewriting the address to the reprocess channel.

Alias expansion of local addresses

Once an address has been determined to be associated with the local channel it automatically undergoes alias expansion. The alias expansion process examines a number of sources of information, including:

The exact alias sources that are checked and the order in which they are checked depends on the setting of the ALIAS_MAGIC MTA option in the option.dat file. For direct LDAP, set the option to 8764. This means that the URL specified by the ALIAS_URL0 MTA option is checked first, then the URL specified by the ALIAS_URL1 MTA option, then the URL specific by the ALIAS_URL2 MTA option, and finally, the alias file. The alias database is not checked when this setting is active.

Alias Checking with LDAP URLs

Checking of aliases in LDAP is implemented by specifying two special LDAP URLs as alias URLs. The first of these handles regular users and groups; vanity domains are handled by subsequent alias URLs. The first URL is specified as ALIAS_URL0:

The $V Metacharacter

Metacharacter expansion occurs prior to URL lookup. The two metacharacters used in the ALIAS_URL0 value are $V and $R.

The $V metacharacter converts the domain part of the address into a base DN. This is similar to the initial steps performed by the $V rewrite rule metacharacter described previously in the section entitled “Rewrite Rule Machinery.” $V processing consists of the following steps:

$V also accepts an optional numeric argument. If it is set to 1 (for example, $1V), a failure to resolve the domain in the domain tree will be ignored and the base of the user tree will be returned.

If the attempt to retrieve the domain's base DN succeeds, the MTA also retrieves a several useful domain attributes which will be needed later. This is done using calls to domainMap_get_value. The names of the attributes retrieved are set by the following MTA options in the option.dat file:

Calling a Mapping from a URL

There might be unusual cases where the mapping from domain to base DN is done some other way. In order to accommodate such setups, the URL resolution process has the ability to call an MTA mapping. This is done with a metacharacter sequence of the general form:

The double quotation (“) initiates and terminates the callout. The character immediately following the $ is the separator between the mapping name and argument; a character should be chosen that does not collide with the expected character values used in either the mapping name or argument.

The $R Metacharacter

The $R metacharacter provides an appropriate filter for the URL. The intent is to produce a filter that searches all the attributes that might contain an email address for a particular user or group. The list of attributes to search comes from the configutil parameter local.imta.mailaliases. If this parameter is not set, the local.imta.schematag configutil parameter is examined, and depending on its value, an appropriate set of default attributes is chosen as follows:

The value of local.imta.schematag can be a comma-separated list. If more than one schema is supported, the combined list of attributes with duplicates eliminated is used. The LDAP_SCHEMATAG MTA option can be used to override the setting of local.imta.schematag specifically for the MTA.

Additionally, the filter searches not only for the address that was originally supplied, but also for an address with the same local part but the domain that was actually found in the domain tree, which was saved in Step 2 in the section entitled “The $V Metacharacter.” The iterative nature of the domain tree lookup means the two addresses may be different. This additional check is controlled by bit 1 (value = 2) of the DOMAIN_UPLEVEL MTA option in the option.dat file. Setting the bit enables the additional address check. The default value of DOMAIN_UPLEVEL is 0.

For example, suppose that the domain siroe.com appears in the domain tree. Assuming Sun ONE LDAP Schema, v.1 is in force, a lookup of the address

The filter that results from the expansion of $R and an ims50 schematag looks like:

(|(mail=u@siroe.com)
     (mail=u@host1.siroe.com)
     (mailAlternateAddress=u@siroe.com)
     (mailAlternateAddress=u@host1.siroe.com)
     (mailEquivalentAddress=u@siroe.com)
     (mailEquivalentAddress=u@host1.siroe.com))

If, on the other hand, DOMAIN_UPLEVEL was set to 1 rather than 3, the filter would be:

(|(mail=u@host1.siroe.com)
(mailAlternateAddress=u@host1.siroe.com)
(mailEquivalentAddress=u@host1.siroe.com))

The ALLOW_UNQUOTED_ADDRS_VIOLATE_RFC2798 MTA Option

Some MTAs have been exceptionally lax about address quoting and canonicalization. In particular, they would allow illegal addresses such as a..b@siroe.com. Even worse, they failed to add the necessary quotes to make the address into the legal “a..b”@siroe.com prior to searching for it in the directory.

The ALLOW_UNQUOTED_ADDRS_VIOLATE_RFC2798 MTA option in the option.dat file accommodates this particular violation of the standards. If set to 1 it will add additional filter terms to search on the syntactically invalid dequoted form of quoted addresses. For example, a search for the address "a..b"@siroe.com might produce a filter of the form:

(|(mail=”a..b”@siroe.com)
     (mail=a..b@siroe.com)
     (mailAlternateAddress=”a..b”@siroe.com)
     (mailAlternateAddress=a..b@siroe.com)
     (mailEquivalentAddress=”a..b”@siroe.com)
     (mailEquivalentAddress=a..b@siroe.com))

This option does not solve the problems caused by the use of illegal addresses. The relevant email and directory standards are specific about what is allowed in the local part of an address. When various messaging components are presented with syntactically illegal addresses, they may behave in different ways. They may quote such addresses to make them legal, they may pass them through unchanged, they may reject them, or they may do something entirely unexpected. Therefore, if end users are given such an illegal address they are likely to find it does not work when it hits other messaging systems provided by other vendors.

Determining the Attributes to Fetch

If the URL specifies an * for the list of attributes to return, we replace the asterisk with the list of attributes the MTA is able to use.

Handling LDAP Errors

At this point the resulting URL is used to perform an LDAP search. If an LDAP error of some kind occurs, processing terminates with a temporary failure indication (4xx error in SMTP). If the LDAP operation succeeds but fails to produce a result, the catchall address attribute for the domain retrieved from the LDAP_DOMAIN_ATTR_CATCHALL_ADDRESS MTA option is checked. If it is set, its value replaces the current address.

If no catchall address attribute is set the smarthost attribute for the domain retrieved from the LDAP_DOMAIN_ATTR_SMARTHOST MTA option is checked. If it is set, an address of the form

is created and alias processing terminates successfully with this result. Additionally, the conversion tag for the domain obtained from the LDAP_DOMAIN_ATTR_CONVERSION_TAG MTA option will, if present, be attached to the address so that conversions can be done prior to forwarding to the smarthost. If no catchall address or smarthost exists for the domain, processing of this alias URL terminates unsuccessfully.

Sanity Checks on the LDAP Result

After the LDAP search has returned a result, it is checked to verify that there is only one entry in it. If there are more than one, each entry is checked to see if it has the right object class for a user or a group, a non-deleted status, and for users, a UID. Entries that do not pass this check are ignored. If the list of multiple entries is reduced to one by this check, processing proceeds. If not, a duplicate or ambiguous directory error is returned.

Support for Vanity Domains

The ALIAS_URL0 check is for a conventional user or a user in a hosted domain. If this fails a vanity domain check is also made. This is done with the following alias URL:

Support for Catchall Addresses

Finally, a check for a catchall address of the form @host needs to be made in the mailAlternateAddress attribute. This form of wildcarding is allowed in both hosted and vanity domains, so the proper alias URL for it is:

Processing the LDAP Result

LDAP alias result processing is done in a number of order-dependent stages. These stages are described in the following sections.

Object Class Check

If the alias search succeeds, the object class of the entry is checked to make sure it contains an appropriate set of object classes for a user or a group. The possible sets of required object classes for users and groups is normally determined by what schemata are active. This is determined by the local.imta.schematag setting.

Table 7-1 shows the user and group object classes that result from various schematag values.

Table 7-1 Object Classes Resulting from Various schematag Values
schematag	User Object Classes	Group Object Classes
sims40	inetMailRouting+inetmailuser	inetMailRouting+inetmailgroup
nms41	mailRecipient + nsMessagingServerUser	mailGroup
ims50	inetLocalMailRecipient+inetmailuser	inetLocalMailRecipient+inetmailgroup

The information in this table, like the rest of schema tag handling, is hard coded. However, there are also two MTA options in the option.dat file, LDAP_USER_OBJECT_CLASSES and LDAP_GROUP_OBJECT_CLASSES, which can be set to specify different sets of object classes for users and groups respectively.

For example, a schema tag setting of ims50,nms41 would be equivalent to the following option settings:

LDAP_USER_OBJECT_CLASSES=inetLocalMailRecipient+inetmailuser, mailRecipient+nsMessagingServerUser

The LDAP result is simply ignored if it does not have a correct set of object classes appropriate for a user or a group. The MTA also determines if it is dealing with a user or a group and saves this information. This saved information will be used repeatedly later.

Note that the object class settings described here are also used to construct an actual LDAP search filter that can be used to check to see that an entry has the right object classes for a user or a group. This filter is accessible through the $K metacharacter. It is also stored internally in the MTA's configuration for use by channel programs and is written to the MTA option file, option.dat, as the LDAP_UG_FILTER option when the command imsimta cnbuild -option is used. This option is only written to the file. The MTA never reads it from the option file.

Entry Status Checks

Next the entry's status is checked. There are two status attributes, one for the entry in general and another specifically for mail service.

Table 7-2 shows the general and mail-specific user or group attributes in the schematag entry to check against depending on what schemata are in effect

Table 7-2 Attributes to Check Against
schematag	Type	General	Mail-specific
sims40	users	inetsubscriberstatus	mailuserstatus
sims40	groups	none	inetmailgroupstatus
nms41	users	none	mailuserstatus
nms41	groups	none	none
Messaging Server 5.0	users	inetuserstatus	mailuserstatus
Messaging Server 5.0	groups	none	inetmailgroupstatus

If necessary the LDAP_USER_STATUS and LDAP_GROUP_STATUS MTA options in the option.dat file can be used to select alternate general status attributes for users and groups respectively. The mail-specific user and group status attributes are controlled by the LDAP_USER_MAIL_STATUS and LDAP_GROUP_MAIL_STATUS MTA options.

Another factor that plays into this are the statuses for the domain itself (LDAP_DOMAIN_ATTR_STATUS and LDAP_DOMAIN_ATTR_MAIL_STATUS). All in all there are four status attributes. They are combined by considering them in the following order:

The first of these that specifies something other than “active” takes precedence over all the others. The other permissible status values are “inactive,” “deleted,” “removed,” “disabled,” “hold,” and “overquota.” “Hold,” “disabled,” and “removed” statuses may only be specified for mail domains, mail users, or mail groups. “Overquota” status can only be specified as a mail domain or mail user status.

All statuses default to “active” if a particular status attribute is not present. Unknown status values are interpreted as “inactive.”

When the four statuses are combined, the following statuses for a user or group are possible: “active,” “inactive,” “deleted,” “removed,” “disabled,” “hold,” and “overquota.” Active status causes alias processing to continue. Inactive or overquota status results in immediate rejection of the address with a 4xx (temporary) error. Deleted, removed, and disabled statuses results in immediate rejection of the address with a 5xx (permanent) error. Hold status is treated as active as far as status handling is concerned but it sets an internal flag so that when delivery options are considered later on, any options that are there are overridden with an option list containing a single “hold” entry.

UID Check

The next step is to consider the entry's UID. UIDs are used for a variety of purposes and must be part of all user entries and may be included in group entries. A user entry without a UID is ignored and processing of this alias URL terminates unsuccessfully. UIDs for entries in a hosted domain can consist of the real UID, a separator character, and then a domain. The MTA only wants the real UID so the rest is removed if present using the domain separator character obtained with the LDAP_DOMAIN_ATTR_UID_SEPARATOR MTA option in the option.dat file.

In the unlikely event that an attribute other than uid is used to store UIDs, the LDAP_UID MTA option can be used to force use of a different attribute.

Message Capture

Next the LDAP attribute used to specify one or more message capture addresses is checked. The attribute used for this purpose must be specified with the LDAP_CAPTURE MTA option. There is no default. Values of this attribute are treated as addresses and a special “capture” notification is generated and sent to these address containing the current message as an attachment. Additionally, the capture addresses are used to seed the address reversal cache in the likely event the address will subsequently appear as an envelope from: address.

Seeding the Reversal Cache

Next the primary address and any aliases attached to the user entry are considered. This information is used to seed the address reversal cache. It plays no part in the current address translation process. First, the primary address, personal name, recipient limit, recipient cutoff, and source block limit attributes are considered. The primary address is normally stored in the “mail” attribute; another attribute can be specified by setting the LDAP_PRIMARY_ADDRESS MTA option appropriately. (The primary address reverses to itself, of course.) There is no default attribute for any of the other attributes. If you want to use them, you must specify them with the LDAP_PERSONAL_NAME, LDAP_RECIPIENTLIMIT, LDAP_RECIPIENTCUTOFF, and LDAP_SOURCEBLOCKLIMIT MTA options. The corresponding domain-level recipient limit, recipient cutoff, and source block limit attributes are also considered at this point. User-level settings completely override any domain-level setting.

Next, any secondary addresses are considered and a cache entry is made for each one. There are two sorts of secondary addresses: Those that undergo address reversal and those that do not. Both must be considered in order to properly seed the address reversal cache because of the need to check for message capture requests in all cases.

Secondary addresses that undergo reversal are normally stored in the mailAlternateAddress attribute. Another attribute can be specified by setting the LDAP_ALIAS_ADDRESSES MTA option. Secondary addresses that do not undergo reversal are normally stored in the mailEquivalentAddress attribute. Another attribute can be specified with the LDAP_EQUIVALENCE_ADDRESSES MTA option.

Mail Host and Routing Address

It is now time to consider the mailhost and mailRoutingAddress attributes. The actual attributes considered can be overridden with the LDAP_MAILHOST and LDAP_ROUTING_ADDRESS MTA options, respectively. These attributes work together to determine whether or not the address should be acted on at this time or forwarded to another system.

The first step is to decide whether or not mailhost is meaningful for this entry. A preliminary check of the delivery options active for the entry is done to see whether or not the entry is mailhost-specific. If it is not, mailhost checking is omitted. See the description of "Delivery Options Processing", and the # flag in particular, to understand how this check is done.

In the case of a user entry, the mailhost attribute must identify the local system in order to be acted on. The mailhost attribute is compared with the value of the local.hostname configutil parameter and against the list of values specified by the local.imta.hostnamealiases configutil parameter. The mailhost attribute is deemed to identify the local host if any of these match.

A successful match means that the alias can be acted on locally and alias processing continues. An unsuccessful match means that the message needs to be forward to the mailhost to be acted on. A new address of the form

The handling of a missing mailhost attribute is different depending on whether the entry is a user or a group. In the case of a user, a mailhost is essential, so if no mailhost attribute is present a new address of the form

is constructed using the smart host for the domain determined by the LDAP_DOMAIN_ATTR_SMARTHOST MTA option. An error is reported if no smart host exists for the domain.

Groups, on the other hand, do not require a mailhost, so a missing mailhost is interpreted as meaning that the group can be expanded anywhere. So alias processing continues.

The mailRoutingAddress attribute adds one final wrinkle. If it is present, alias processing terminates with the mailRoutingAddress as the result. However, if a mailhost is present, it is added to the mailRoutingAddress as a source route.

Miscellaneous Attribute Support

Next the mailMsgMaxBlocks attribute is considered. First it is minimized with the domain block limit returned from the LDAP_DOMAIN_ATTR_BLOCKLIMIT MTA option. If the size of the current message is known to exceed the limit, alias processing terminates with a size-exceeded error. If the size is not known or does not exceed the limit, the limit is nevertheless stored and will be rechecked when the message itself is checked later. The use of mailMsgMaxBlocks can be overridden with the LDAP_BLOCKLIMIT MTA option.

Next a number of attributes are accessed and saved. Eventually these will be written into the queue file entry for use by the ims_master channel program, which will then use them to update the store's user information cache. If the attributes are not found for individual users, domain-level attributes can be used to set defaults.

This step is skipped if the LDAP entry is for a group rather than a user or if the LDAP entry came from the alias cache and not from the LDAP directory. The logic behind the latter criteria is that frequent updates of this information are unnecessary and using the alias cache offers a reasonable criteria for when updates should be done. The names of the attributes retrieved are set by various MTA options.

Table 7-3 shows the MTA options which set the retrieved disk quota and message quota attributes.

Next a number of attributes are stored for possible use in conjunction with metacharacter substitutions later.

Spare slots for additional attributes are included so that you can use them to build customized address expansion facilities.

Table 7-3 MTA Options Which Set the Retrieved Disk Quota and Message Quota Attributes
MTA option	Attribute
LDAP_DISK_QUOTA	mailQuota
LDAP_DOMAIN_ATTR_DISK_QUOTA	mailQuota
LDAP_DOMAIN_ATTR_MESSAGE_QUOTA	mailMsgQuota
LDAP_MESSAGE_QUOTA	mailMsgQuota

Table 7-4 MTA Options, Default Attributes, and Metacharacters
MTA Option	Default Attribute	Metacharacters
LDAP_PROGRAM_INFO	mailProgramDeliveryInfo	$P
LDAP_DELIVERY_FILE	mailDeliveryFileURL	$F
LDAP_SPARE_1	no default	$1E $1G $E
LDAP_SPARE_2	no default	$2E $2G $G
LDAP_SPARE_3	no default	$3E $3G
LDAP_SPARE_4	no default	$4E $4G
LDAP_SPARE_5	no default	$5E $5G

Next any values associated with the mailConversionTag attribute are added to the current set of conversion tags. The name of this attribute can be changed with the LDAP_CONVERSION_TAG MTA option. If any values were associated with the domain's mailDomainConversionTag attribute, they are attached as well.

Delivery Options Processing

Next the mailDeliveryOption attribute is checked. The name of this attribute can be changed with the LDAP_DELIVERY_OPTION MTA option. This is a multi valued option and its values determine the addresses produced by the alias translation process. Additionally, the permissible values are different for users and groups. Common permissible values are program, forward, and hold.” User-only values are mailbox, native, unix, and autoreply. The group-only values are members, members_offline, and file.

The conversion of the mailDeliveryOption attribute into appropriate addresses is controlled by the DELIVERY_OPTIONS MTA option. This option not only specifies what addresses are produced by each permissible mailDeliveryOption value, but also what the permissible mailDeliveryOption values are and whether or not each one is applicable to users, groups, or both.

The value of this option consists of a comma-separated list of deliveryoption=template pairs, each pair with one or more optional single character prefixes.

DELIVERY_OPTIONS=*mailbox=$M%$\\$2I$_+$2S@ims-ms-daemon,
     &members=*,
     *native=$M@native-daemon,
     /hold=@hold-daemon:$A,
     *unix=$M@native-daemon,
     &file=+$F@native-daemon,
     &@members_offline=*,
     program=$M%$P@pipe-daemon,
     #forward=**,
     *^!autoreply=$M+$D@bitbucket

Each delivery option corresponds to a possible mailDeliveryOption attribute value and the corresponding template specifies the resulting address using the same metacharacter substitution scheme used by URL processing.

Table 7-5 shows the single character prefixes available for the DELIVERY_OPTIONS options.

Table 7-5 Single-Character Prefixes for options in the DELIVERY_OPTIONS MTA option.
Character Prefix	Description
*	Delivery option applies to users.
&	Delivery option applies to groups.
$	Sets a flag saying expansion of this user or group is to be deferred.
^	Sets a flag saying that the vacation start and end times should be checked to see if this delivery option really is in effect.
#	Sets a flag saying expansion of this delivery option does not need to take place on the entry’s designated mailhost.
/	Sets a flag that causes all addresses produced by this delivery option to be held. Message files containing these recipient addresses will have a .HELD extension.
!	Sets a flag that says that autoreply operations should be handled internally by the MTA. It only makes sense to use this prefix on an autoreply delivery option. The option’s value should direct the message to the bitbucket channel.

If neither * nor & are present, the delivery option is taken to apply to both users and groups.

Additional Metacharacters for Use in Delivery Options

Several additional metacharacters have been added to support this new use of the MTA's URL template facility. These include:

Table 7-6 shows additional metacharacters and their descriptions for use in delivery options.

Table 7-6 Additional Metacharacters for Use in Delivery Options
Metacharacter	Description
$\\	Force subsequent text to lower case.
$^	Force subsequent text to upper case.
$_	Perform no case conversion on subsequent text.
$nA	Insert the nth character of the address. The first character is character 0. The entire address is substituted if n is omitted. This is intended to be used to construct autoreply directory paths.
$D	Insert the domain part of the address.
$nE	Insert the value of the nth spare attribute. If n is omitted the first attribute is used.
$F	Insert the name of the delivery file (mailDeliveryFileURL attribute).
$nG	Insert the value of the nth spare attribute. If n is omitted the second attribute is used.
$nH	Insert the nth component of the domain from the original address counting from 0. The default is 0 if n is omitted.
$nI	Insert hosted domain associated with alias. This metacharacter accepts an integer parameter n whose semantics are described in Table 7-7.
$nJ	Insert the nth part of the host domain counting from 0. The default for n is 0.
$K	Insert an LDAP filter that matches the object classes for a user or group. See the description of the LDAP_UG_FILTER output-only MTA option.
$L	Insert the local part of the address.
$M	Insert the UID associated with the current alias.
$P	Insert the program name (mailProgramDeliveryInfo attribute).
$nS	Insert subaddress associated with the current address. This metacharacter accepts an integer parameter n whose semantics are described in Table 7-7.
$nU	Insert the nth character of the dequoted form of the mailbox part of the current address. The first character is character 0. The entire dequoted mailbox is substituted if n is omitted.
$nX	Insert the nth component of the mailhost. The entire mailhost is inserted if n is omitted.

Table 7-7 shows how the integer parameter modifies the behavior of the $nI and $nS metacharacters.

Table 7-7 Integers Controlling Behavior Modification of the $nI and $nS Metacharacters
Integer	Description of Behavior
0	Fail if no value is available (default).
1	Insert value if one is available. Insert nothing if not.
2	Insert value if one is available. Insert nothing and delete preceding character is one is not (this particular behavior is needed by the ims-ms channel).
3	Insert value if one is available. Insert nothing and ignore following character if one is not.

In addition to the metacharacters, Table 7-8 shows two special template strings.

Table 7-8 Special Template Strings
Special Template String	Description
*	Perform group expansion. This value is not valid for user entries.
**	Expand the attribute named by the LDAP_FORWARDING_ADDRESS MTA option. This defaults to mailForwardingAddress.

With group expansion, for example, if a user's mailDeliveryOption value is set to mailbox, we form a new address consisting of the stripped UID, a percent sign followed by the hosted domain if one is applicable, a plus sign followed by the subaddress if one was specified, and finally @ims-ms-daemon.

Delivery Option Defaults

If the list of active delivery options is empty at this point, the first option on the list (usually mailbox) is activated for users and the second option on the list (usually members) is activated for groups.

Start and End Date Checks

Start and end dates are checked after the delivery option list has been read. There are two attributes whose names are controlled by the LDAP_START_DATE (default vacationStartDate) and LDAP_END_DATE (default vacationEndDate) MTA options, respectively. If one or more of the active delivery options specified the ^ prefix character, the values of these options are checked against the current date. If the current date is outside the range specified by these options, the delivery options with the ^ prefix are removed from the active set.

Optin, Presence, and Autosecretary Attributes

The LDAP_OPTIN MTA option can be used to specify an LDAP attribute containing a list of spam filter opt-in values. If the option is specified and the attribute is present, it is appended to the current spam filter opt-in list. Any values set by the domain level attribute set by the LDAP_DOMAIN_ATTR_OPTIN MTA option will also be appended to the list.

The LDAP_PRESENCE MTA option can be used to specify a URL that can be resolved to return presence information about the user. If the option is specified and the attribute is present, its value is saved for possible use in conjunction with Sieve presence tests. The domain level attribute set by the LDAP_DOMAIN_ATTR_PRESENCE MTA option is used as source for this URL if no value exists for the user entry.

The LDAP_AUTOSECRETARY MTA option can be used to specify a URL that controls the location where autosecretary information is stored. If the option is specified and the attribute is present, its value is saved for possible use with the Messaging Server autosecretary facility. The domain level attribute set by the LDAP_DOMAIN_ATTR_AUTOSECRETARY MTA option is used as source for this URL if no value exists for the user entry.

Sieve Filter Handling

Next the mailSieveRuleSource attribute is checked for a Sieve filter that applies to this entry. If this attribute exists, it is parsed and stored at this point. The two possible forms for the value of this attribute are a single value that contains a complete Sieve script or multiple values where each value contains a piece of a Sieve script. The latter form is produced by the web filter construction interface. Special code is used to order the values and glue them together properly.

The use of the mailSieveRuleSource attribute specifically can be overridden by using the LDAP_FILTER MTA option.

Deferred Processing Control

Next the mailDeferProcessing attribute is checked. This attribute can be changed by using the LDAP_REPROCESS MTA option. Processing continues normally if this attribute exists and is set to no. But if this attribute is set to yes and the current source channel is not the reprocess channel, expansion of this entry is aborted and the original user@domain address is simply queued to the reprocess channel. If this attribute does not exist, the setting of the deferred processing character prefix associated with delivery options processing is checked. (See the section Delivery Options Processing for examples.) Processing is deferred if it is set. If it is not set, the default for users is no. The default for groups is controlled by the MTA option DEFER_GROUP_PROCESSING, which defaults to 1 (yes). Alias processing concludes at this point for user entries.

Group Expansion Attributes

A number of additional attributes are associated with group expansion and must be dealt with at this point. The names of these attributes are all configurable via various MTA options.

Table 7-9 lists the default attribute names, the MTA option to set the attribute name, and the way the attribute is processed by the MTA. The ordering of the elements in the table shows the order in which the various group attributes are processed. This ordering is essential for correct operation.

Table 7-9 Group Expansion Attributes
Default Attribute	MTA option to Set Attribute Name	How the Attribute is Processed
mgrpMsgRejectAction	LDAP_REJECT_ACTION	Single valued attribute that controls what happens if any of the subsequent access checks fail. Only one value is defined: TOMODERATOR, which if set instructs the MTA to redirect any access failures to the moderator specified by the mgrpModerator attribute. The default (and any other value of this attribute) causes an error to be reported and the message rejected.
mailRejectText	LDAP_REJECT_TEXT	The first line of text stored in the first value of this attribute is saved. This text will be returned if any of the following authentication attributes cause the message to be rejected. This means the text can appear in SMTP responses so value has to be limited to US-ASCII to comply with current messaging standards.
mgrpBroadcasterPolicy	LDAP_AUTH_POLICY	Specifies level of authentication needed to access list. Possible values are SMTP_AUTH_REQUIRED or AUTH_REQ, both of which mean that the SMTP AUTH command must be used to identify the sender in order to post to the list, and PASSWORD_REQUIRED, PASSWD_REQUIRED, or PASSWD_REQ, all of which mean the password to the list specified by the mgrpAuthPassword attribute must appear in an Approved: header field in the message, and NO_REQUIREMENTS, which means no special requirements apply. If SMTP AUTH is called for it also implies that any subsequent authorization checks will be done against the email address provided by the SASL layer rather than the MAIL FROM address.
mgrpAllowedDomain	LDAP_AUTH_DOMAIN	Domains allowed to submit messages to this list. Can be multi valued.
mgrpDisallowedDomain	LDAP_CANT_DOMAIN	Domains not allowed to submit messages to this list. Can be multi valued.
mgrpAllowedBroadcaster	LDAP_AUTH_URL	URL identifying mail addresses allowed to send mail to this group. Can be multi valued. Each URL is expanded into a list of addresses and each address is checked against the current envelope from address. A match means the message is allowed.
mgrpDisallowedBroadcaster	LDAP_CANT_URL	URL identifying mail addresses not allowed to send mail to this group. Can be multi valued. Each URL is expanded into a list of addresses and each address is checked against the current envelope from address. A match means the message is not allowed.
mgrpMsgMaxSize	LDAP_ATTR_MAXIMUM_MESSAGE_SIZE	Maximum message size in bytes that can be sent to the group. This attribute is obsolete but still supported for backwards compatibility; the new mailMsgMaxBlocks attribute should be used instead.
mgrpAuthPassword	LDAP_AUTH_PASSWORD	Specifies a password needed to post to the list. The presence of this attribute forces a reprocessing pass. As the message is enqueued to the reprocessing channel, the password is taken from the header and placed in the envelope. Then while reprocessing, the password is taken from the envelope and checked against this attribute. Additionally, only passwords that actually are used are removed from the header field.
mgrpModerator	LDAP_MODERATOR_URL	The list of URLs given by this attribute to be expanded into a series of addresses. The interpretation of this address list depends on the setting of the LDAP_REJECT_ACTION MTA option. If LDAP_REJECT_ACTION is set to TOMODERATOR, this attribute specifies the moderator address(es) the message is to be sent to should any of the access checks fail. If LDAP_REJECT_ACTION is missing or has any other value, the address list is compared with the envelope from address. Processing continues if there is a match. If there is no match, the message is again sent to all of the addresses specified by this attribute. Expansion of this attribute is implemented by making the value of this attribute the list of URLs for the group. Any list of RFC822 addresses or DNs associated with the group is cleared, and the delivery options for the group are set to members. Finally, subsequent group attributes listed in this table are ignored.
mgrpDeliverTo	LDAP_GROUP_URL1	List of URLs which, when expanded, provides a list of mailing list member addresses.
memberURL	LDAP_GROUP_URL2	Another list of URLs which, when expanded, provides another list of mailing list member addresses.
uniqueMember	LDAP_GROUP_DN	List of DNs of group members. DNs may specify an entire subtree. Unique member DNs are expanded by embedding them in an LDAP URL. The exact URL to use is specified by the GROUP_DN_TEMPLATE MTA option. The default value for this option is: ldap:///$A?mail?sub?(mail=*) $A specified the point where the uniqueMember DN is inserted.
mgrpRFC822MailMember	LDAP_GROUP_RFC822	Mail addresses of members of this list.
rfc822MailMember	LDAP_GROUP_RFC822	rfc822MailMember is supported for backwards compatibility. Either rfc822MailMember or mgrpRFC822MailMember, but not both, can be used in any given group.
mgrpErrorsTo	LDAP_ERRORS_TO	Sets the envelope originator (MAIL FROM) address to whatever the attribute specifies.
mgrpAddHeader	LDAP_ADD_HEADER	Turns the headers specified in the attribute into header trimming ADD options.
mgrpRemoveHeader	LDAP_REMOVE_HEADER	Turns the headers specified into header trimming MAXLINES=-1 options.
mgrpMsgPrefixText	LDAP_PREFIX_TEXT	Adds the specified text to the beginning of the message text, if any.
mgrpMsgSuffixText	LDAP_SUFFIX_TEXT	Adds the specified text to the ending of the message text, if any.

One final attribute is checked in the special case of group expansion as part of an SMTP EXPN command: mgmanMemberVisibility or expandable. The LDAP_EXPANDABLE MTA option can be used to select different attributes to check. Possible values are: anyone, which means that anyone can expand the group, all or true, which mean that the user has to successfully authenticate with SASL before expansion will be allowed, and none, which means that expansion is not allowed. Unrecognized values are interpreted as none. If the attribute is not present, the EXPANDABLE_DEFAULT MTA option controls whether the expansion is allowed.

Alias entries are cached in a fashion similar to domain entries. The MTA options controlling the alias cache are ALIAS_ENTRY_CACHE_SIZE (default 1000 entries) and ALIAS_ENTRY_CACHE_TIMEOUT (default 600 seconds). The entire LDAP return value for a given alias is retained in the cache.

Negative caching of alias entries is controlled by the ALIAS_ENTRY_CACHE_NEGATIVE MTA option. A nonzero value enables caching of alias match failures. A zero value disables it. Negative caching of alias entries is disabled by default. The theory is that repeated specification of an invalid address is unlikely to occur very often in practice. In addition, negative caching may interfere with timely recognition of new users added to the directory. However, sites should consider re enabling negative caching of aliases in situations where vanity domains are heavily used. The search performed by the URL specified in ALIAS_URL0 is less likely to be successful.

Address Reversal

Address reversal with direct LDAP starts with a USE_REVERSE_DATABASE value of 4, which disables the use of any reverse database. It then builds on the routing facilities previously discussed. In particular, in the previous version, it began with a reverse URL specification of the form:

The $V metacharacter has already been described in the context of alias URLs However, the $Q metacharacter, while quite similar in function to the $R metacharacter used in alias URLs, is specifically intended for use in address reversal. Unlike $R, it produces a filter that searches attributes containing addresses that are candidates for address reversal. The list of attributes to search comes from the MTA option LDAP_MAIL_REVERSES. If this option is not set, the local.imta.schematag configutil parameter is examined, and depending on its value, an appropriate set of default attributes is chosen.

Table 7-10 shows the local.imta.schematag values and the default attributes chosen.

Table 7-10 local.imta.schematag Values and Attributes
Schema Tag Value	Attributes
sims40	mail,rfc822mailalias
nms41	mail,mailAlternateAddress
ims50	mail,mailAlternateAddress

The use of $Q is no longer appropriate, however. In order for message capture and other facilities to work correctly, address reversal has been enhanced to pay attention to the attribute that matched in addition to the fact that a match occurred. This means that $R should be used to specify the filter instead of $Q. Additionally, the $N metacharacter has been added, which returns a list of the attributes of interest to address reversal. The resulting option value is:

As always, local.imta.schematag can be a comma-separated list. If more than one schema is supported, the combined list of attributes, with duplicates eliminated, is used.

For example, suppose that the domain siroe.com appears in the domain tree and the MTA looks the address:

The filter that results from the expansion of $R and an ims50 schematag will be something like:

(|(|(mail=u@siroe.com)(mail=u@host1.siroe.com))
(|(mailAlternateAddress=u@siroe.com)
(mailAlternateAddress=u@host1.siroe.com)))

Note that the reverse URL explicitly specifies the attribute containing the canonicalized address. Normally this will be the mail attribute.

After the URL is constructed an LDAP search is performed. If the search is successful the first attribute value returned replaces the original address. Unsuccessful searches or errors leave the original address unchanged.

Due to the frequency with which address reversal operations are performed, especially given the number of addresses that can appear in a message header, and the expense of the directory queries involved, both negative and positive results need to be cached. This is implemented with an in-memory, open-chained, dynamically-expanded hash table. The maximum size of the cache is set by the REVERSE_ADDRESS_CACHE_SIZE MTA option (default 100000) and the timeout for entries in the cache is set by the REVERSE_ADDRESS_CACHE_TIMEOUT MTA option (default 600 seconds). The cache actually stores addresses themselves, not the LDAP URLs and LDAP results.

Asynchronous LDAP Operations

Asynchronous lookups avoid the need to store an entire large LDAP result in memory, which can cause performance problems in some cases. The MTA provides the ability to perform various types of lookups done by the MTA asynchronously.

Use of asynchronous LDAP lookups is controlled by the MTA option LDAP_USE_ASYNC. This option is a bit-encoded value. Each bit, if set, enables the use of asynchronous LDAP lookups in conjunction with a specific use of LDAP within the MTA.

Table 7-11 shows the bit and value settings for the LDAP_USE_ASYNC MTA option in the option.dat file.

Table 7-11 Settings for the LDAP_USE_ASYNC MTA Option
Bit	Value	Specific Use of LDAP
0	1	LDAP_GROUP_URL1 (mgrpDeliverTo) URLs
1	2	LDAP_GROUP_URL2 (memberURL) URLs
2	4	LDAP_GROUP_DN (UniqueMember) DNs
3	8	auth_list, moderator_list, sasl_auth_list, and sasl_moderator_list nonpositional list parameter URLs
4	16	cant_list, sasl_cant_list nonpositional list parameter URLs
5	32	originator_reply nonpositional list parameter URLs
6	64	deferred_list, direct_list, hold_list, nohold_list nonpositional list parameter URLs
7	128	username_auth_list, username_moderator_list, username_cant_list nonpositional list parameter URLs
8	256	alias file list URLs
9	512	alias database list URLs
10	1024	LDAP_CANT_URL (mgrpDisallowedBroadcaster) outer level URLs
11	2048	LDAP_CANT_URL inner level URLs
12	4096	LDAP_AUTH_URL (mgrpAllowedBroadcaster) outer level URLs
13	8192	LDAP_AUTH_URL inner level URLs
14	16384	LDAP_MODERATOR_URL (mgrpModerator) URLs

The default value of the LDAP_USE_ASYNC MTA option is 0, which means that asynchronous LDAP lookups are disabled by default.

Settings Summary

ALIAS_MAGIC=8764
ALIAS_URL0=ldap:///$V?*?sub?$R
USE_REVERSE_DATABASE=4
USE_DOMAIN_DATABASE=0
REVERSE_URL=ldap:///$V?mail?sub?$Q

If vanity domains are to be supported, the following additional options must be set:

DOMAIN_MATCH_URL=ldap:///$B?msgVanityDomain?sub?(msgVanityDomain=$D)
ALIAS_URL1=ldap:///$B?*?sub?(&(msgVanityDomain=$D)$R)
ALIAS_URL2=ldap:///$1V?*?sub?(mailAlternateAddress=@$D)

Note that the last of these options also handle the case of wild carded local parts in hosted as well as vanity domains. If wild carded local part support is desired but vanity domain support is not, the following option should be used instead:

The filter ssrd:$A clause needs to be removed from the ims-ms channel definition in the MTA configuration file (imta.cnf).