Appendix B
MTA Direct LDAP Operation
Prior to the iPlanet Messaging Server 5.2 release, directory information about users and groups used by the MTA was accessed through a number of files and databases. Data in these files and databases was updated by the dirsync process which monitored changes to the directory and updated the file and database data accordingly. This remains the default behavior in version 5.2, however there is a new option that allows the MTA to interact directly with the directory. This option is called the direct LDAP mode.
When the MTA is configured to operate in direct LDAP mode, the dirsync process and its databases are not used. Instead the MTA makes equivalent LDAP calls, first to determine whether a domain is hosted on the MTA and then to access the desired delivery information. There is almost no net effect on address translation when changing from the dirsync mode of operation to the direct LDAP mode, except that the direct LDAP mode is configurable and the mechanism more transparent. There is, however, a change in the way hosted domains work, and there is also an impact on the way the system performs. See "Implications of Changing to Direct LDAP Mode" for details.
This chapter consists of the following sections:
To Enable Direct LDAP Mode
To enable direct LDAP mode, make the following changes to the standard MTA configuration:
Add the line to the rewrite section in the file .../imta/config/imta.cnf
-
$* $E$F$U%$H$V$H@localhost
-
where localhost is the primary host name of the MTA.
-
For instance if the MTA is called island.siroe.com, you would modify the start of the rules section of .../imta/config/imta.cnf to read something like
-
! Rules to select local users
$* $E$F$U%$H$V$H@island.siroe.com
island.siroe.com $U%$D@island.siroe.com
siroe.com $U%$D@island.siroe.com
Change the definition of the ims-ms channel in .../imta/config/imta.cnf to remove the clause filter ssrd:$A.
-
If your ims-ms channel definition reads
-
! ims-ms
ims-ms defragment subdirs 20 notices 1 7 14 21 28 \
backoff "pt5m" "pt10m" "pt30m" "pt1h" "pt2h" "pt4h" \
maxjobs 1 pool IMS_POOL fileinto $U+$S@$D filter ssrd:$A
ims-ms-daemon
-
it should be changed to read
-
! ims-ms
ims-ms defragment subdirs 20 notices 1 7 14 21 28 \
backoff "pt5m" "pt10m" "pt30m" "pt1h" "pt2h" "pt4h" \
maxjobs 1 pool IMS_POOL fileinto $U+$S@$D
ims-ms-daemon
Add the following lines to the file .../imta/config/option.dat:
-
ALIAS_MAGIC=8764
ALIAS_URL0=ldap:///$V?*?sub?$R
USE_REVERSE_DATABASE=4
REVERSE_URL=ldap:///$V?mail?sub?$Q
USE_DOMAIN_DATABASE=0
-
If vanity domains are to be supported the following additional options must also be set:
-
DOMAIN_MATCH_URL=ldap:///$B?msgVanityDomain?sub? \
(msgVanityDomain=$D)
ALIAS_URL1=ldap:///$B?*?sub?(&(msgVanityDomain=$D)$R)
ALIAS_URL2=ldap:///$1V?*?sub?(mailAlternateAddress=@$D)
Remove the following lines from .../imta/config/job_controller.cnf.
-
[PERIODIC_JOB=dirsync_incr]
command=IMTA_TABLE:../../imsimta dirsync
time=/00:10
!
[PERIODIC_JOB=dirsync_full]
command=IMTA_TABLE:../../imsimta dirsync -F
time=02:00/24:00
!
Remove the following lines from the SEND_ACCESS mapping at the end of the file .../imta/config/mappings
-
*|*|inactive|* $X4.2.1|$NMailbox$ temporarily$ disabled
*|*|deleted|* $X5.1.6|$NRecipient$ no$ longer$ on$ server
Delete, or at least move these MTA databases:
-
.../imta/db/aliasesdb.db
.../imta/db/domaindb.db
.../imta/db/reversedb.db
Compiled the modified MTA configuration. This must happen before it comes into effect.
How Direct LDAP Mode Works
MTA processing of destination email addresses is fundamentally unchanged. Briefly, the process is as follows: First, the MTA uses the rewrite rules to 1) determine if a domain is recognized, 2) rewrite the address as appropriate, and 3) route the message to the appropriate channel. If the message is routed to the l channel, then the address is transformed using the alias lookup process (see "Direct LDAP Alias Resolution"), and the resulting address or addresses are rewritten again using the rewrite rules to route these addresses to the channel associated with the aliases. Typically this is the ims-ms channel, auto_reply or one of the other standard MTA channels.
The direct LDAP mode of operation changes the rewrite rule phase and the alias phase of the address processing. These changes are described in the following subsections.
Resolving Addresses Using the Direct LDAP Rewrite Rule ($V)
The MTA resolves addresses by first checking the domain part of the address (the part to the right of the @) against the rewrite rules. Rewrite rules are found in the first half of the .../imta/config/imta.cnf file. If a match is found, the rules specify to which channel the mail is to be routed. For instance mail is routed to tcp_local for outbound internet traffic, or the local channel, l, for users provisioned in the directory.
When the MTA is configured in dirsync mode, the rule evaluation process uses information from the domain database, which is one of those databases maintained by the dirsync process. When the MTA is configured in direct LDAP mode, a special "try me first" rewrite rule is used. This rule looks like this:
$* $E$F$U%$H$V$H@localhost
The left hand side of this rule, the $* pattern, means try this rule first, and try it on all addresses. The right hand side says:
$E - use on envelope addresses only.
$F - use on forward pointing (To:) addresses only.
$U%$H - "Rewrite" an address to the form user@host. (Actually the rules specify that the unmodified original address be used.)
$V$H - Only match this rule if the host part of the address, the part of the address to the right of the @ sign, matches a domain defined in the directory.
@localhost - route to the l channel.
How the LDAP Domain Lookup Works
The new part of the rewrite rule process is the $V matching parameter. $V is used to determine if an address is local, and if so, to find its location in the directory tree. $V takes a parameter, in this case $H, the host part of the address. The $V tag brings into play a number of LDAP lookups. The process involves looking up the domain part of the address in the DC tree to find the appropriate subtree of the user and group tree. For instance if the address under consideration is
robinson.crusoe@desert.island.siroe.com
the MTA first checks for the domain desert.island.siroe.com, then, if that fails, for island.siroe.com, siroe.com, and com. This LDAP lookup takes place in the DC tree in the directory (a detailed description of the iPlanet Messaging Server namespace and DIT structures is described in the iPlanet Messaging Server Provisioning Guide). This tree is rooted at the location specified by the service.dcroot configutil attribute, default value o=internet. The lookups will be for entries with distinguished names of
dc=desert,dc=island,dc=siroe,dc=com,o=internet
dc=island,dc=siroe,dc=com,o=internet
dc=siroe,dc=com,o=internet
dc=com,o=internet
A domain lookup is only considered a success if the entry found has either an object class of inetDomain and an attribute of inetDomainBaseDn, or an object class of inetDomainAlias and an attribute of aliasedObjectName.
If you want to prevent the checking of upper level domains, in the example this would be island.siroe.com, siroe.com, and com, you can do so by clearing the least significant bit of the option DOMAIN_UPLEVEL. DOMAIN_UPLEVEL is specified in .../imta/config/option.dat. Its default value is 1, so to prevent the up level checks add the line
DOMAIN_UPLEVEL=0
to .../imta/config/option.dat.
There is another new tag, $Z, which has exactly the opposite meaning to $V. $V makes a rule match if the host is found in the directory, $Z makes a rule match if the host is not found in the directory.
Vanity Domain Lookup
If you have any vanity domains (NOT hosted domains) defined for any users, you need to enable the LDAP check for these too. The check for vanity domains is disabled by default. To enable it, add the following line to .../imta/config/option.dat:
DOMAIN_MATCH_URL=ldap:///$B?msgVanityDomain?sub? \
(msgVanityDomain=$D)
The check for vanity domains only takes place if the checks for hosted domains fail.
Domain Lookup Cache
Checking the directory for all domains is a potentially expensive operation, as it has to be performed for all domains, including any internet domains to which anyone sends mail. To reduce the cost, the results of the lookups are cached by the MTA. By default the results of up to 100000 lookups (successful or otherwise) are cached for up to 600 seconds. This caching can be controlled setting by the following options in .../imta/config/option.dat
DOMAIN_MATCH_CACHE_SIZE=100000
DOMAIN_MATCH_CACHE_TIMEOUT=600
Managing LDAP Errors During Address Rewrite
There are four possible outcomes to a domain lookup in the directory:
The domain is found and is good
The domain is found and is not good
The domain is not found
The lookup fails (an LDAP error)
The first case presents no problems. The second and third are treated as equivalent, and cause the $V rule to fail. The last case is more difficult. There are two reasonable courses of action the MTA could take in this case:
Reject the address with a 400 Temporary lookup failure SMTP response.
Redirect the mail to the reprocess channel for processing later.
The first action is the obvious and correct action if the mail is coming from some remote MTA. The second action is more appropriate if the mail is coming from a user agent submitting mail. The MTA needs to tell the difference between these two cases and act accordingly. The mechanism to enable this is the MTA option DOMAIN_FAILURE. DOMAIN_FAILURE specifies a string that is used to replace the unused part of a rewrite rule in the case of a domain lookup failure. Thus, if DOMAIN_FAILURE has its default value of
DOMAIN_FAILURE=reprocess-daemon$Mtcp_local$1M$1~-error$4000000?Temporary lookup failure
and the rewrite rule being processed is the standard
$* $E$F$U%$H$V$H@localhost
and the domain lookup caused by the $V$H phrase fails, then processing continues as if the rewrite rule had been
$* $E$F$U%$H$V$H@reprocess-daemon$Mtcp_local$1M$1~-error$4000000?Temporary lookup failure
The processing of this resulting rule is as follows:
$E - use on envelope addresses only.
$F - use on forward pointing (To:) addresses only.
$U%$H - "Rewrite" an address to the form user@host. (Actually the rules specify that the unmodified original address be used.)
$V$H - Only match this rule if the host part of the address—the part of the address to the right of the @ sign—matches a domain defined in the directory. This encounters an LDAP error, thus generating the modified rule.
@reprocess-daemon - route to the reprocess channel.
$Mtcp_local - "fail" if the source channel is not tcp_local. This failure is outcome of processing so far. Processing of the rule continues.
($1M) - "fail" if the channel is not an internal reprocessing channel such as reprocess or conversion.
$~ - stop processing with a successful match if the rule is currently failing.
-error - change the destination channel to the invalid channel reprocess-daemon-error
$4000000?Temporary lookup failure - set the SMTP extended error code to 4.0.0 and the error text to "Temporary lookup failure."
Thus, if the source channel is tcp_local (in all likelihood a connection from some remote MTA) the rewrite rule succeeds, but no channel exists for reprocess-daemon-error, so the address is rejected, and rejected with the 400 error code specified in the rule.
If the source channel is tcp_intranet (probably a user agent), the rule succeeds routing the message to the reprocess channel.
The DOMAIN_FAILURE option and the effective rewrite rule constructed from it uses some new rewrite tags.
$1M is similar to the existing $Mchannel tag in that it causes a rule to fail if the source channel is a reprocessing channel. It is more or less equivalent to $Mreprocess$Mprocess$Mdefragment$conversion.
$~ causes the channel matching checks specified in any $M or $1M (or $M or $1N) tags to be performed, and if the result is a failure, to terminate processing immediately with a success.
$abbbccc?text specifies the error code and error text to be used in the event of a failure. The error code is actually three decimal numbers a, bbb, ccc, and generates an extended SMTP result code of a.bbb.ccc.
Direct LDAP Alias Resolution
The object of alias resolution is to take a message's incoming address (alias) and generate an email address used to deliver the message to a channel. This address is called the delivery address and typically has the form uid@channel_name.
Rewrite rules look only at the part of the address to the right of the @ sign. Alias resolution, however, potentially looks at the whole of the address. The mechanisms used for address resolution is controlled by the option ALIAS_MAGIC in .../imta/config/option.dat. The default behavior is to check for a match in the aliases file and then in the aliases database, which is maintained by the dirsync process. (See "Aliases".)
To enable direct LDAP operation, add the following line to .../imta/config/option.dat.
ALIAS_MAGIC=8764
This causes alias resolution to be attempted using the aliases file (typically only used for the site postmaster), then through the LDAP directory. LDAP alias resolution goes through a number of steps before generating a delivery channel. These are as follows:
Find the address's user/group entry in the LDAP directory.
Determine the entry type (user or group).
Extract the entry status (example, active, inactive, deleted, hold).
Extract the uid attribute.
Find the location of the user.
Verify that the size of the message does not exceed specified limits.
Generate delivery address based on the mailDeliveryOption attribute (example, mailbox, autoreply, program, and forward).
The remainder of this section describes each of these steps in detail.
Finding the User/Group Entry in the LDAP Directory
The LDAP queries to find the user/group entry of the alias address are defined by the URLs specified by the following options in .../imta/config/option.dat:
ALIAS_URL0
ALIAS_URL1
ALIAS_URL2
Unless you support vanity domains, only ALIAS_URL0 is used. The recommended setting for this option is
ALIAS_URL0=ldap:///$V?*?sub?$R
The processing for the $V tag is similar to the processing for the $V tag described "How the LDAP Domain Lookup Works". If the lookup of the domain part of the address succeeds, the $V in the URL is replaced by the DN pointed to by the inetDomainBaseDn or aliasedObjectName attribute in the entry found. If the lookup fails, the alias expansion fails. (There is a variant of the $V tag available, $1V, that, if the lookup fails, returns the DN of the top of the user and group tree—the value of local.ugldapbasedn.)
$R is replaced by a filter appropriate for the schema in use as defined by the configutil parameter local.imta.schematag. The possible schema values and the attributes to be searched for a matching email address are as follows:
ims50 mail,mailalternateAddress,mailEquivalentAddress
nms41 mail,mailalternateAddress
sims40 mail,rfc822mailalias
local.imta.schematag can specify more than one of these values, comma separated. If more than one schema is specified, the union of the attributes are searched for a match. If your directory schema does not exactly match any of these schemata, you can override the list of attributes to be searched by specifying the configutil parameter local.imta.mailaliases. For example:
local.imta.mailaliases=mail,mailAlternateAddresses,email
would cause a search for a match on the attributes mail, mailAlternateAddresses, and email.
By default, the filter generated by the $R tag only searches for the address given. However you might want implied aliases in higher level domains. Thus, although you have provisioned robinson.crusoe in the desert.island.siroe.com domain, you might want to match for his user name in all domains in the domain tree. Thus, if the domain that matched in the evaluation of the rewrite rules was siroe.com, then the addresses searched for in the directory would be
robinson.crusoe@desert.island.siroe.com
robinson.crusoe@island.siroe.com
robinson.crusoe@siroe.com
To get this behavior, you have to set the next to least significant bit of the option DOMAIN_UPLEVEL, for instance by adding the line to the file .../imta/config/option.dat:
DOMAIN_UPLEVEL=3
Domain Lookup in Nonstandard Directories
If you cannot use the standard iPlanet directory structure with its DC tree separate from the user and group tree, another mechanism is available for finding the base of the tree in which to search for aliases. Instead of using $V in the ALIAS_URL0 as described above, you can invoke a mapping. The syntax for achieving this is to place in the URL instead of $V the following:
$|/mapping-name/mapping-argument|
The | initiates and terminates the callout. The character immediately following the $| is the separator between the mapping name and argument; a character should be chosen that doesn't collide with the expected character values used in either the mapping name or argument. mapping name is the name of the domain lookup mapping table. mapping-argument is the name of a domain, for example, $D becomes the name of the domain.
Domain Lookup for Vanity Domain Aliases
To support vanity domain aliases, the following additional URLs must be defined in .../imta/config/option.dat
ALIAS_URL1=ldap:///$B?*?sub?(&(msgVanityDomain=$D)$R)
ALIAS_URL2=ldap:///$1V?*?sub?(mailAlternateAddress=@$D)
LDAP Failures During Alias Resolution
The result of an alias lookup in the directory can return zero, one, or several results. If more than one entry matches, the lookup is considered a failure, just as if no results had been returned, and the address will be rejected as invalid. If, for any reason, none of the configured directory servers can be reached, or if an LDAP query results in an error, the address will be rejected with a temporary failure indication (4xx error in SMTP). The sending MTA should retry the mail later, by which time the problem with the directory may have been resolved.
Determining Entry Type
Once an entry has been found in the directory, it can be processed and mail can be delivered to the appropriate channel. The first step in processing the entry is to determine whether it represents a user, group, or something unrecognizable. If we discover that the entry is a user or a group, processing continues as appropriate. If the entry is neither a user nor a group, then the entry, and therefore the address being processed, is quietly ignored.
Entry type is determined by looking at the object classes to which the entry belongs. The required object classes for users and groups is implied by the schema in use for the directory, as defined by the local.imta.schematag setting. The object classes that have to be present to define an entry as a user or group for the different schemata are:
ims50: users: inetLocalMailRecipient + inetmailuser
groups: inetLocalMailRecipient + inetmailgroup
nms41: users: mailRecipient + nsMessagingServerUser
groups: mailGroup
sims40: users: inetMailRouting + inetmailuser
groups: netMailRouting + inetmailgroup
If your directory schema does not exactly match any of these schemata, you can define you own determinant for discriminating between user and group directory entries. The MTA options LDAP_USER_OBJECT_CLASSES and LDAP_GROUP_OBJECT_CLASSES can be used to specify the object classes that must be present for an entry to be classified as a user or group respectively. For instance, adding the following lines to .../imta/config/option.dat
LDAP_USER_OBJECT_CLASSES=inetLocalMailRecipient+inetmailUser,mailRecipient+nsMessagingServerUser
LDAP_GROUP_OBJECT_CLASSES=inetLocalMailRecipient+inetmailgroup,mailGroup
is equivalent to setting local.imta.schematag=ims50,nms41, in that an entry will be determined to be a user if it has the object classes inetLocalMailRecipient and inetmailUser, or the object classes mailRecipient and nsMessagingServerUser.
Extracting Attributes Used to Build the Delivery Address
Once the address entry type is determined, the MTA needs to extract a set of attributes from the domain and user or group entries to build the delivery address and deliver the message. Some or all of the following attributes (see Table B-1, Table B-2, and Table B-3) from the domain and user or group entries are extracted. The following tables give the required default attribute names used, and the MTA option that can be used to chose different attribute names. Typically these options will not be set as the default values correspond to the standard schema. However, should your directory use a different attribute name for one or more of these attributes, they can be changed by setting the appropriate option in .../imta/config/option.dat
Table B-1 Default Domain Attributes and Override Options
|
LDAP attribute name
|
MTA override option
|
|
domainUidSeparator
|
LDAP_DOMAIN_ATTR_UID_SEPARATOR
|
|
mailDomainCatchallAddress
|
LDAP_DOMAIN_ATTR_CATCHALL_ADDRESS
|
|
mailDomainConversionTag
|
LDAP_DOMAIN_ATTR_CONVERSION_TAG
|
|
mailDomainMsgMaxBlocks
|
LDAP_DOMAIN_ATTR_BLOCKLIMIT
|
|
mailDomainReportAddress
|
LDAP_DOMAIN_ATTR_REPORT_ADDRESS
|
|
mailDomainSieveRuleSource
|
LDAP_DOMAIN_ATTR_FILTER
|
|
mailDomainStatus
|
LDAP_DOMAIN_ATTR_STATUS
|
|
mailRoutingHosts
|
LDAP_DOMAIN_ATTR_ROUTING_HOSTS
|
|
mailRoutingSmarthost
|
LDAP_DOMAIN_ATTR_SMARTHOST
|
Table B-2 Default User Attributes and Override Options
|
LDAP attribute name
|
MTA override option
|
|
mailConversionTag
|
LDAP_CONVERSION_TAG
|
|
mailDeliveryFileURL
|
LDAP_PROGRAM_INFO
|
|
mailDeliveryOption
|
LDAP_DELIVERY_OPTION
|
|
mailhost
|
LDAP_MAILHOST
|
|
mailMsgMaxBlocks
|
LDAP_BLOCKLIMIT
|
|
mailMsgQuota
|
LDAP_MESSAGE_QUOTA
|
|
mailProgramDeliveryInfo
|
LDAP_PROGRAM_INFO
|
|
mailQuota
|
LDAP_DISK_QUOTA
|
|
mailRoutingAddress
|
LDAP_ROUTING_ADDRESS
|
|
mailSieveRuleSource
|
LDAP_FILTER
|
|
UID
|
LDAP_UID
|
|
|
LDAP_SPARE_1*
|
|
|
LDAP_SPARE_2*
|
|
* The two spare LDAP options may be important as they can be used to substitute into delivery options patterns. This is described later in Processing Delivery Options.
|
Table B-3 Default Group Attributes and Override Options
|
LDAP attribute name
|
MTA override option
|
|
mailRejectText
|
LDAP_REJECT_TEXT
|
|
memberURL
|
LDAP_GROUP_URL2
|
|
mgrpAddHeader
|
LDAP_ADD_HEADER
|
|
mgrpAllowedBroadcaster
|
LDAP_AUTH_URL
|
|
mgrpAllowedDomain
|
LDAP_AUTH_DOMAIN
|
|
mgrpAuthPassword
|
LDAP_AUTH_PASSWORD
|
|
mgrpBroadcasterPolicy
|
LDAP_AUTH_POLICY
|
|
mgrpDeliverTo
|
LDAP_GROUP_URL1
|
|
mgrpDisallowBroadcaster
|
LDAP_CANT_URL
|
|
mgrpDisallowDomain
|
LDAP_CANT_DOMAIN
|
|
mgrpErrorsTo
|
LDAP_ERRORS_TO
|
|
mgrpMsgMaxSize
|
LDAP_ATTR_MAXIMUM_MESSAGE_SIZE
|
|
mgrpMsgPrefixText
|
LDAP_PREFIX_TEXT
|
|
msgpMsgSuffixText
|
LDAP_SUFFIX_TEXT
|
|
mgrpModerator
|
LDAP_MODERATOR_URL
|
|
mgrpRemoveHeader
|
LDAP_REMOVE_HEADER
|
|
mgrpRFC822MailMember*
|
LDAP_GROUP_RFC822
|
|
rfc822MailMember*
|
LDAP_GROUP_RFC822
|
|
uniqueMember
|
LDAP_GROUP_DN
|
|
* Note that by default either mgrpRFC822MailMember and rfc822MailMember can be used, but not both.
|
Extracting the User/Group Status
One of the key attributes to control the generated delivery address is the user/group and domain status. If the status of the domain, as defined by mailDomainStatus, is inactive or deleted, then this used as the status of the user, and the user's status is not checked. If the status of the domain is active, then the status of the user or group entry is used. The attributes used to define the status of the entry depends upon the schema used. This is shown below:
ims50: users: inetuserstatus or mailuserstatus
groups: inetmailgroupstatus
nms41: NO STATUS ATTRIBUTES
sims40: users: inetsubsscriberstatus
groups: inetmailgroupstatus
If necessary, the attribute names used to determine the status of users and groups can be overridden. The option LDAP_USER_STATUS can be used to specify the attribute to be used for user status, and the LDAP_GROUP_STATUS option can be used to specify the attribute to be used for group status. Once the user or group's status has been determined, it will be one of active, inactive, deleted, or hold.
active - If the user or group's status is found to be active, processing continues as described in "The Location of the User".
inactive - If the user or group's status is found to be inactive, the address is immediately rejected with a temporary error status (4xx SMTP error code).
deleted - If the user or group's status is found to be deleted, the address is immediately rejected with a permanent error status (5xx SMTP error code).
hold - If the user or group's status is found to be hold, an alias is generated so as to cause the address to be rewritten to the hold channel. The alias generated is controlled by the pattern specified by the HOLD_TEMPLATE MTA option. The default value for the template is
$M?$2I@hold-daemon
The meaning of the tags in the pattern are described in "Generating Delivery Addresses Using DELIVERY_OPTIONS". If the address given was
robinson.crusoe@desert.island.siroe.com
and the matching entry gave a UID of rcrusoe in a hosted domain of island.siroe.com, the alias generated would be
rcusoe?island.siroe.com@hold-daemon
This address then matches the rewrite rule in .../imta/config/imta.cnf
hold-daemon $U%$H@hold-daemon
which matches but does not modify the address so that mail is delivered to the hold channel.
Extracting the UID
All valid user entries in the directory must have a uid attribute, and group entries may have a uid. The uid is used to generate the delivery address. If a user entry does not have a uid attribute, the entry is ignored. If a user entry has multiple uid attributes, only the first is used.
Sometimes, the uid attribute in the directory may contain more information than is needed. For example, entries in a hosted domain may have the form: real uid, a separator character (defined by the domainUidSeparator attribute), and then a domain (example: uid=walter@siroe.com). If the separator character is present in the uid, the uid used for the construction of aliases is only that part before the separator character.
If it is necessary to use an attribute other than uid as the uid for the delivery address, then the LDAP_UID option can be used to specify that other attribute name.
The Location of the User
Once a user or group has been identified as an active user, the MTA must check that the user is local to this MTA. To be deemed local, an entry must have a mailhost attribute that matches either the local.hostname configutil attribute, or one of the names specified by the local.imta.hostnamealiases configutil attributes. If the user is local, the MTA goes to the next step—making sure the message does not exceed the size limit.
If the mailhost cannot be matched against any of the names for this MTA, a new address of the form
@mailhost:user@domain
is generated. This is a source routed RFC822 address, and will be processed through the rewrite rules. For a source routed address, the rewrite rules look at the source route address rather the domain part.
If a user entry has no mailhost attribute, then the generated address will use the mailRoutingSmarthost associated with the domain:
@smarthost:user@domain
If a user entry has no mailhost attribute and the domain has no mailRoutingSmartHost, the address is discarded and a 5xx error reported.
If a group entry has no mailhost attribute, the group will be processed locally. This apparent inconsistency is important because sometimes it makes sense for groups to be expanded on any inbound relay MTA, rather than on a particular server.
Extracting the Size Limit
There is one final check that the MTA has to perform before the delivery addresses are constructed (for users), or the group is expanded. This last check ensures that the mail message does not, by itself, exceed the mailMsgMaxBlocks attribute for the user, or, if that attribute is not set, the mailDomainMsgMaxBlocks attribute for the domain. If the message is too big, the address is rejected with a 5xx size exceeded error.
Generating Delivery Addresses Using DELIVERY_OPTIONS
If the entry found is a user entry, it remains only to generate the delivery addresses for the user that will cause the mail to be routed back through the rewrite rules to the appropriate channels. The delivery address generating process also takes place for groups, but for groups there are some additional concerns that will be addressed in a later section.
Delivery addresses are generated through a set of patterns. The patterns used depend upon the values defined for the attribute mailDeliveryOption. A delivery address is generated for each valid mailDeliveryOption. The patterns are defined by the MTA option DELIVERY_OPTIONS, which can be defined in .../imta/config/option.dat. The default value for DELIVERY_OPTIONS is
DELIVERY_OPTIONS=*mailbox=$M%$2I+$2S@ims-ms-daemon,
&members=*,
*native=$M@native-daemon,
*unix=$M@native-daemon,
&file=+$F@native-daemon,
hold=$M?$2I@hold-daemon,
&$members_offline=*,
program=$M%$P@pipe-daemon,
forward=**,
*autoreply=$M@autoreply-daemon
The value of DELIVERY_OPTIONS is a set of rules separated by commas. The left side of each rule is the name of a delivery method (example, mailbox, unix, forward) the right hand side is the pattern for construction the delivery address. Each rule may be preceded by one or two special flag characters that affect how and when the rule is applicable. The flag characters are
* this rule applies to users only
& this rule applies to groups only
$ this tag causes the message to be enqueued on the reprocess channel so that the expansion can take place offline
Thus, the delivery methods mailbox, native, unix and autoreply can only be used by users. The delivery methods members and members_offline can only be used by groups, and the delivery methods program and forward can be used by both users and groups.
The right hand side consists of simple substitution text and tags that insert the values of various LDAP attributes. See "Substitution Tags (Case-insensitive)".
Generating Delivery Address—Example
Consider, as an example, a message sent to the address
robinson.crusoe+goats@desert.island.siroe.com
and suppose for the sake of the example that his directory entry contains the attributes
UID: rcrusoe@desert.island.siroe.com
mail: robinson.crusoe@desert.island.siroe.com
mailDeliveryOption: mailbox
mailDeliveryOption: native
mailDeliveryOption: program
mailDeliveryOption: forward
mailDeliveryOption: autoreply
mailProgramDeliveryInfo: capriform.msg
mailForwardingAddress: friday@desert.island.siroe.com
mailForwardingAddress: hulahula@londonbank.siroe.com
then the original address will generate six aliases, one for each delivery method, mailbox, native, program and autoreply, and two for delivery method forward.
The pattern for mailbox, $M%$2I+$2S@ims-ms-daemon, is one of the more complicated.
Table B-4 Pattern Expansion for Delivery Option mailbox
|
Pattern piece
|
Action
|
result
|
|
$M
|
generates rcrusoe
|
rcrusoe
|
|
%
|
generates %
|
rcrusoe%
|
|
$2I
|
generates desert.island.siroe.com
|
rcrusoe%desert.island.siroe.com
|
|
+
|
generates +
|
rcrusoe%desert.island.siroe.com+
|
|
$2S
|
generates goats
|
rcrusoe%desert.island.siroe.com+goats
|
|
@ims-ms-daemon
|
generates @ims-ms-daemon
|
rcrusoe%desert.island.siroe.com+goats@ims-ms-daemon
|
This resulting address has a domain part that exactly matches the channel tag for the ims-ms channel, so is routed to that channel without further rewriting.
The pattern for native, $M@native-daemon, is simpler.
Pattern expansion for delivery option native.
Table B-5 Pattern Expansion for Delivery Option native
|
Pattern piece
|
Action
|
Result
|
|
$M
|
generates rcrusoe
|
rcrusoe
|
|
@native-daemon
|
generates @native-daemon
|
rcrusoe@native-daemon
|
The resulting address has a domain part that exactly matches the channel tag for the native channel, so is routed to that channel without further rewriting.
The pattern for autoreply, $M@autoreply-daemon, is very similar.
Table B-6 Pattern Expansion for Delivery Option autoreply
|
Pattern piece
|
Action
|
Result
|
|
$M
|
generates rcrusoe
|
rcrusoe
|
|
@autoreply-daemon
|
generates @autoreply-daemon
|
rcrusoe@autoreply-daemon
|
This resulting address has a domain part that exactly matches the channel tag for the autoreply channel, so is routed to that channel without further rewriting.
The pattern for program, $M%$P@pipe-daemon, is almost the same:
Table B-7 Pattern Expansion for Delivery Option program
|
Pattern piece
|
Action
|
Result
|
|
$M
|
generates rcrusoe
|
rcrusoe
|
|
%
|
generates %
|
rcrusoe%
|
|
$P
|
generates prog
|
rcrusoe%prog
|
|
@pipe-daemon
|
generates @pipe-daemon
|
rcrusoe%prog@pipe-daemon
|
This resulting address has a domain part that exactly matches the channel tag for the pipe channel, so is routed to that channel without further rewriting.
The pattern for forward, **, simply results in the values of the attribute mailForwardingAddress being used. This results in the addresses
friday@desert.island.siroe.com
hulahula@londonbank.siroe.com
being generated. Thus, the message sent to robinson.crusoe generates the following delivery addresses and is delivered to the following channels:
rcrusoe%desert.island.siroe.com+goats@ims-ms-daemon ims-ms
rcrusoe@native-daemon native
rcrusoe@autoreply-daemon autoreply
rcrusoe%prog@pipe-daemon pipe
friday@desert.island.siroe.com
hulahula@londonbank.siroe.com
SIEVE Rules
The final LDAP attribute obtained from the user's entry is mailSieveRuleSource. This contains the SIEVE filter rules for the user. These rules are not applied until the message is on the point of being enqueued to the delivery channel. Although the SIEVE filter is obtained while the MTA is expanding aliases, the SIEVE rules are not used until after the resulting delivery addresses have been expanded and are being sent to the ims-ms, native, autoreply or pipe channels. Note that this is a change in behavior from the non-dirsync mode of operation where only mail delivered to the ims-ms channel is processed through the SIEVE rules.
Processing Group Entries
There are four program delivery options available for groups. These are program, forward, members, and members_offline.
Program and forward are handled exactly as for users.
The pattern for both members and members_offline is *, which invokes the full weight of the group expansion processing that will be described in the next section.
The rule for members_offline is preceded by a $, which means that the group expansion takes place on the reprocess channel. If the enqueuing channel is other than the reprocess channel—and initially the enqueuing channel is almost certainly one of the tcp_ channels—then the processing for the address stops and the original address is accepted and the message enqueued for the reprocess channel. When the reprocess channel runs, the same logic for processing the address is engaged, except that the enqueuing channel is the reprocess channel, and so members_offline, with its $, is processed exactly the same as members.
In principle, processing groups is straight forward: there are a couple of attributes that list the members of the group, either as email addresses or as distinguished names. In either case, those addresses are used as part of the result for the expansion of the group.
In practice, there is a lot more to group processing than that, and there are over a dozen other attributes that can affect the processing of a group entry.
Details on Processing Group Entries
The MTA processes a group entry by considering each of the various group processing options in turn. The order in which the options is processed is important. The group attributes can be divided loosely into three types:
Attributes that provide parameters to the processing such as mailRejectText. These don't affect what can or can't be done, but provide some input to the process.
Attributes that control under what circumstances the mail can be sent to the list. This include such attributes as mgrpAllowDomain, which specifies which domains are allowed to submit messages to the group. These attributes are processed in the order in which they are listed in the table below.
Attributes that give the actual membership of the list.
The tables below list group processing attributes.
Table B-8 Attributes Providing Parameters for Group Processing
|
Attribute
|
Description
|
|
mailRejectText
|
Provides the text to be returned as an SMTP response if any of the authentication mechanisms associated with the group cause the message to be rejected. This attribute ought to be a single valued attribute in US-ASCII only to comply with the protocol rules of SMTP. If the attribute is multivalued, only the first attribute is used. If the value is more than one line, only the first line is used.
|
|
mgrpMsgMaxSize
|
Obsolete attribute. You should use mailmsgMaxBlocks instead as it is checked earlier in the processing of the entry. If the message exceeds this size (specified in bytes), the message will be rejected with a message too large error.
|
|
mgrpAuthPassword
|
Specifies the password for the group and is used if the mgrpBroadcasterPolicy specified requires a password.
|
|
mgrpErrorsTo
|
The envelope originator (MAIL FROM) address is set to the value of this attribute if it is specified. If it is not specified the envelope originator of the message is left unchanged.
|
|
mgrpAddHeader
|
(unsupported as yet)
|
|
mgrpRemoveHeader
|
(unsupported as yet)
|
|
mgrpMsgPrefixText
|
(unsupported as yet)
|
|
mgrpMsgSuffixText
|
(unsupported as yet)
|
Table B-9 Mail Group Access Control Attributes
|
Attribute
|
Description
|
|
mgrpBroadcasterPolicy
|
Specifies what level of authentication is required to send a message to the group. Possible values are:
SMTP_AUTH_REQUIRED or AUTH_REQ either of which mean that the SMTP AUTH command must have been used to identify the sender before they can post to the group.
PASSWORD_REQUIRED, PASSWD_REQUIRED, or PASSWD_REQ, any of which means that the password to the list specified by the mgrpAuthPassword attribute must appear in an Approved: header field in the message.
NO_REQUIREMENTS, which is the same as the attribute not being present and means that there are no special requirements.
|
|
mgrpAllowedDomain
|
Multivalued. Lists the domains from which users can submit messages to the group. If absent, users from any domain can post to the group.
|
|
mgrpDisallowedDomain
|
Multivalued. Lists the domains from which users cannot post to the group.
|
|
mgrpAllowedBroadcaster
|
URLs which, when expanded, generate a list of addresses permitted to send to the list. If an address resulting from the expansion of a URL is a group, the group is expanded to generate a further list, but that is the limit to which the MTA expands the URL. The envelope from address for the message is checked against each address resulting from this expansion, and only if it matches one is the message allowed.
|
|
mgrpDisallowedBroadcaster
|
URLs which, when expanded, generate a list of addresses not permitted to send to the list. If an address resulting from the expansion of a URL is a group, the group is expanded to generate a further list, but that is the limit to which the MTA expands the URL. The envelope from address for the message is checked against each address resulting from this expansion and only if it does not match any is the message allowed.
|
|
mgrpModerator
|
URLs which, when expanded generate a list of addresses permitted to send to the list. If an address resulting from the expansion of a URL is a group, the group is expanded to generate a further list, but that is the limit to which the MTA expands the URL. The envelope from address for the message is checked against each address resulting from this expansion.
If the envelope from address matches one of these addresses, the message is from a moderator and is allowed to post to the group.
If the envelope from address does not match any of these addresses, then the message will be sent to the list of moderator addresses just expanded and will not be delivered to the group in any other way. That is, all the attributes in the group member table below are ignored, and any program or forward delivery options are ignored.
|
Table B-10 Mail Group Expansion Attributes
|
Attribute
|
Description
|
|
mgrpDeliverTo
|
URLs which, when expanded, generate a list of addresses. If an address resulting from the expansion of a URL is a group, the group is expanded to generate a further list, and so on. Duplicate addresses are eliminated, but it is possible to construct groups that refer to each other so that infinite recursion occurs. The MTA resolves this be allowing nested expansion of lists to 10 levels.
|
|
memberURL
|
Another list of URLs which are expanded in the same way as mgrpDeliverTo.
|
|
uniqueMember
|
Distinguished names of group members. Each DN can refer either to a user entry, a group entry, or to a subtree in the directory, in which case all entries in that tree are expanded.
|
|
mgrpRFC822MailMember, rfc822MailMember
|
The values of these attributes are the mail addresses of members of the group. In any given entry only one of these attributes is permitted. rfc822MailMember is supported only to provide backwards compatibility with the Netscape Messaging Server.
|
Alias caching
All this LDAP activity could seriously impact the performance of the MTA. To mitigate this, the MTA process caches the results of the LDAP lookups. This caching is controlled by the following options, shown with their default values:
ALIAS_ENTRY_CACHE_SIZE=1000
ALIAS_ENTRY_CACHE_TIMEOUT=600
ALIAS_ENTRY_CACHE_NAGATIVE=0
This means that the maximum number of cache entries held is 1000, and the maximum length of time an entry is held is ten minutes (600 seconds). The cache entries are larger than the domain cache entries, but if you have sufficient memory on your system, it might well be worth increasing the cache size. ALIAS_ENTRY_CACHE_NEGATIVE controls whether or not alias match failures are cached. By default they are not. Not caching failures speeds up the activation of new users, and it is unlikely that repeated failed attempts are going to be made to deliver the same user at a frequency that will impact the performance of the system.
Reverse Address Translation
Reverse addresses, such as From: headers, generally are normalized as they flow through the MTA. (Normalized means that in the header address, the personal name is moved to the front, and comments get moved to the back. Furthermore, for From: addresses, the address is looked up, and if it is found as a mailalternateaddress, that mail address is used instead.) The general principle applied is that the first mail address listed in a user's directory entry is the address that should be used. The process involves looking up the domain part of the address in the DC tree to find the subtree of the user and group tree to search for the address to look up, and then looking for an entry that contains any email address that matches the one we were given, and returning the first mail address in that entry. This is a very similar process to alias processing.
Direct LDAP address translation relies on two options being set in .../imta/config/option.dat
USE_REVERSE_DATABASE=4
REVERSE_URL=ldap:///$V?mail?sub?$Q
USE_REVERSE_DATABASE=4 tells the MTA not to use the old reverse database, but instead to use the direct LDAP mechanisms. The REVERSE_URL is very similar to the ALIAS_URL0 URL discussed in "Finding the User/Group Entry in the LDAP Directory". The $V tag is expanded in the way described in that section. The $Q tag is similar to the $R tag used in the standard ALIAS_URL0, but produces a filter that searches attributes containing addresses that may match the reverse address the MTA is trying to match. The filter generated by $R depends on the setting of the local.imta.schematag configutil option:
ims50 mail,mailalternateAddress
nms41 mail,mailalternateAddress
sims40 mail,rfc822mailalias
You can override the attributes to be used for the search by specifying them in the MTA option LDAP_MAIL_REVERSES.
The actual search generated is very similar to the search generated by the $R tag used for alias lookup: it searches not only for the address originally given, but also for the address with the domain actually found in the DC tree substituted. This is described in "Finding the User/Group Entry in the LDAP Directory".
If the reverse address lookup fails, the reverse address is left unchanged.
As with the other LDAP lookups, the results of reverse address lookups are cached. The size and time-out for this cache are controlled by the options, shown below with their default values:
REVERSE_ADDRESS_CACHE_SIZE=10000
REVERSE_ADDRESS_CACHE_TIMEOUT=600
Implications of Changing to Direct LDAP Mode
For MTA address translation, there is almost no net effect of changing from the dirsync mode of operation to the direct LDAP mode, except that the process is configurable and the mechanism more transparent. There is, however, a change in the way hosted domains work. In the dirsync mode, all subdomains of hosted domains are implicitly owned, rather like setting DOMAIN_UPLEVEL=3 in the direct LDAP mode. However, only the actual domain configured is owned for the principle domain, rather like setting DOMAIN_UPLEVEL=0 in the direct LDAP mode. This dichotomous mode of operation is not available in the direct LDAP mode: you have to make up your mind which way you want domain ownership to work. This difference probably makes no difference, but be aware.
Clearly there is an impact on the overall way the system performs. The areas affected are:
A changed LDAP load.
Reduced dependency on databases.
Changed overall mail throughput.
Changed LDAP Load
The dirsync process made few but potentially massive queries against the LDAP directory. In the direct LDAP mode, the MTA makes many small queries on the directory. The net effect of this would be a substantial reduction in throughput were it not for the caching that is used. However the load imposed on the directory becomes much more conventional, and therefore the system is more scalable. Whereas with dirsync it was difficult to scale an MTA much above about six million users, in direct LDAP mode it should be possible to provision for tens of millions of users.
Reduced Dependency on Databases
In the dirsync mode, the MTA depended on a number of databases—in particular the alias and domain databases—for its operation. These databases are complicated disk structures and can become corrupt if a system fails suddenly. This has proved to be a problem area for high availability systems. In direct LDAP mode, the MTA has almost no dependency on databases.
Changed Overall Mail Throughput
The increased use of the directory and decreased use of databases has an impact on throughput. Extracting information from the directory and processing it into the form required is more expensive than simply looking up the result in a database. However, if the entry is found in a cache, the overall cost is less than looking it up in a database. This means that if most mail is processed to a few users whose entries have been cached, throughput goes up. If mail is scattered throughout a large user community, throughput will go down.
Performance Tuning Mail Throughput for Direct LDAP Mode
The performance of the system is sensitive to the size of the alias cache, set by ALIAS_ENTRY_CACHE_SIZE. The default value for this, 1000, is probably too small for any significant system: these cache entries can be large, about 2K bytes, and the default value was chosen to avoid overloading a small evaluation system. It is probably reasonable to take this value up to 10,000, or even 50,000 on a large system. For this change to be useful, it is also important to increase the value of MAX_LIFE_CONNS in dispatcher.cnf. MAX_LIFE_CONNS should be at least twice, and probably four times ALIAS_ENTRY_CACHE_SIZE, to take advantage of the cache.The net effect of changing from the dirsync mode of operation to the direct LDAP mode is that the address translation achieved is almost completely unchanged, but is configurable and the mechanism more transparent.