11.6 Template Substitutions and Rewrite Rule Control Sequences

Substitutions are used to rewrite user names or addresses by inserting a character string into the rewritten address, the value of which is determined by the particular substitution sequence used. This section consists of the following subsections:

• 11.6.1 Username and Subaddress Substitution, $U, $0U, $1U

• 11.6.2 Host/Domain and IP Literal Substitutions, $D, $H, $nD, $nH, $L

• 11.6.3 Literal Character Substitutions, $$, $%, $@

• 11.6.4 LDAP Query URL Substitutions, $]...[

• 11.6.5 General Database Substitutions, $(...)

• 11.6.6 Apply Specified Mapping, ${...}

• 11.6.7 Customer-supplied Routine Substitutions, $[...]

• 11.6.8 Single Field Substitutions, $&, $!, $*, $#

• 11.6.9 Unique String Substitutions

• 11.6.10 Source-Channel-Specific Rewrite Rules ($M, $N)

• 11.6.11 Destination-Channel-Specific Rewrite Rules ($C, $Q)

• 11.6.12 Direction-and-Location-Specific Rewrite Rules ($B, $E, $F, $R)

• 11.6.13 Host-Location-Specific Rewrites ($A, $P, $S, $X)

• 11.6.14 Changing the Current Tag Value, $T

• 11.6.15 Controlling Error Messages Associated with Rewriting ($?)

For example, in the following template, the $U is a substitution sequence. It causes the username portion of the address being rewritten to be substituted into the output of the template. Thus, if jdoe@mailhost.siroe.com was being rewritten by this template, the resulting output would be jdoe@siroe.com, the $U substituting in the username portion, jdoe, of the original address:

$U@siroe.com

Control sequences impose additional conditions to the applicability of a given rewrite rule. Not only must the pattern portion of the rewrite rule match the host or domain specification being examined, but other aspects of the address being rewritten must meet conditions set by the control sequence or sequences. For example, the $E control sequence requires that the address being rewritten be an envelope address, while the $F control sequence requires that it be a forward pointing address. The following rewrite rule only applies to (rewrite) envelope To: addresses of the form user@siroe.com:

siroe.com $U@mail.siroe.com$E$F

If a domain or host specification matches the pattern portion of a rewrite rule but doesn’t meet all of the criteria imposed by a control sequences in the rule’s template, then the rewrite rule fails and the rewriter continues to look for other applicable rules.

Table 11–4 summarizes the template substitutions and control sequences.

11.6.1 Username and Subaddress Substitution, $U, $0U, $1U

Any occurrences of $U in the template are replaced with the username (RFC 822 “local-part”) from the original address. Note that user names of the form a.“b” will be replaced by “a.b” as RFC2822 deprecates the former syntax from RFC 822 and it is expected that the latter usage will become mandatory in the future.

Any occurrences of $0U in the template are replaced with the username from the original address, minus any subaddress and subaddress indication character (+). Any occurrences of $1U in the template are replaced with the subaddress and subaddress indication character, if any, from the original address. So note that $0U and $1U are complementary pieces of the username, with $0U$1U being equivalent to a simple $U.

11.6.2 Host/Domain and IP Literal Substitutions, $D, $H, $nD, $nH, $L

Any occurrences of $H are replaced with the portion of the host/domain specification that was not matched by the rule. Any occurrences of $D are replaced by the portion of the host/domain specification that was matched by the rewrite rule. The $nH and $nD characters are variants that preserve the normal $H or $D portion from the nth leftmost part starting counting from 0. That is, $nH and $nD omit the leftmost n parts (starting counting from 1) of what would normally be a $H or $D, substitution, respectively. In particular, $0H is equivalent to $H and $0D is equivalent to $D.

For example, assume the address jdoe@host.siroe.com matches the following rewrite rule:

host.siroe.com    $U%$1D@TCP-DAEMON

The resulting address is jdoe@siroe.com with TCP-DAEMON used as the outgoing channel. Here where $D would have substituted in the entire domain that matched, host.siroe.com, the $1D instead substitutes in the portions of the match starting from part 1 (part 1 being siroe), so substitutes in siroe.com.

$L substitutes the portion of a domain literal that was not matched by the rewrite rule.

11.6.3 Literal Character Substitutions, $$, $%, $@

The $, %, and @ characters are normally metacharacters in rewrite rule templates. To perform a literal insertion of such a character, quote it with a dollar character, $. That is, $$ expands to a single dollar sign, $; $% expands to a single percent, % (the percent is not interpreted as a template field separator in this case); and $@ expands to a single at sign, @ (also not interpreted as a field separator).

11.6.4 LDAP Query URL Substitutions, $]...[

A substitution of the form $]ldap-url[ is interpreted as an LDAP query URL and the result of the LDAP query is substituted. Standard LDAP URLs are used with the host and port omitted. The host and port are instead specified in the msg.conf file (local.ldaphost and local.ldapport attributes).

That is, the LDAP URL should be specified as follows where the square bracket characters, [ ], indicate optional portions of the URL:

ldap:///dn[?attributes[?scope?filter]]

The dn is required and is a distinguished name specifying the search base. The optional attributes, scope, and filter portions of the URL further refine what information to return. For a rewrite rule, the desired attributes to specify returning might be a mailRoutingSystem attribute (or some similar attribute). The scope may be any of base (the default), one, or sub. And the desired filter might be to request the return of the object whose mailDomain value matches the domain being rewritten.

If the LDAP directory schema includes attributes mailRoutingSystem and mailDomain, then a possible rewrite rule to determine to which system to route a given sort of address might appear as the following where here the LDAP URL substitution sequence $D is used to substitute in the current domain name into the LDAP query constructed:

.siroe.com \
  $U%$H$D@$]ldap:///o=siroe.com?mailRoutingSystem?sub? \
  (mailDomain=$D)

For ease in reading, the backslash character is used to continue the single logical rewrite rule line onto a second physical line. Table 11–5 lists the LDAP URL Substitution Sequences.

The MTA now caches URL results from lookups done in rewrite rules and mappings. This new URL result cache is controlled by two new MTA options, URL_RESULT_CACHE_SIZE (default 10000 entries) and URL_RESULT_CACHE_TIMEOUT (default 600 seconds).

11.6.5 General Database Substitutions, $(...)

A substitution of the form $(text) is handled specially. The text part is used as a key to access the special general text database. This database consists of the file specified with the IMTA_GENERAL_DATABASE option in the msg-svr-base/config/imta_tailor file, which is usually the file msg-svr-base/db/generaldb.db.

If “text-string” is found in the database, the corresponding template from the database is substituted. If “text-string” does not match an entry in the database, the rewrite process fails; it is as if the rewrite rule never matched in the first place. If the substitution is successful, the template extracted from the database is re-scanned for additional substitutions. However, additional $(text) substitutions from the extracted template are prohibited in order to prevent endless recursive references.

As an example, suppose that the address jdoe@siroe.siroenet matches the following rewrite rule:

.SIROENET $($H)

Then, the text string siroe will be looked up in the general database and the result of the look up, if any, is used for the rewrite rule’s template. Suppose that the result of looking up siroe is $u%eng.siroe.com@siroenet. Then the output of the template will be jdoe@eng.siroe.com (i.e., username = jdoe, host/domain specification = eng.siroe.com), and the routing system will be siroenet.

If a general text database exists it should be world readable to insure that it operates properly. See 10.9.1 MTA Text Databases for more information.

11.6.6 Apply Specified Mapping, ${...}

A substitution of the form.SIROENET $($H) ${mapping,argument} is used to find and apply a mapping from the MTA mapping file. The mapping field specifies the name of the mapping table to use while argument specifies the string to pass to the mapping. The mapping must exist and must set the $Y flag in its output if it is successful; if it doesn’t exist or doesn’t set $Y the rewrite will fail. If successful the result of the mapping is merged into the template at the current location and re-expanded.

This mechanism allows the MTA rewriting process to be extended in various complex ways. For example, the username part of an address can be selectively analyzed and modified, which normally isn’t a feature the MTA rewriting process is capable of.

11.6.7 Customer-supplied Routine Substitutions, $[...]

A substitution of the form $[image,routine,argument] is used to find and call a customer-supplied routine. At run-time on UNIX, the MTA uses dlopen and dlsym to dynamically load and call the specified routine from the shared library image. The routine is then called as a function with the following argument list:

status := routine (argument, arglength, result, reslength)

argument and result are 252 byte long character string buffers. On UNIX, argument and result are passed as a pointer to a character string, (for example, in C, as char*.) arglength and reslength are signed, long integers passed by reference. On input, argument contains the argument string from the rewrite rule template, and arglength the length of that string. On return, the resultant string should be placed in result and its length in reslength. This resultant string will then replace the “$[image,routine,argument]” in the rewrite rule template. The routine should return 0 if the rewrite rule should fail and -1 if the rewrite rule should succeed.

This mechanism allows the rewriting process to be extended in all sorts of complex ways. For example, a call to some type of name service could be performed and the result used to alter the address in some fashion. Directory service lookups for forward pointing addresses (e.g., To: addresses) to the host siroe.com might be performed as follows with the following rewrite rule. The $F, described in 11.6.12 Direction-and-Location-Specific Rewrite Rules ($B, $E, $F, $R) causes this rule to be used only for forward pointing addresses:

siroe.com $F$[LOOKUP_IMAGE,LOOKUP,$U]

A forward pointing address jdoe@siroe.com will, when it matches this rewrite rule, cause LOOKUP_IMAGE (which is a shared library on UNIX) to be loaded into memory, and then cause the routine LOOKUP called with jdoe as the argument parameter. The routine LOOKUP might then return a different address, say, John.Doe%eng.siroe.com in the result parameter and the value -1 to indicate that the rewrite rule succeeded. The percent sign in the result string (see 11.4.2 Repeated Rewrites Template, A%B John.Doe@eng.siroe.com as the address to be rewritten.

On UNIX systems, the site-supplied shared library image should be world readable.

11.6.8 Single Field Substitutions, $&, $!, $*, $#

Single field substitutions extract a single subdomain part from the host/domain specification being rewritten. The available single field substitutions are shown in Table 11–6.

Suppose the address jdoe@eng.siroe.com matches the following rewrite rule:

*.SIROE.COM     $U%$&0.siroe.com@mailhub.siroe.com

Then the result from the template will be jdoe@eng.siroe.com with mailhub.siroe.com used as the routing system.

11.6.9 Unique String Substitutions

Each use of the $W control sequence inserts a text string composed of upper case letters and numbers that is designed to be unique and not repeatable. $W is useful in situation where non-repeating address information must be constructed.

11.6.10 Source-Channel-Specific Rewrite Rules ($M, $N)

It is possible to have rewrite rules that act only in conjunction with specific source channels. This is useful when a short-form name has two meanings:

1. When it appears in a message arriving on one channel.

2. When it appears in a message arriving on a different channel.

Source-channel-specific rewriting is associated with the channel program in use and the channel keywords rules and norules. If norules is specified on the channel associated with an MTA component that is doing the rewriting, no channel-specific rewrite checking is done. If rules is specified on the channel, then channel-specific rule checks are enforced. The keyword rules is the default.

Source-channel-specific rewriting is not associated with the channel that matches a given address. It depends only on the MTA component doing the rewriting and that component’s channel table entry.

Channel-specific rewrite checking is triggered by the presence of a $N or $M control sequence in the template part of a rule. The characters following the $N or $M, up until either an at sign (@), percent sign (%), or subsequent $N, $M, $Q, $C, $T, or $? are interpreted as a channel name.

For example, $Mchannel causes the rule to fail if channel is not currently doing the rewriting. $Nchannel causes the rule to fail if channel is doing the rewriting. Multiple $M and $N clauses may be specified. If any one of multiple $M clauses matches, the rule succeeds. If any of multiple $N clauses matches, the rules will fail.

11.6.11 Destination-Channel-Specific Rewrite Rules ($C, $Q)

It is possible to have rewrite rules whose application is dependent upon the channel to which a message is being enqueued. This is useful when there are two names for some host, one known to one group of hosts and one known to another. By using different channels to send mail to each group, addresses can be rewritten to refer to the host under the name known to each group.

Destination channel-specific rewriting is associated with the channel to which the message is to be dequeued and processed by, and the channel keywords rules and norules on that channel. If norules is specified on the destination channel, no channel-specific rewrite checking is done. If rules is specified on the destination channel, channel-specific rule checks are enforced. The keyword rules is the default.

Destination channel-specific rewriting is not associated with the channel matched by a given address. It depends only on the message’s envelope To: address. When a message is enqueued, its envelope To: address is first rewritten to determine to which channel the message is enqueued. During the rewriting of the envelope To: address, any $C and $Q control sequences are ignored. After the envelope To: address is rewritten and the destination channel determined, then the $C and $Q control sequences are honored, as other addresses associated with the message are rewritten.

Destination-channel-specific rewrite checking is triggered by the presence of a $C or $Q control sequence in the template part of a rule. The characters following the $C or $Q, up until either an at sign (@), percent sign (%), or subsequent $N, $M, $C, $Q, $T, or $? are interpreted as a channel name.

For example, $Qchannel causes the rule to fail if channel is not the destination. For another example, $Cchannel causes the rule to fail if channel is the destination. Multiple $Q and $C clauses may be specified. If any one of multiple $Q clauses matches, the rule succeeds. If any of multiple $C clauses matches, the rule fails.

11.6.12 Direction-and-Location-Specific Rewrite Rules ($B, $E, $F, $R)

Sometimes you need to specify rewrite rules that apply only to envelope addresses or, alternately, only to header addresses. The control sequence $E forces a rewrite to fail if the address being rewritten is not an envelope address. The control sequence $B forces a rewrite to fail if the address being rewritten is not from the message header or body. These sequences have no other effects on the rewrite and may appear anywhere in the rewrite rule template.

Addresses may also be categorized by direction. A forward pointing address is one that originates on a To:, Cc:, Resent-to:, or other header or envelope line that refers to a destination. A backward pointing address is something like a From:, Sender:, or Resent-From:, that refers to a source. The control sequence $F causes the rewrite to be applied if the address is forward pointing. The control sequence $R causes the rewrite to be applied if the address is reverse pointing.

11.6.13 Host-Location-Specific Rewrites ($A, $P, $S, $X)

Circumstances occasionally require rewriting that is sensitive to the location where a host name appears in an address. Host names can appear in several different contexts in an address:

• In a source route

• To the right of the at sign (@)

• To the right of a percent sign (%) in the local-part

• To the left of an exclamation point in the local-part

Under normal circumstances, a host name should be handled in the same way, regardless of where it appears. Some situations might require specialized handling.

Four control sequences are used to control matching on the basis of the host’s location in the address.

• $S specifies that the rule can match a host extracted from a source route.

• $A specifies that the rule can match a host found to the right of the @ sign.

• $P specifies that the rule can match a host found to the right of a % sign.

• $X specifies that the rule can match a host found to the left of an exclamation point (!).

The rule fails if the host is from a location other than the one specified. These sequences can be combined in a single rewrite rule. For example, if $S and $A are specified, the rule matches hosts specified in either a source route or to the right of the at sign. Specifying none of these sequences is equivalent to specifying all of them; the rule can match regardless of location.

11.6.14 Changing the Current Tag Value, $T

The $T control sequence is used to change the current rewrite rule tag. The rewrite rule tag is prepended to all rewrite rule patterns before they are looked up in the configuration file and domain database. Text following the $T, up until either an at sign, percent sign, $N, $M, $Q, $C, $T, or $? is taken to be the new tag.

Tags are useful in handling special addressing forms where the entire nature of an address is changed when a certain component is encountered. For example, suppose that the special host name internet, when found in a source route, should be removed from the address and the resulting address forcibly matched against the TCP-DAEMON channel.

This could be implemented with rules like the following (localhost is assumed to be the official name of the local host):

internet               $S$U@localhost$Tmtcp-force|

mtcp-force|.           $U%$H@TCP-DAEMON

The first rule will match the special host name internet if it appears in the source route. It forcibly matches internet against the local channel, which insures that it will be removed from the address. A rewrite tag is then set. Rewriting proceeds, but no regular rule will match because of the tag. Finally, the default rule is tried with the tag, and the second rule of this set fires, forcibly matching the address against the TCP-DAEMON channel regardless of any other criteria.

11.6.15 Controlling Error Messages Associated with Rewriting ($?)

The MTA provides default error messages when rewriting and channel matching fail. The ability to change these messages can be useful under certain circumstances. For example, if someone tries to send mail to an Ethernet router box, it may be considered more informative to say something like “our routers cannot accept mail” rather than the usual “illegal host/domain specified.”

A special control sequence can be used to change the error message that is printed if the rule fails. The sequence $? is used to specify an error message. Text following the $?, up to either an at sign (@), percent sign (%), $N, $M, $Q, $C, $T, or $? is taken to be the text of the error message to print if the result of this rewrite fails to match any channel. The setting of an error message is “sticky” and lasts through the rewriting process.

A rule that contains a $? operates just like any other rule. The special case of a rule containing only a $? and nothing else receives special attention --- the rewriting process is terminated without changing the mailbox or host portions of the address and the host is looked up as-is in the channel table. This lookup is expected to fail and the error message will be returned as a result.

For example, assume the final rewrite rule in the MTA configuration file is as follows:

. $?Unrecognized address; contact postmaster@siroe.com

In this example, any unrecognized host or domain specifications that can fail will, in the process of failing, generate the error message: Unrecognized address; contact postmaster@siroe.com.