10.3.2 Mapping Operations

All mappings in the mappings file are applied in a consistent way. The only things that change from one mapping to the next is the source of input strings and what the output from the mapping is used for.

A mapping operation always starts off with an input string and a mapping table. The entries in the mapping table are scanned one at a time from top to bottom in the order in which they appear in the table. The left side of each entry is used as pattern, and the input string is compared in a case-blind fashion with that pattern. For details about which MTA processes uses which tables and when, see Table 10–2. This section consists of the following subsections:

• 10.3.2.1 Mapping Entry Patterns

• 10.3.2.2 IP Matching

• 10.3.2.3 Mapping Entry Templates

10.3.2.1 Mapping Entry Patterns

Patterns can contain wildcard characters. In particular, the usual wildcard characters are allowed: an asterisk (*) matches zero or more characters, and each percent sign (%) matches a single character. Asterisks, percent signs, spaces, and tabs can be quoted by preceding them with a dollar sign ($). Quoting an asterisk or percent sign robs it of any special meaning. Spaces and tabs must be quoted to prevent them from ending prematurely a pattern or template. Literal dollar sign characters should be doubled ($$), the first dollar sign quoting the second one.

Within globs, that is, within a $[...] construct, the backslash character, (\) is the quote character. To represent a literal hyphen, -, or right bracket, ], within a glob the hyphen or right bracket must be quoted with a backslash.

All other characters in a pattern just represent and match themselves. In particular, single and double quote characters as well as parentheses have no special meaning in either mapping patterns or templates; they are just ordinary characters. This makes it easy to write entries that correspond to illegal addresses or partial addresses.

To specify multiple modifiers, or to specify modifiers and a back match, the syntax uses just one dollar character. For instance, to back match the initial wild card, without saving the back match itself, one would use $@0, not $@$0.

Note that the imsimta test -match utility may be used to test mapping patterns and specifically to test wildcard behavior in patterns.

Asterisk wildcards maximize what they match by working from left to right across the input string. For instance, when the input string a/b/c is compared to the pattern */*, the left asterisk matches a/b and the right asterisk matches the remainder, c.

The $_ modifier causes wildcard matching to be minimized, where the least possible match is considered the match, working from left to right across the pattern. For instance, when the string a/b/c is compared to the pattern $_*/$_*, the left $_* matches a and the right $_* matches b/c.

10.3.2.2 IP Matching

With IPv4 prefix matching, an IP address or subnet is specified, optionally followed by a slash and the number of bits from the prefix that are significant when comparing for a match. For example, the following matches anything in the 123.45.67.0 subnet:

$(123.45.67.0/24)

With IPv4 ignore bits matching, an IP address or subnet is specified, optionally followed by a slash and the number of bits to ignore when checking for a match. For example, the following matches anything in the 123.45.67.0 subnet:

$<123.45.67.0/8>

The following example matches anything in the range 123.45.67.4 through 123.45.67.7:

$<123.45.67.4/2>

IPv6 matching matches an IPv6 address or subnet. Note that IPv6 connection handling is not currently supported in Messaging Server.

10.3.2.3 Mapping Entry Templates

If the comparison of the pattern in a given entry fails, no action is taken; the scan proceeds to the next entry. If the comparison succeeds, the right side of the entry is used as a template to produce an output string. The template effectively causes the replacement of the input string with the output string that is constructed from the instructions given by the template.

Almost all characters in the template simply produce themselves in the output. The one exception is a dollar sign ($).

A dollar sign followed by a dollar sign, space, or tab produces a dollar sign, space, or tab in the output string. Note that all these characters must be quoted in order to be inserted into the output string.

A dollar sign followed by a digit n calls for a substitution; a dollar sign followed by an alphabetic character is referred to as a “metacharacter.” Metacharacters themselves do not appear in the output string produced by a template, but produce some special substitution or processing. See Table 10–4 for a list of the special substitution and standard processing metacharacters. Any other metacharacters are listed in the sections about specific mappings tables,. See Table 18–1.

This section consists of the following subsections:

• Wildcard Field Substitutions ($n)

• Controlling Text Case ($\, $^, $_)

• Processing Control ($C, $L, $R, $E)

• Check for Special Flags

• Entry Randomly Succeeds or Fails ($?x?)

• Sequence Number Substitutions ($#...#)

• URL substitutions, $]...[

• Mapping Table Substitutions ($|...|)

• General Lookup Table or Database Substitutions (${...})

• Site-Supplied Routine Substitutions ($[...])

• Generate UTF-8 Strings

Wildcard Field Substitutions ($n)

A dollar sign followed by a digit n is replaced with the material that matched the nth wildcard in the pattern. The wildcards are numbered starting with 0. For example, the following entry would match the input string PSI%A::B and produce the resultant output string b@a.psi.siroe.com:

PSI$%*::*    $1@$0.psi.siroe.com

The input string PSI%1234::USER would also match producing USER@1234.psi.siroe.com as the output string. The input string PSIABC::DEF would not match the pattern in this entry and no action would be taken; that is, no output string would result from this entry.

Controlling Text Case ($\, $^, $_)

The metacharacter $\ forces subsequent text to lowercase, $^ forces subsequent text to uppercase, and $_ causes subsequent text to retain its original case. For instance, these metacharacters may be useful when using mappings to transform addresses for which case is significant.

Processing Control ($C, $L, $R, $E)

The $C, $L, $R, and $E metacharacters control whether and when the mapping process terminates. As described in 10.3.2 Mapping Operations, the mapping process normally consists of a single pass, from top to bottom, ending at the first entry with a pattern matching the input string, with the result specified by the template of that entry. Even if the processing of a metacharacter in the template fails, the mapping process would normally terminate because the input string matched the pattern of that entry. The metacharacters $C, $L, and $R cause the mapping process to continue instead of terminate. The metacharacter $E explicitly terminates the mapping process, thus overriding a previous $C on the same line.

The metacharacters $C, $L, and $R can be used to change the input string and continue the mapping process. If the template modifies the output string and $C, $L, or $R are used, the mapping process continues with the output string of the current entry used as the input string for further operations. $C continues with the next entry. $L continues with the next entry, but if no matching entry is found before the end of the table, one more pass will be made starting again at the top of the table. $R continues from the top of the table.

Templates are processed from left to right. When the intention is that an entry should succeed and terminate the mapping process if the template succeeds, but that the mapping process should continue if the template fails, then the entry should use the $C (or $R or $L) metacharacter to the left of the part that may fail and $E to the right of that part.

For example, see the PORT_ACCESS table where the $|...| callout is used in 18.7.4 To Use DNS Lookups Including RBL Checking for SMTP Relay Blocking to check the client IP address against the INTERNAL_IP table. If the lookup succeeds, the template processing ends with $Y and the mapping process is terminated by the $E. But if the lookup fails, the template processing ends with the failure, but the $C causes the mapping process to continue. Because this template generated no output string, the original input string remains unmodified.

Because metacharacters $L and $R reiterate the mapping process, the number of iterative passes through a mapping table is limited to prevent infinite loops. A counter is incremented each time a pass is restarted with a new input string that is the same length or longer than the previous pass. If the resulting output string has a shorter length than the input string, the counter is reset to zero. A request to reiterate a mapping is ignored after the counter has exceeded 10.

Note that any of the metacharacters $C, $E, $L, or $R, when present in the template, influences the mapping process and control whether it terminates or continues. That is, it is possible to set up iterative mapping table entries, where the output of one entry becomes the input of another entry. If the template of a matching pattern does not contain any of the metacharacters $C, $E, $L, or $R, then $E (immediate termination of the mapping process) is assumed.

The number of iterative passes through a mapping table is limited to prevent infinite loops. A counter is incremented each time a pass is restarted with a pattern that is the same length or longer than the previous pass. If the string has a shorter length than previously, the counter is reset to zero. A request to reiterate a mapping is not honored after the counter has exceeded 10.

Check for Special Flags

Some mapping probes have special flags set. These are flags that can be set, and then their presence/absence tested-for using the general mapping table facility of $:, $; tests. $:x causes an entry to match only if the flag x is set. $;x causes an entry to match only if the flag x is clear. See specific mapping table descriptions Table 10–2for any special flags that may apply for that table.

When the intention is that an entry should succeed and terminate if the flag check succeeds, but that the mapping process should continue if the flag check fails, then the entry should use the $C metacharacter to the left of the flag check and use the $E flag to the right of the flag check.

Entry Randomly Succeeds or Fails ($?x?)

The metacharacters $?x? in a mapping table entry cause the entry to “succeed” x percent of the time; the rest of the time, the entry “fails” and the output of the mapping entry's input is taken unchanged as the output. (Note that, depending upon the mapping, the effect of the entry failing is not necessarily the same as the entry not matching in the first place.) The x should be a real number specifying the success percentage.

For instance, suppose that a system with IP address 123.45.6.78 is sending your site just a little too much SMTP email and you'd like to slow it down; you can use a PORT_ACCESS mapping table in the following way. Suppose you'd like to allow through only 25 percent of its connection attempts and reject the other 75 percent of its connection attempts. The following PORT_ACCESS mapping table uses $?25? to cause the entry with the $Y (accept the connection) to succeed only 25 percent of the time; the other 75 percent of the time, when this entry fails, the initial $C on that entry causes the MTA to continue the mapping from the next entry, which causes the connection attempt to be rejected with an SMTP error and the message: Try again later.

PORT_ACCESS

   TCP|*|25|123.45.6.78|*         $C$?25?$Y
   TCP|*|25|123.45.6.78|*         $N45s$ 4.40$ Try$ again$ later

Sequence Number Substitutions ($#...#)

A $#...# substitution increments the value stored in an MTA sequence file and substitutes that value into the template. This can be used to generate unique, increasing strings in cases where it is desirable to have a unique qualifier in the mapping table output; for instance, when using a mapping table to generate file names.

Permitted syntax is any one of the following:

$#seq-file-spec|radix|width|m#

$#seq-file-spec|radix|width#

$#seq-file-spec|radix#

$#seq-file-spec#

The required seq-file-spec argument is a full file specification for an already existing MTA sequence file. The optional radix and width arguments specify the radix (base) in which to output the sequence value, and the number of digits to output, respectively. The default radix is 10. Radices in the range -36 to 36 are also allowed; for instance, base 36 gives values expressed with digits 0,...,9,A,...,Z. By default, the sequence value is printed in its natural width, but if the specified width calls for a greater number of digits, then the output is padded with 0s on the left to obtain the correct number of digits. Note that if a width is explicitly specified, then the radix must be explicitly specified also.

The optional m argument is a modulus. If this fourth argument is specified the value inserted is the sequence number retrieved from the file mod m. The default is not to perform any modulus operation.

As noted above, the MTA sequence file referred to in a mapping must already exist. To create an MTA sequence file, use the following UNIX command:

touch seq-file-spec

or

cat >seq-file-spec

               

A sequence number file accessed using a mapping table must be world readable in order to operate properly. You must also have an MTA user account (configured to be nobody in the imta_tailor file) in order to use such sequence number files.

URL substitutions, $]...[

A substitution of the form $]url[ is specially handled. url can be any supported URL type including file: and data:.Standard LDAP URLs can also be used, with the host and port omitted; the host and part are instead specified with the LDAP_HOST and LDAP_PORT options. That is, the LDAP URL should be specified as:

ldap:///dn[?attributes[?scope?filter]]

where the square bracket characters [ and ] shown above indicate optional portions of the URL. The dn is required and is a distinguished name specifying the search base. The optional attributes, scope, and filter portions of the URL further refine the information to return. That is, attributes specifies the attribute or attributes to be returned from LDAP directory entries matching this LDAP query. The scope may be any of base (the default), one, or sub. filter describes the characteristics of matching entries.

Certain LDAP URL substitution sequences are available for use within the LDAP query URL. The length of URLs can be 1024 characters. This also applies to expressions created by mappings and mapping calls to other mappings.

Mapping Table Substitutions ($|...|)

A substitution of the form $|mapping;argument| is handled specially. The MTA looks for an auxiliary mapping table named mapping in the MTA mappings file, and uses argument as the input to that named auxiliary mapping table. The named auxiliary mapping table must exist and must set the $Y flag in its output if it is successful; if the named auxiliary mapping table does not exist or doesn't set the $Y flag, then that auxiliary mapping table substitution fails and the original mapping entry is considered to fail: the original input string is used as the output string.

Note that when you want to use processing control metacharacters such as $C, $R, or $L in a mapping table entry that does a mapping table substitution, the processing control metacharacter should be placed to the left of the mapping table substitution in the mapping table template; otherwise the “failure” of a mapping table substitution means that the processing control metacharacter is not seen.

General Lookup Table or Database Substitutions (${...})

A substitution of the form ${text} is handled specially. The text part is used as a key to access the general lookup table or database (see 10.9.1 MTA Text Databases for more information). If text is found in the table, the corresponding template from the table is substituted. If text does not match an entry in the table, the input string is used unchanged as the output string.

If you are using the general lookup table you need to set the low order bit of the MTA option use_text_databases. That is, set it to an odd number. Changes to the general.txt need to be compiled into the MTA configuration using the imsimta cnbuild to compile and imsimta reload to reload the reloadable data.

If you are using a general database, it should be world readable to insure that it operates properly.

When you want to use processing control metacharacters such as $C, $R, or $L in a mapping table entry that does a general table substitution, the processing control metacharacter should be placed to the left of the general table substitution in the mapping table template; otherwise the “failure” of a general table substitution means that the processing control metacharacter is not seen.

Site-Supplied Routine Substitutions ($[...])

A substitution of the form $[image,routine,argument] is handled specially. The image, routine, argument part is used to find and call a customer-supplied routine. At runtime on UNIX, the MTA uses dlopen and dlsym to dynamically load and call the routine routine from the shared library image. The routine routine is then called as a function with the following argument list:

status = routine (argument, arglength, result, reslength)

The argument and result are 252-byte long character string buffers. The argument and result are passed as a pointer to a character string (for example, in C, as char*). The arglength and reslength are signed, long integers passed by reference. On input, argument contains the argument string from the mapping table template, and arglength the length of that string. On return, the resultant string should be placed in result and its length in reslength. This resultant string then replaces the $[image,routine,argument] in the mapping table template. The routine routine should return 0 if the mapping table substitution should fail and -1 if the mapping table substitution should succeed. If the substitution fails, then normally the original input string is used unchanged as the output string.

If you want to use processing control metacharacters such as $C, $R, or $L in a mapping table entry that does a site-supplied routine substitution, you place the processing control metacharacter to the left of the site-supplied routine substitution in the mapping table template; otherwise, the “failure” of a mapping table substitution means that the processing control metacharacter is not seen.

The site-supplied routine callout mechanism allows the MTA's mapping process to be extended in all sorts of complex ways. For example, in a PORT_ACCESS or ORIG_SEND_ACCESS mapping table, a call to some type of load monitoring service could be performed and the result used to decide whether or not to accept a connection or message.

The site-supplied shared library image image should be world readable.

Generate UTF-8 Strings

You can generate UTF-8 strings from Unicode character values in the general mapping table facility. A Unicode metacharacter sequence of the form:

$&A0A0,20,A1A1&

produces a UTF-8 string containing the characters at position A0A0, 20, and A1A1 in it.