Mappings File

Many components of the MTA employ table lookup-oriented information. This type of table is used to transform, that is, map, an input string into an output string. Such tables, called mapping tables, are usually presented as two columns. The first (left-hand) column provides possible input strings against which to match (pattern), and the second (right-hand) column gives the resulting output string for which the input string is mapped (template).

Most of the MTA databases that contain different types of MTA data and which should not be confused with mapping tables—are instances of just this type of table. The MTA database files, however, do not provide wildcard-lookup facilities, owing to inherent inefficiencies in having to scan the entire database for wildcard matches.

The MTA mappings file supports multiple mapping tables. Wildcard capabilities are provided, as well as multistep and iterative mapping methods. This approach is more compute-intensive than using a database, especially when the number of entries is large. However, the attendant gain in flexibility may serve to eliminate the need for most of the entries in an equivalent database, and this may result in lower overhead overall.

Mapping tables are kept in the MTA mappings file. This is the file specified with the IMTA_MAPPING_FILE option in the MTA tailor file; by default, this is msg_svr_base/config/mappings. The contents of the mappings file is incorporated into the compiled configuration as part of the reloadable section (seeCompiling the MTA Configuration Failure to allow world-read access leads to erratic behavior. Whenever changes are made to the mappings file, the MTA configuration must be recompiled. See Compiling the MTA Configuration

Table 10–2 lists the mapping tables described in this book.

File Format in the Mappings File

The mappings file consists of a series of separate tables. Each table begins with its name. Names always have an alphabetic character in the first column. The table name is followed by a required blank line, and then by the entries in the table. Entries consist of zero or more indented lines. Each entry line consists of two columns separated by one or more spaces or tabs. Any spaces within an entry must be quoted using the $ character. A blank line must appear after each mapping table name and between each mapping table; no blank lines can appear between entries in a single table. Comments are introduced by an exclamation mark (!) in the first column.

The resulting format looks like:

TABLE1_NAME
   pattern1-1    template1-1
   pattern1-2    template1-2
   pattern1-3    template1-3
      .              .
      .              .
      .              .
   pattern1-n    template1-n

TABLE2_NAME
   pattern2-1    template2-1
   pattern2-2    template2-2
   pattern2-3    template2-3
       .            .
       .            .
       .            .
   pattern2-n    template2-n
          .
          .
          .


TABLE3_NAME
           .
          .
          .

         

An application using the mapping table TABLE2_NAME would map the string pattern2-2 into whatever is specified by template2-2. Each pattern or template can contain up to 252 characters. There is no limit to the number of entries that can appear in a mapping (although excessive numbers of entries may consume huge amounts of CPU and can consume excessive amounts of memory). Long lines (over 252 characters) may be continued by ending them with a backslash (\). The white space between the two columns and before the first column may not be omitted.

Duplicate mapping table names are not allowed in the mappings file.

Including Other Files in the Mappings File

Other files may be included in the mappings file. This is done with a line of the form:

<file-spec
            

This effectively substitutes the contents of the file file-spec into the mappings file at the point where the include appears. The file specification should specify a full file path (directory, and so forth). All files included in this fashion must be world readable. Comments are also allowed in such included mappings file. Includes can be nested up to three levels deep. Include files are loaded at the same time the mappings file is loaded—they are not loaded on demand, so there is no performance or memory savings involved in using include files.

Mapping Operations

All mappings in the mappings file are applied in a consistent way. The only things that change from one mapping to the next is the source of input strings and what the output from the mapping is used for.

A mapping operation always starts off with an input string and a mapping table. The entries in the mapping table are scanned one at a time from top to bottom in the order in which they appear in the table. The left side of each entry is used as pattern, and the input string is compared in a case-blind fashion with that pattern.

Mapping Entry Patterns

Patterns can contain wildcard characters. In particular, the usual wildcard characters are allowed: an asterisk (*) matches zero or more characters, and each percent sign (%) matches a single character. Asterisks, percent signs, spaces, and tabs can be quoted by preceding them with a dollar sign ($). Quoting an asterisk or percent sign robs it of any special meaning. Spaces and tabs must be quoted to prevent them from ending prematurely a pattern or template. Literal dollar sign characters should be doubled ($$), the first dollar sign quoting the second one.

Within globs, that is, within a $[...] construct, the backslash character, (\) is the quote character. To represent a literal hyphen, -, or right bracket, ], within a glob the hyphen or right bracket must be quoted with a backslash.

All other characters in a pattern just represent and match themselves. In particular, single and double quote characters as well as parentheses have no special meaning in either mapping patterns or templates; they are just ordinary characters. This makes it easy to write entries that correspond to illegal addresses or partial addresses.

To specify multiple modifiers, or to specify modifiers and a back match, the syntax uses just one dollar character. For instance, to back match the initial wild card, without saving the back match itself, one would use $@0, not $@$0.

Note that the imsimta test -match utility may be used to test mapping patterns and specifically to test wildcard behavior in patterns.

Asterisk wildcards maximize what they match by working from left to right across the input string. For instance, when the input string a/b/c is compared to the pattern */*, the left asterisk matches a/b and the right asterisk matches the remainder, c.

The $_ modifier causes wildcard matching to be minimized, where the least possible match is considered the match, working from left to right across the pattern. For instance, when the string a/b/c is compared to the pattern $_*/$_*, the left $_* matches a and the right $_* matches b/c.

IP Matching

With IPv4 prefix matching, an IP address or subnet is specified, optionally followed by a slash and the number of bits from the prefix that are significant when comparing for a match. For example, the following matches anything in the 123.45.67.0 subnet:

$(123.45.67.0/24)

With IPv4 ignore bits matching, an IP address or subnet is specified, optionally followed by a slash and the number of bits to ignore when checking for a match. For example, the following matches anything in the 123.45.67.0 subnet:

$<123.45.67.0/8>

The following example matches anything in the range 123.45.67.4 through 123.45.67.7:

$<123.45.67.4/2>

IPv6 matching matches an IPv6 address or subnet.

Mapping Entry Templates

If the comparison of the pattern in a given entry fails, no action is taken; the scan proceeds to the next entry. If the comparison succeeds, the right side of the entry is used as a template to produce an output string. The template effectively causes the replacement of the input string with the output string that is constructed from the instructions given by the template.

Almost all characters in the template simply produce themselves in the output. The one exception is a dollar sign ($).

A dollar sign followed by a dollar sign, space, or tab produces a dollar sign, space, or tab in the output string. Note that all these characters must be quoted in order to be inserted into the output string.

A dollar sign followed by a digit n calls for a substitution; a dollar sign followed by an alphabetic character is referred to as a “metacharacter.” Metacharacters themselves do not appear in the output string produced by a template, but produce some special substitution or processing. See Table 10–4 for a list of the special substitution and standard processing metacharacters. Any other metacharacters are reserved for mapping-specific applications.

Note that any of the metacharacters $C, $E, $L, or $R, when present in the template of a matching pattern, influences the mapping process and control whether it terminates or continues. That is, it is possible to set up iterative mapping table entries, where the output of one entry becomes the input of another entry. If the template of a matching pattern does not contain any of the metacharacters $C, $E, $L, or $R, then $E (immediate termination of the mapping process) is assumed.

The number of iterative passes through a mapping table is limited to prevent infinite loops. A counter is incremented each time a pass is restarted with a pattern that is the same length or longer than the previous pass. If the string has a shorter length than previously, the counter is reset to zero. A request to reiterate a mapping is not honored after the counter has exceeded 10.

Wildcard Field Substitutions ($n)

A dollar sign followed by a digit n is replaced with the material that matched the nth wildcard in the pattern. The wildcards are numbered starting with 0. For example, the following entry would match the input string PSI%A::B and produce the resultant output string b@a.psi.siroe.com:

PSI$%*::*    $1@$0.psi.siroe.com

The input string PSI%1234::USER would also match producing USER@1234.psi.siroe.com as the output string. The input string PSIABC::DEF would not match the pattern in this entry and no action would be taken; that is, no output string would result from this entry.

Controlling Text Case ($\, $^, $_)

The metacharacter $\ forces subsequent text to lowercase, $^ forces subsequent text to uppercase, and $_ causes subsequent text to retain its original case. For instance, these metacharacters may be useful when using mappings to transform addresses for which case is significant.

Processing Control ($C, $L, $R, $E)

The $C, $L, $R, and $E metacharacters influence the mapping process, controlling whether and when the mapping process terminates. The metacharacter:

• $C causes the mapping process to continue with the next entry, using the output string of the current entry as the new input string for the mapping process.

• $L causes the mapping process to continue with the next entry, using the output string of the current entry as the new input string for the mapping process, and, if no matching entry is found, making one more pass through the table starting with the first table entry; a subsequent matching entry with a $C, $E, or $R metacharacter overrides this condition.

• $R causes the mapping process to continue from the first entry of the table, using the output string of the current entry as the new input string for the mapping process.

• $E causes the mapping process to terminate; the output string of this entry is the final output. $E is the default.

Mapping table templates are scanned left to right. To set a $C, $L, or $R flag for entries that may “succeed” or “fail” (for example, general database substitutions or random-value controlled entries), put the $C, $L, or $R metacharacter to the left of the part of the entry that may succeed or fail; otherwise, if the remainder of the entry fails, the flag is not seen.

Check for Special Flags

Some mapping probes have special flags set. These are flags that can be set, and then their presence/absence tested-for using the general mapping table facility of $:, $; tests. $:x causes an entry to match only if the flag x is set. $;x causes an entry to match only if the flag x is clear. See specific mapping table descriptions for any special flags that may apply for that table. (See $A, $T, $S, $F, and $D in Table 17–2.

When the intention is that an entry should succeed and terminate if the flag check succeeds, but that the mapping process should continue if the flag check fails, then the entry should use the $C metacharacter to the left of the flag check and use the $E flag to the right of the flag check.

Entry Randomly Succeeds or Fails ($?x?)

The metacharacters $?x? in a mapping table entry cause the entry to “succeed” x percent of the time; the rest of the time, the entry “fails” and the output of the mapping entry's input is taken unchanged as the output. (Note that, depending upon the mapping, the effect of the entry failing is not necessarily the same as the entry not matching in the first place.) The x should be a real number specifying the success percentage.

For instance, suppose that a system with IP address 123.45.6.78 is sending your site just a little too much SMTP email and you'd like to slow it down; you can use a PORT_ACCESS mapping table in the following way. Suppose you'd like to allow through only 25 percent of its connection attempts and reject the other 75 percent of its connection attempts. The following PORT_ACCESS mapping table uses $?25? to cause the entry with the $Y (accept the connection) to succeed only 25 percent of the time; the other 75 percent of the time, when this entry fails, the initial $C on that entry causes the MTA to continue the mapping from the next entry, which causes the connection attempt to be rejected with an SMTP error and the message: Try again later.

PORT_ACCESS

   TCP|*|25|123.45.6.78|*         $C$?25?$Y
   TCP|*|25|123.45.6.78|*         $N45s$ 4.40$ Try$ again$ later

Sequence Number Substitutions ($#...#)

A $#...# substitution increments the value stored in an MTA sequence file and substitutes that value into the template. This can be used to generate unique, increasing strings in cases where it is desirable to have a unique qualifier in the mapping table output; for instance, when using a mapping table to generate file names.

Permitted syntax is any one of the following:

$#seq-file-spec|radix|width|m#

$#seq-file-spec|radix|width#

$#seq-file-spec|radix#

$#seq-file-spec#

The required seq-file-spec argument is a full file specification for an already existing MTA sequence file. The optional radix and width arguments specify the radix (base) in which to output the sequence value, and the number of digits to output, respectively. The default radix is 10. Radices in the range -36 to 36 are also allowed; for instance, base 36 gives values expressed with digits 0,...,9,A,...,Z. By default, the sequence value is printed in its natural width, but if the specified width calls for a greater number of digits, then the output is padded with 0's on the left to obtain the correct number of digits. Note that if a width is explicitly specified, then the radix must be explicitly specified also.

The optional m argument is a modulus. If this fourth argument is specified the value inserted is the sequence number retrieved from the file mod m. The default is not to perform any modulus operation.

As noted above, the MTA sequence file referred to in a mapping must already exist. To create an MTA sequence file, use the following UNXI command:

touch seq-file-spec

or

cat >seq-file-spec

               

A sequence number file accessed using a mapping table must be world readable in order to operate properly. You must also have an MTA user account (configured to be nobody in the imta_tailor file) in order to use such sequence number files.

LDAP query URL substitutions, $]...[

A substitution of the form $]ldap-url[ is specially handled. ldap-url is interpreted as an LDAP query URL and the result of the LDAP query is substituted. Standard LDAP URLs are used, with the host and port omitted; the host and part are instead specified with the LDAP_HOST and LDAP_PORT options. That is, the LDAP URL should be specified as:

ldap:///dn[?attributes[?scope?filter]]

where the square bracket characters [ and ] shown above indicate optional portions of the URL. The dn is required and is a distinguished name specifying the search base. The optional attributes, scope, and filter portions of the URL further refine the information to return. That is, attributes specifies the attribute or attributes to be returned from LDAP directory entries matching this LDAP query. The scope may be any of base (the default), one, or sub. filter describes the characteristics of matching entries.

Certain LDAP URL substitution sequences are available for use within the LDAP query URL.

Mapping Table Substitutions ($|...|)

A substitution of the form $|mapping;argument| is handled specially. The MTA looks for an auxiliary mapping table named mapping in the MTA mappings file, and uses argument as the input to that named auxiliary mapping table. The named auxiliary mapping table must exist and must set the $Y flag in its output if it is successful; if the named auxiliary mapping table does not exist or doesn't set the $Y flag, then that auxiliary mapping table substitution fails and the original mapping entry is considered to fail: the original input string is used as the output string.

Note that when you want to use processing control metacharacters such as $C, $R, or $L in a mapping table entry that does a mapping table substitution, the processing control metacharacter should be placed to the left of the mapping table substitution in the mapping table template; otherwise the “failure” of a mapping table substitution means that the processing control metacharacter is not seen.

General Lookup Table or Database Substitutions (${...})

A substitution of the form ${text} is handled specially. The text part is used as a key to access the general lookup table or database. The database is generated with the imsimta crdb utility. If text is found in the table, the corresponding template from the table is substituted. If text does not match an entry in the table, the input string is used unchanged as the output string.

If you are using the general lookup table you need to set the low order bit of the MTA option use_text_databases. That is, set it to an odd number. Changes to the general.txt need to be compiled into the MTA configuration using the imsimta cnbuild to compile and imsimta reload to reload the reloadable data.

If you are using a general database, it should be world readable to insure that it operates properly.

When you want to use processing control metacharacters such as $C, $R, or $L in a mapping table entry that does a general table substitution, the processing control metacharacter should be placed to the left of the general table substitution in the mapping table template; otherwise the “failure” of a general table substitution means that the processing control metacharacter is not seen.

Site-Supplied Routine Substitutions ($[...])

A substitution of the form $[image,routine,argument] is handled specially. The image, routine, argument part is used to find and call a customer-supplied routine. At runtime on UNIX, the MTA uses dlopen and dlsym to dynamically load and call the routine routine from the shared library image. The routine routine is then called as a function with the following argument list:

status = routine (argument, arglength, result, reslength)

The argument and result are 252-byte long character string buffers. The argument and result are passed as a pointer to a character string (for example, in C, as char*). The arglength and reslength are signed, long integers passed by reference. On input, argument contains the argument string from the mapping table template, and arglength the length of that string. On return, the resultant string should be placed in result and its length in reslength. This resultant string then replaces the $[image,routine,argument] in the mapping table template. The routine routine should return 0 if the mapping table substitution should fail and -1 if the mapping table substitution should succeed. If the substitution fails, then normally the original input string is used unchanged as the output string.

If you want to use processing control metacharacters such as $C, $R, or $L in a mapping table entry that does a site-supplied routine substitution, you place the processing control metacharacter to the left of the site-supplied routine substitution in the mapping table template; otherwise, the “failure” of a mapping table substitution means that the processing control metacharacter is not seen.

The site-supplied routine callout mechanism allows the MTA's mapping process to be extended in all sorts of complex ways. For example, in a PORT_ACCESS or ORIG_SEND_ACCESS mapping table, a call to some type of load monitoring service could be performed and the result used to decide whether or not to accept a connection or message.

The site-supplied shared library image image should be world readable.

Generate UTF-8 Strings

You can generate UTF-8 strings from Unicode character values in the general mapping table facility. A Unicode metacharacter sequence of the form:

$&A0A0,20,A1A1&

produces a UTF-8 string containing the characters at position A0A0, 20, and A1A1 in it.