Sun Java System Messaging Server 6.3 Administration Guide

Chapter 13 Using Predefined Channels

When you first install Messaging Server, several channels are already defined (see Table 13–1). This chapter describes how to use pre-defined channel definitions in the MTA.

If you have not already read Chapter 10, About MTA Services and Configuration you should do so before reading this chapter. For information about configuring the rewrite rules in the imta.cnf file, see Chapter 11, Configuring Rewrite Rules.

This chapter contains the following sections:

The defaults Channel is described in 12.1 Configuring Channel Defaults.

13.1 Predefined Channels

The table below lists some of the predefined channels.

Table 13–1 Predefined Channels

Channel 

Definition 

defaults

Used to specify which keywords are defaults for various channels. See 12.1 Configuring Channel Defaults.

l

UNIX only. Used to make routing decisions and for submitting mail using UNIX mail tools. 

ims-ms

Performs final delivery of mail to the local store. 

native

UNIX only. Delivers mail to /var/mail. (Note that Messaging Server does not support /var/mail access. User must use UNIX tools to access mail from the /var/mail store.)

pipe

Used to perform delivery via a site-supplied program or script. Commands executed by the pipe channel are controlled by the administrator via the imsimta program interface. 

reprocessprocess

These channels are used for deferred, offline message processing. The reprocess channel is normally invisible as a source or destination channel; the process channel is visible like other MTA channels.

defragment

Provides the means to reassemble MIME fragmented messages. 

conversion

Performs body-part-by-body-part conversions on messages flowing through the MTA. 

bitbucket

Used for messages that need to be discarded. 

inactive/deleted

Used to process messages for users who have been marked as inactive/deleted in the directory. Typically, bounces the message and returns custom bounce message to the sender of the message. 

hold

Used to hold messages for users. For example, when a user is migrated from one mail server to another. 

sms

Provides support for one-way email to an SMS gateway. 


tcp_local
tcp_intranet
tcp_auth
tcp_submit
tcp_tas

Implements SMTP over TCP/IP. The multithreaded TCP SMTP channel includes a multithreaded SMTP server that runs under the control of the Dispatcher. Outgoing SMTP mail is processed by the channel program tcp_smtp_client, and runs as needed under the control of the Job Controller. 

tcp_local receives inbound messages from remote SMTP hosts. Depending on whether you use a smarthost/firewall configuration, either sends outbound messages directly to remote SMTP hosts or sends outbound messages to the smarthost/firewall system. Sometimes tcp_local gets mail from remote SMTP hosts via proxy or firewall. tcp_local is also sometimes used for internal relay activities.

tcp_intranet receives and sends messages within the intranet.

tcp_auth is used as a switch channel for tcp_local; authenticated users switch to the tcp_auth channel to avoid relay-blocking restrictions.

tcp_submit accepts message submissions—usually from user agents—on the reserved submission port 587 (see RFC 2476).

tcp_tas is a special channel used by sites doing Unified Messaging.

13.2 To Deliver Messages to Programs Using the Pipe Channel

Users might want incoming mail passed to a program instead of to their mailbox. For example, users might want their incoming mail sent to a mail sorting program. The pipe channel performs delivery of messages using per-user, site-supplied programs.

To facilitate program delivery, you must first register programs as able to be invoked by the pipe channel. Do this by using the imsimta program utility. This utility gives a unique name to each command that you register as able to be invoked by the pipe channel. End users can then specify the method name as a value of their mailprogramdeliveryinfo LDAP attribute.

For example, to add a UNIX command myprocmail as a program that can be invoked by a user, you would first register the command by using the imsimta program utility as shown in the following example. This example registers a program called myprocmail that executes the program procmail with the arguments -d username and executes as the user:

imsimta program -a -m myprocmail -p procmail -g "-d %s" -e user

Make sure the executable exists in the programs directory msg-svr-base/data/site-programs. Make sure also that the execute permissions are set to “others.”

To enable a user to access the program, the user’s LDAP entry must contain the following attributes and values:

maildeliveryoption: program 
mailprogramdeliveryinfo: myprocmail

For more information about the imsimta program utility, see the Sun Java System Messaging Server 6.3 Administration Reference.

Alternative delivery programs must conform to the following exit code and command-line argument restrictions:

Exit Code Restrictions. Delivery programs invoked by the pipe channel must return meaningful error codes so that the channel knows whether to dequeue the message, deliver for later processing, or return the message.

If the subprocess exits with an exit code of 0 (EX_OK), the message is presumed to have been delivered successfully and is removed from the MTA queues. If it exits with an exit code of 71, 74, 75, or 79 (EX_OSERR, EX_IOERR, EX_TEMPFAIL, or EX_DB), a temporary error is presumed to have occurred and delivery of the message is deferred. If any other exit code is returned, then the message will be returned to its originator as undeliverable. These exit codes are defined in the system header file sysexits.h.

Command Line Arguments. Delivery programs can have any number of fixed arguments as well as the variable argument, %s, representing the user name for programs executed by the user or username+domain for programs executed by the postmaster, “inetmail.” For example, the following command line delivers a recipient’s mail using the program procmail:

/usr/lib/procmail -d %s

13.3 To Configure the Native (/var/mail) Channel

An option file may be used to control various characteristics of the native channel. This native channel option file must be stored in the MTA configuration directory and named native_option (for example, msg-svr-base/config/native_option).

Option files consist of several lines. Each line contains the setting for one option. An option setting has the form:

option=value

The value may be either a string or an integer, depending on the option's requirements.

Table 13–2 Local Channel Options

Options 

Descriptions 

FORCE_CONTENT_LENGTH

(0 or 1; UNIX only) 

If FORCE_CONTENT_LENGTH=1, then the MTA adds a Content-length: header line to messages delivered to the native channel, and causes the channel not to use the “>From” syntax when “From” is at the beginning of the line. This makes local UNIX mail compatible with Sun’s newer mail tools, but potentially incompatible with other UNIX mail tools.

FORWARD_FORMAT (string)

Specifies the location of the users’ .forward files. The string %u indicates that it is substituted in each user id. The string %h indicates that it is substituted in each user’s home directory. The default behavior, if this option is not explicitly specified, corresponds to:

FORWARD_FORMAT=%h/.forward


REPEAT_COUNT (integer)
SLEEP_TIME (integer)

In case the user’s new mail file is locked by another process when the MTA tries to deliver the new mail, these options provide a way to control the number and frequency of retries the native channel program should attempt. If the file can not be opened after the number of retries specified, the messages remain in the native queue and the next run of the native channel attempts to deliver the new messages again. 

The REPEAT_COUNT option controls how many times the channel programs attempt to open the mail file before giving up. REPEAT_COUNT defaults to 30, (30 attempts).

The SLEEP_TIME option controls how many seconds the channel program waits between attempts. SLEEP_TIME defaults to 2 (two seconds between retries).

SHELL_TIMEOUT (integer)

Controls the length of time in seconds the channel waits for a user’s shell command in a .forward to complete. Upon such time-outs, the message are returned to the original sender with an error message resembling “Time-out waiting for user’s shell command command to complete.” The default is 600 (10 minutes).

SHELL_TMPDIR (directory-specific)

Controls the location where the local channel creates its temporary files when delivering to a shell command. By default, such temporary files are created in users’ home directories. Using this option, the administrator may instead choose the temporary files to be created in another (single) directory. For example: 

SHELL_TMPDIR=/tmp

13.4 To Temporarily Hold Messages Using the Hold Channel

The hold channel is used to hold the messages of a recipient temporarily prevented from receiving new messages. Messages may be held because a user’s name is being changed or their mailbox is being moved from one mailhost or domain to another. There may also be other reasons to temporarily hold messages.

When messages are to be held, they are directed to the hold channel, in the msg-svr-base/queue/hold directory, using the same mechanism used to direct messages to the reprocess channel. In this way, the envelope To: addresses are unchanged. The messages are written to the hold channel queue, in the msg-server/queue/hold directory, as ZZxxx.HELD files. This prevents them from being seen by the job controller, and thus they are “held.” Use the imsimta qm dir -held command to view a list of .HELD files. These messages can be selected and released using the imsimta qm release command. Releasing them changes their name to ZZxxx.00 and informs the job controller. The messages are then processed by the master program associated with the hold channel, reprocess.exe. Thus the message (and the To: addresses) are processed using the normal rewriting machinery.

For more information on the imsimta qm command, see imsimta qm in Sun Java System Messaging Server 6.3 Administration Reference.

13.5 The Conversion Channel

The conversion channel allows you to perform arbitrary body part-by-body part processing on specified messages flowing through the MTA. (Note that a body part is different than a message in that a message can contain multiple body parts as, for instance, in an attachment. Also, body parts are specified and delineated by MIME headers.) This processing can be done by any site-supplied programs or command procedures and may do such things such as convert text or images from one format to another, virus scanning, language translation and so forth. Various message types of the MTA traffic are selected for conversion, and specific processes and programs can be specified for each type of message body part.

The prerequisite for using this chapter is understanding the concept of channels (see 8.5 Channels). For supplemental information on virus scanning using the conversion channel, refer to Virus Screening with the iPlanet Messaging Server Conversion Channel.

Implementing the conversion channel consists of A) selecting message traffic for processing, and B) specifying how different messages will be processed. These procedures will discussed in further detail.


Note –

A default conversion channel is automatically created in the MTA configuration file (imta.cnf). This channel can be used as is and requires no modification.


This section consists of the following sections:

13.5.1 MIME Overview

The conversion channel makes extensive use of the MIME (Multipurpose Internet Mail Extensions) header lines. Knowledge of message construction and MIME header fields is required. For complete information on MIME, refer to RFCs 1806, 2045 through 2049, and 2183. A short overview of MIME is presented here for convenience.

13.5.1.1 Message Construction

A simple message consists of a header and a body. The header is at the top of the message and contains certain control information such as date, subject, sender, and recipient. The body is everything after the first blank line after the header. MIME specifies a way to construct more complex messages which can contain multiple body parts, and even body parts nested within body parts. Messages like these are called multi-part messages, and, as mentioned earlier, the conversion channel performs body part-by-body part processing of messages.

13.5.1.2 MIME Headers

The MIME specification defines a set of header lines for body parts. These include MIME-Version, Content-type, Content-Transfer-Encoding, Content-ID, and Content-disposition. The conversion channel uses the Content-type and Content-disposition headers most frequently. An example of some MIME header lines is shown below:


Content-type: APPLICATION/wordperfect5.1;name=Poem.wpc
Content-transfer-encoding: BASE64
Content-disposition: attachment; filename=Poem.wpc
Content-description: "Project documentation Draft1 wordperfect format"

Note –

MIME header lines are not the same as general, non-MIME header lines such as To:, Subject: and From:. Basically, for Conversion channel discussion, MIME header lines start with the string Content-.


Content-type Header

The MIME Content-Type header describes the content of the body-part. The Content-Type header format (with an example) is shown below:

Content-type: type/subtype; parameter=value; parameter=value...

type describes the type of content of the body part. Examples of type are Text, Multipart, Message, Application, Image, Audio, and Video.

subtype further describes content type. Each Content-type has its own set of subtypes. For examples: text/plain, application/octet-stream, and image/jpeg. Content Subtypes for MIME mail are assigned and listed by the IANA (Internet Assigned Numbers Authority). A copy of the list is at http://www.iana.org/assignments/media-types.

parameter is specific to Content-type/subtype pairs. For example, the charset and the name parameters are shown below:


Content-type: text/plain; charset=us-ascii
Content-type: application/msword; name=temp.doc

The charset parameter specifies a character set for a textual message. The name parameter gives a suggested file name to be used if the data were to be written to a file.


Note –

Content-Type values, subtypes, and parameter names are case-insensitive.


Content-disposition Header

The MIME Content-disposition header provides presentation information for the body-part. It is often added to attachments specifying whether the attachment body part should be displayed (inline) or presented as a file name to be copied (attachment). The Content-disposition header has the following format:

Content-disposition: disposition_type; parameter=value;parameter=value...

disposition_type is usually inline (display the body part) or attachment (present as file to save.) Attachment usually has the parameter filename with a value specifying the suggested name for the saved file.

For details on the Content-disposition header, refer to RFC2183.

13.5.2 Selecting Traffic for Conversion Processing

Unlike other MTA channels, the conversion channel is not normally specified in an address or MTA rewrite rule. Instead, messages are sent through the conversion channel if they meet the criteria specified in the CONVERSIONS mapping table (the name of the mappings file is specified by the parameter IMTA_MAPPING_FILE in the imta_tailor file. The default is msg_srv_base/conversions). Entries to the table have the following format:

IN-CHAN=source-channel;OUT-CHAN=destination-channel;CONVERT Yes/No

As the MTA processes each message it probes the CONVERSIONS mapping table (if one is present). If the source-channel is the channel from which the message is coming and destination-channel is the channel to which the message is going, then the action following CONVERT is taken. Yes means the MTA diverts the message through the conversion channel on its way to its destination-channel. If no match is found, or if No was specified, the message will be queued to the regular destination channel.

If you route messages to the conversion channel using conversion mappings your other mappings should continue to work. However, if you use rewrite rules to route messages to the conversion channel, you may have to adjust your mappings to accommodate what you've done.


Note –

An address of the form user@conversion.localhostname or user@conversion will be routed through the conversion channel, regardless of the CONVERSIONS mapping table.


The following example routes all non-internal messages—messages originating from, or destined to, the Internet—through the conversion channel.

CONVERSIONS

   IN-CHAN=tcp_local;OUT-CHAN=*;CONVERT   Yes
   IN-CHAN=*;OUT-CHAN=tcp_local;CONVERT   Yes

The first line specifies that messages coming from the tcp_local channel will be processed. The second line specifies that messages going to the tcp_local channel will also be processed. The tcp_local channel handles all messages going to and coming from the Internet. Since the default is to not go through the conversion channel, any other messages won’t go through the conversion channel.

Note that this is a very basic table, and that it might not be sufficient for a site with a more customized configuration, for example, one using multiple outbound-to-the-Internet tcp_* channels, or using multiple inbound-from-the-Internet tcp_* channels.

13.5.3 To Control Conversion Processing

This section describes how to controls conversion processing. It consists of the following subsections:

When a message is sent to the conversion channel, it is processed body part-by-body part. Processing is controlled by the MTA conversions file, which is specified by the IMTA_CONVERSION_FILE option in the imta_tailor file (default: msg-svr-base/conversions). The conversions file consists of line-separated entries that 1) qualify which types of body parts will be processed, and 2) how they will be processed.

Each entry consists of one or more lines containing one or more name=value parameter clauses. Where the name in the > parameter clauses is one of the parameters in Table 11-5.The values in the parameter clauses conform to MIME conventions. Every line except the last must end with a semicolon (;). A physical line in this file is limited to 252 characters. You can split a logical line into multiple physical lines using the back slash (\) continuation character. Entries are terminated either by a line that does not end in a semicolon, one or more blank lines, or both.

Below is a simple example of a conversion file entry:


Example 13–1 conversions File Entry


out-chan=ims-ms; in-type=application; in-subtype=wordperfect5.1;
  out-type=application; out-subtype=msword; out-mode=block;
  command="/usr/bin/convert -in=wordp -out=msword 'INPUT_FILE' 'OUTPUT_FILE’"

The clauses out-chan=ims-ms; in-type=application; in-subtype=wordperfect5.1 qualify the body part. That is, they specify the type of part to be converted. The header of each part is read and its Content-Type: and other header information is extracted. The entries in the conversion file are then scanned in order from first to last; any in-* parameters present, and the OUT-CHAN parameter, if present, are checked. If all of these parameters match the corresponding information for the body part being processed, then the conversion specified by the command= or delete= clause is performed, and the out-* parameters are set.

If no match occurs, then the part is matched against the next conversions file entry. Once all body parts have been scanned and processed (assuming there is a qualifying match), then the message is sent onwards to the next channel. If there are no matches, no processing occurs, and the message is sent to the next channel.

out-chan=ims-ms specifies that only message parts destined for the ims-ms channel will be converted. in-type=application and in-subtype=wordperfect5.1 specifies that the MIME Content-type header for the message part must be application/wordperfect5.1.

Message parts can be further qualified with additional in-* parameters. (See Table 13–6.) The entry above will trigger conversion actions on a message part which has the following MIME header lines:


Content-type: APPLICATION/wordperfect5.1;name=Draft1.wpc
Content-transfer-encoding: BASE64
Content-disposition: attachment; filename=Draft1.wpc
Content-description: "Project documentation Draft1 wordperfect format"

After the three conversion file qualifying parameters in Example 13–1, the next two parameters, out-type=application and out-subtype=msword, specify replacement MIME header lines to be attached to the “processed” body part. out-type=application and out-subtype=msword specify that the MIME Content-type/subtype of the outgoing message be application/msword.

Note that since the in-type and out-type parameters are the same, out-type=application is not necessary since the conversion channel defaults to the original MIME labels for outgoing body parts. Additional MIME labels for outgoing body parts can be specified with additional output parameters.

out-mode=block (Example 13–1) specifies the file type that the site-supplied program will return. In other words, it specifies how the file will be stored and how the conversion channel should be read back in the returned file. For example, an html file is stored in text mode, while an .exe program or a zip file is stored in block/binary mode. Mode is a way of describing that the file being read is in a certain storage format.

The final parameter in Example 13–1 specifies the action to take on the body part:


command="/usr/bin/convert -in=wordp -out=msword 'INPUT_FILE’ 'OUTPUT_FILE’"

The command= parameter specifies that a program will execute on the body part. /usr/bin/convert is the hypothetical command name; -in=wordp and -out=msword are hypothetical command line arguments specifying the format of the input text and output text; INPUT_FILE and OUTPUT_FILE are conversion channel environmental variables (see 13.5.3.2 To Use Conversion Channel Environmental Variables program should store its converted body part.


Note –

Envelope originator and recipient information is now provided as x-envelope-from and x-envelope-to fields respectively when a file containing the outer message header is requested by a regular conversion entry.


Instead of executing a command on the body part, the message part can simply be deleted by substituting DELETE=1 in place of the command parameter.


Note –

Whenever the conversions file is modified, you must recompile the configuration (see 10.1 Compiling the MTA Configuration).).


13.5.3.1 Conversion Channel Information Flow

The flow of information is as follows: a message containing body parts comes into the conversion channel. The conversion channel parses the message, and processes the parts one by one. The conversion channel then qualifies the body part, that is, it determines if it should be processed or not by comparing its MIME header lines to the qualifying parameters (Table 13–6). If the body part qualifies, the conversion processing commences.

If MIME or body part information is to be passed to the conversion script, it is stored in an environmental variable (13.5.3.2 To Use Conversion Channel Environmental Variables) as specified by information passing parameters (Table 13–6).

At this point, an action specified by an action parameter, (Table 13–6)is taken on the body part. Typically the action is that the body part be deleted or that it be passed to a program wrapped in a script. The script processes the body part and then sends it back to the conversion channel for reassembling into the post-processed message. The script can also send information to the conversion channel by using the conversion channel output options (Table 13–4). This can be information such as new MIME header lines to add to the output body part, error text to be returned to the message sender, or special directives instructing the MTA to initiate some action such as bounce, delete, or hold a message.

Finally, the conversion channel replaces the header lines for the output body part as specified by the output parameters (Table 13–6).

13.5.3.2 To Use Conversion Channel Environmental Variables

When operating on message body parts, it is often useful to pass MIME header line information, or entire body parts, to and from the site-supplied program. For example, a program may require Content-type and Content-disposition header line information as well as a message body part. Typically a site-supplied program’s main input is a message body part which is read from a file. After processing the body part, the program will need to write it to a file from which the conversion channel can read it. This type of information passing is done by using conversion channel environmental variables.

Environmental variables can be created in the conversions file using the parameter-symbol-* parameter or by using a set of pre-defined conversion channel environmental variables (see 13.5.3.3 To Use Conversion Channel Output Options).

The following conversions file entry and incoming header show how to pass MIME information to the site-supplied program using environment variables.

conversions file entry:


in-channel=*; in-type=application; in-subtype=*;
  parameter-symbol-0=NAME; parameter-copy-0=*;
  dparameter-symbol-0=FILENAME; dparameter-copy-0=*;
  message-header-file=2; original-header-file=1;
  override-header-file=1; override-option-file=1;
  command="/bin/viro-scan500.sh ”INPUT_FILE’ ”OUTPUT_FILE’"
 

Incoming header:


Content-type: APPLICATION/msword; name=Draft1.doc
Content-transfer-encoding: BASE64
Content-disposition: attachment; filename=Draft1.doc
Content-description: "Project documentation Draft1 msword format"

in-channel=*; in-type=application; in-subtype=* specify that a message body part from any input channel of type application will be processed.

parameter-symbol-0=NAME specifies that value of the Content-type parameter name, if present (Draft1.doc in our example), be stored in an environment variable called NAME.

parameter-copy-0=* specifies that all Content-type parameters of the input body part be copied to the output body part.

dparameter-symbol-0=FILENAME specifies that the value of the Content-disposition parameter filename (Draft1.doc in our example), be stored in an environment variable called FILENAME.

dparameter-copy-0=* specifies that all Content-disposition parameters of the input body part be copied to the output body part.

message-header-file=2 specifies that the original header of the message as a whole (the outermost message header) be written to the file specified by the environment variable MESSAGE_HEADERS.

original-header-file=1 specifies that the original header of the enclosing MESSAGE/RFC822 part are written to the file specified by the environment variable INPUT_HEADERS.

override-header-file=1 specifies that MIME headers are read from the file specified by environmental variable OUTPUT_HEADERS, overriding the original MIME header lines in the enclosing MIME part. $OUTPUT_HEADERS is an on-the-fly temporary file created at the time conversion runs. A site-supplied program would use this file to store MIME header lines changed during the conversion process. The conversion channel would then read the MIME header lines from this file when it re-assembles the body part. Note that only MIME header lines can be modified. Other general, non-MIME header lines cannot be cannot be altered by the conversion channel.

override-option-file=1 specifies that the conversion channel read conversion channel options from the file named by the OUTPUT_OPTIONS environmental variable. See 13.5.3.3 To Use Conversion Channel Output Options.

command="msg-svr-base/bin/viro-scan500.sh" specifies the command to execute on the message body part.

Table 13–3 Conversion Channel Environment Variables

Environment Variable  

Description  

ATTACHMENT_NUMBER

Attachment number for the current part. This has the same format as the ATTACHMENT-NUMBER conversion match parameter. 

CONVERSION_TAG

The current list of active conversion tags. This corresponds to the TAG conversion match parameter. 

INPUT_CHANNEL

The channel that enqueued the message to the conversion channel. This corresponds to the IN-CHANNEL conversion match parameter. 

INPUT_ENCODING

Encoding originally present on the body part. 

INPUT_FILE

Name of the file containing the original body part. The site-supplied program should read this file. 

INPUT_HEADERS

Name of the file containing the original header lines for the body part. The site-supplied program should read this file. 

INPUT_TYPE

MIME Content-type of the input message part.

INPUT_SUBTYPE

MIME content subtype of the input message part. 

INPUT_DESCRIPTION

MIME content-description of the input message part.

INPUT_DISPOSITION

MIME content-disposition of the input message part.

MESSAGE_HEADERS

Name of the file containing the original outermost header for an enclosing message (not just the body part) or the header for the part’s most immediately enclosing MESSAGE/RFC822 part. The site-supplied program should read this file. 

OUTPUT_CHANNEL

The channel the message is headed for. This corresponds to the OUT-CHANNEL conversion match parameter. 

OUTPUT_FILE

Name of the file where the site-supplied program should store its output. The site-supplied program should create and write this file. 

OUTPUT_HEADERS

Name of the file where the site-supplied program should store MIME header lines for an enclosing part. The site-supplied program should create and write this file. Note that file should contain actual MIME header lines (not option=value lines) followed by a blank line as its final line. Note also that only MIME header lines can be modified. Other general, non-MIME header lines cannot be cannot be altered by the conversion channel.

OUTPUT_OPTIONS

Name of the file from which the site-supplied program should read conversion channel options. See 13.5.3.3 To Use Conversion Channel Output Options.

PART_NUMBER

The part number for the current part. This has the same format as the PART-NUMBER conversion match parameter. 

PART_SIZE

The size in bytes of the part being processed. 

Mail Conversion Tags

Mail conversion tags are special tags which are associated with a particular recipient or sender. When a message is being delivered, the tag is visible to the conversion channel program, which may make use of it for special processing. Conversion tags are stored in the LDAP directory.

Mail conversion tags could be used as follows: the administrator can set up selected users with a mail conversion tag value of harmonica. The administrator then has a conversion channel setup which, when processing that mail, will detect the presence of the tag and the value of harmonica. When that happens, the program will perform some arbitrary function.

Mail conversion tags can be set on a per user or a per domain basis. The recipient LDAP attribute at the domain level is MailDomainConversionTag (modifiable with the MTA option LDAP_DOMAIN_ATTR_CONVERSION_TAG). At the user level it is MailConversionTag (modifiable with the MTA option LDAP_CONVERSION_TAG). Both of these attributes can be multivalued with each value specifying a different tag. The set of tags associated with a given recipient is cumulative, that is, tags set at the domain level are combined with tags set at the user level.

Sender-based conversion tags can be set with the MTA options LDAP_SOURCE_CONVERSION_TAG and LDAP_DOMAIN_ATTR_SOURCE_CONVERSION_TAG, which specify user and domain level LDAP attributes respectively for conversion tags associated with these source address. There is no default attribute for either of these options.

Two new actions are available to system Sieves: addconversiontag and setconversiontag. Both accept a single argument: A string or list of conversion tags. addconversiontag adds the conversion tag(s) to the current list of tags while setconversiontag empties the existing list before adding the new ones. Note that these actions are performed very late in the game so setconversiontag can be used to undo all other conversion tag setting mechanisms. These allow you put conversion tags in the Sieves filters.

The Sieve envelope test accepts conversiontag as an envelope field specifier value. The test checks the current list of tags, one at a time. Note that the :count modifier, if specified, allows checking of the number of active conversion tags. This type of envelope test is restricted to system Sieves. Also note that this test only sees the set of tags that were present prior to Sieve processing—the effects of setconversiontag and addconversiontag actions are not visible.

Including Conversion Tag Information in Various Mapping Probes

A new MTA option, INCLUDE_CONVERSIONTAG, has been added to selectively enable the inclusion of conversion tag information in various mapping probes. This is a bit-encoded value. The bits are assigned are shown in the table below. In all cases the current set of tags appears in the probe as a comma separated list.

Position 

Value 

Mapping 

CHARSET_CONVERSION - added as ;TAG= field before ;CONVERT.

CONVERSION - added as ;TAG= field before ;CONVERT

FORWARD - added just before current address (| delim)

ORIG_SEND_ACCESS - added at end of probe (| delim)

16 

SEND_ACCESS - added at end of probe (| delim)

32 

ORIG_MAIL_ACCESS - added at end of probe (| delim)

64 

MAIL_ACCESS - added at end of probe (| delim)

13.5.3.3 To Use Conversion Channel Output Options

Conversion channel output options (Table 13–4) are dynamic variables used to pass information and special directives from the conversion script to the conversion channel. For example, during body part processing the script may want to send a special directive asking the conversion channel to bounce the message and to add some error text to the returned message stating that the message contained a virus.

The output options are initiated by setting OVERRIDE-OPTION-FILE=1 in the desired conversion entry. Output options are then set by the script as needed and stored in the environmental variable file, OUTPUT_OPTIONS. When the script is finished processing the body part, the conversion channel reads the options from the OUTPUT_OPTIONS file.

The OUTPUT_OPTION variable is the name of the file from which the conversion channel reads options. Typically it is used as an on-the-fly temporary file to pass information. The example below shows a script that uses output options to return an error message to a sender who mailed a virus.


/usr/local/bin/viro_screen2k $INPUT_FILE   # run the virus screener

if [ $? -eq 1 ]; then
   echo "OUTPUT_DIAGNOSTIC=’Virus found and deleted.’" > $OUTPUT_OPTIONS
   echo "STATUS=178029946" >> $OUTPUT_OPTIONS
else
   cp $INPUT_FILE $OUTPUT_FILE # Message part is OK
fi

            

In this example, the system diagnostic message and status code are added to the file defined by $OUTPUT_OPTIONS. If you read the $OUTPUT_OPTIONS temporary file out you would see something like:


OUTPUT_DIAGNOSTIC="Virus found and deleted."
STATUS=178029946

The line OUTPUT_DIAGNOSTIC='Virus found and deleted’ tells the conversion channel to add the text Virus found and deleted to the message.

178029946 is the PMDF__FORCERETURN status per the pmdf_err.h file which is found in the msg-svr-base/include/deprecated/pmdf_err.h. This status code directs the conversion channel to bounce the message back to the sender. (For more information on using special directives refer to 13.5.4 To Bounce, Delete, Hold, Retry Messages Using the Conversion Channel Output

A complete list of the output options is shown below.

Table 13–4 Conversion Channel Output Options

Option  

Description  

OUTPUT_TYPE

MIME content type of the output message part. 

OUTPUT_SUBTYPE

MIME content subtype of the output message part. 

OUTPUT_DESCRIPTION

MIME content description of the output message part. 

OUTPUT_DIAGNOSTIC

Text to include as part of the message sent to the sender if a message is forcibly bounced by the conversion channel. 

OUTPUT_DISPOSITION

MIME content-disposition of the output message part.

OUTPUT_ENCODING

MIME content transfer encoding to use on the output message part.

OUTPUT_MODE

MIME Mode with which the conversion channel should write the output message part, hence the mode with which recipients should read the output message part.

STATUS

Exit status for the converter. This is typically a special directive initiating some action by the conversion channel. A complete list of directives can be viewed in msg-svr-base/include/deprecated/pmdf_err.h

13.5.3.4 Headers in an Enclosing MESSAGE/RFC822 Part

When performing conversions on a message part, the conversion channel has access to the header in an enclosing MESSAGE/RFC822 part, or to the message header if there is no enclosing MESSAGE/RFC822 part. Information in the header may be useful for the site-supplied program.

If an entry is selected that has ORIGINAL-HEADER-FILE=1, then all the original header lines of the enclosing MESSAGE/RFC822 part are written to the file represented by the ORIGINAL_HEADERS environment variable. If OVERRIDE-HEADER-FILE=1, then the conversion channel will read and use as the header on that enclosing part the contents of the file represented by the ORIGINAL_HEADERS environment variable.

13.5.3.5 To Call Out to a Mapping Table from a Conversion Entry

out-parameter-* values may be stored and retrieved in an arbitrarily named mapping table. This feature is useful for renaming attachments sent by clients that send all attachments with a generic name like att.dat regardless of whether they are postscript, msword, text or whatever. This is a generic way to relabel the part so that other clients (Outlook for example) will be able to open the part by reading the extension.

The syntax for retrieving a parameter value from a mapping table is as follows:

”mapping-table-name:mapping-input[$Y, $N]’

$Y returns a parameter value. If there is no match found or the match returns $N, then that parameter in the conversions file entry is ignored or treated as a blank string. Lack of a match or a $N does not cause the conversion entry itself to be aborted.

Consider the following mapping table:


X-ATT-NAMES

   postscript       temp.PS$Y
   wordperfect5.1   temp.WPC$Y
   msword           temp.DOC$Y

The following conversion entry for the above mapping table results in substituting generic file names in place of specific file names on attachments:

out-chan=tcp_local; in-type=application; in-subtype=*; 
   in-parameter-name-0=name; in-parameter-value-0=*; 
   out-type=application; out-subtype=’INPUT-SUBTYPE’; 
   out-parameter-name-0=name; 
   out-parameter-value-0=”’X-ATT-NAMES:\\’INPUT_SUBTYPE\\''"; 
   command="cp  ”INPUT_FILE’  ”OUTPUT_FILE’"

In the example above, out-chan=tcp_local; in-type=application; in-subtype=* specifies that a message to be processed must come from the tcp_local channel with the content-type header of application/* (* specifies that any subtype would do).

in-parameter-name-0=name; in-parameter-value-0=* additionally specifies that the message must have a content-type parameter called name=* and that any value for that parameter will be accepted (again, * specifies that any parameter value would do.)

out-type=application; specifies that the MIME Content-type parameter for the post-processing message be application.

out-subtype=’INPUT-SUBTYPE’; specifies that the MIME subtype parameter for the post-processing body part be the INPUT-SUBTYPE environmental variable, which is the original value of the input subtype. Thus, if you wanted change

Content-type: application/xxxx; name=foo.doc

to

Content-type: application/msword; name=foo.doc

then you would use

out-type=application; out-subtype=msword

out-parameter-name-0=name; specifies that the output body part will have a MIME Content-type name= parameter.

out-parameter-value-0=’X-ATT-NAMES:\\’INPUT_SUBTYPE\\’’; says to take the value of the INPUT_SUBTYPE variable (that is, the original content-type header subtype value of the original body part) and search the mapping table X-ATT-NAMES. If a match is found, the content-type parameter specified by out-parameter-name-0 (that is, name) receives the new value specified in the X-ATT-NAMES mapping table. Thus, if the original subtype was msword, the value of the name parameter will be temp.DOC.

13.5.4 To Bounce, Delete, Hold, Retry Messages Using the Conversion Channel Output

This section describes how to use the conversion channel options to bounce, delete, or hold messages. The basic procedure is as follows:

  1. Set OVERRIDE-OPTION-FILE=1 in the appropriate conversions file entry. This tells the conversion channel to read the output options from the OUTPUT_OPTIONS file.

  2. Use the conversion script to determine what action is required on a particular message body part.

  3. In the script, specify the special directive for that action by writing the STATUS=directive_code option in the OUTPUT_OPTIONS file.

A complete listing of special directives can be found in msg-svr-base/include/deprecated/pmdf_err.h. The ones commonly used by the conversion channel are:

Table 13–5 Special Directives Commonly Used By the Conversion Channel

NAME  

Hex Value  

Decimal Value  

PMDF__FORCEHOLD

0x0A9C86AA

178030250

PMDF__FORCERETURN

0x0A9C857A

178029946

PMDF__FORCEDELETE

0x0A9C8662

178030178

PMDF__FORCEDISCARD

0x0A9C86B3

178030259

PMDF__AGN

0x0A9C809A

178028698

We will explain the functions of these directives using examples.

13.5.4.1 To Bounce Messages

To bounce a message using the conversion channel set OVERRIDE-OPTION-FILE=1 in the appropriate conversions file entry and add the following line to your conversion script:

echo "STATUS=178029946" >> $OUTPUT_OPTIONS

If you wish to add a short text string to the bounced message add the following line to the conversion script:

echo OUTPUT_DIAGNOSTIC=text-string >> $OUTPUT_OPTIONS

where text string is something like: “The message sent from your machine contained a virus which has been removed. Be careful about executing email attachments.

13.5.4.2 To Conditionally Delete a Message or Its Parts

It may be useful to delete parts conditionally, depending on what they contain. This can be done using the output options. By contrast, the DELETE=1 conversion parameter clause unconditionally deletes a message part.

To delete a message part using the output options, set OVERRIDE-OPTION-FILE=1 in the appropriate conversions file entry and add the following line to your conversion script:

echo "STATUS=178030178" >> $OUTPUT_OPTIONS

Similarly, to delete the entire message you could use:

echo "STATUS=178030259" >> $OUTPUT_OPTIONS

13.5.4.3 To Hold a Message

It may be useful to hold messages conditionally, depending on what they contain. To delete a message part using the output options, set OVERRIDE-OPTION-FILE=1 in the appropriate conversions file entry and add the following line to your conversion script:

echo "STATUS=178030250" >> $OUTPUT_OPTIONS

This requests that the conversion channel hold the message as a .HELD file in the conversion channel queue.

13.5.4.4 To Cause Messages to Be Reprocessed

When a converter script encounters a temporary resource problem (for example, the system can't connect to an external server, a needed file is locked, and so on), you can use PMDF_AGN to tell the conversion channel to consider processing messages that have encountered a temporary error. The MTA will record a "Q" status message in the mail.log_current, retain the message in the conversion channel, and retry the processing later.

Add the following line to your conversion script:

echo "STATUS=178028698" >> $OUTPUT_OPTIONS

13.5.5 Conversion Channel Example

The CONVERSIONS mapping and set of conversion rules seen in examples below cause GIF, JPEG, and BITMAP files sent to the hypothetical channel tcp_docuprint to be converted into PostScript automatically. Several of these conversions use the hypothetical /usr/bin/ps-converter.sh to make that transformation. An additional rule that converts WordPerfect 5.1 files into Microsoft Word files is included.


CONVERSIONS  

    IN-CHAN=*;OUT-CHAN=tcp_docuprint;CONVERT    Yes

out-chan=ims-ms; in-type=application; in-subtype=wordperfect5.1; 
  out-type=application; out-subtype=msword; out-mode=block; 
  command="/bin/doc-convert -in=wp -out=msw   ’INPUT_FILE’  ’OUTPUT_FILE’"

out-chan=tcp_docuprint; in-type=image; in-subtype=gif; 
  out-type=application; out-subtype=postscript; out-mode=text; 
  command="/bin/ps-convert -in=gif -out=ps   ’INPUT_FILE’  ’OUTPUT_FILE’"

out-chan=tcp_docuprint; in-type=image; in-subtype=jpeg; 
  out-type=application; out-subtype=postscript; out-mode=text; 
  command="/bin/ps-convert -in=jpeg -out=ps  ’INPUT_FILE’  ’OUTPUT_FILE’"

out-chan=tcp_docuprint; in-type=image; in-subtype=bitmap; 
  out-type=application; out-subtype=postscript; out-mode=text; 
  command="/bin/ps-convert -in=bmp -out=ps   ’INPUT_FILE’  ’OUTPUT_FILE’"

The conversion parameters are shown below:

Table 13–6 Conversion Parameters

Parameter  

Description  

Part 1: Qualifying Parameters (Specifies the parameters for which the message must match before it will be converted.)

OUT-CHAN,OUT-CHANNEL

Output channel to match for conversion (wildcards allowed). The conversion specified by this entry is performed only if the message is destined for this specified channel. 

IN-CHAN,IN-CHANNEL

Input channel to match for conversion (wildcards allowed). The conversion specified by this entry is only performed if the message is coming from the specified channel. 

IN-TYPE

Input MIME type to match for conversion (wildcards allowed). The conversion specified is performed only if this field matches the MIME type of the body part. 

IN-SUBTYPE

Input MIME subtype to match for conversion (wildcards allowed). The conversion specified by this entry is performed only if this field matches the MIME subtype of the body part. 

IN-PARAMETER-NAME-n

Specifies the name of the Input MIME Content-Type parameter that must match for conversion; The n = 0, 1, 2.... is used to optionally pair the specified parameter name requirement with a value required by using IN-PARAMETER-VALUE-n with the same value of n.

IN-PARAMETER-VALUE-n

Specifies the value required of the input MIME Content-Type parameter whose name is specified in the corresponding IN-PARAMETER-NAME-n. The conversion specified by this entry is performed only if the input body part has the content-type parameter specified by the corresponding IN-PARAMETER-NAME-n and its value matches the value of this parameter. Wildcards allowed.

IN-PARAMETER-DEFAULT-n

Default value to use if the input MIME Content-Type parameter specified by the corresponding IN-PARAMETER-NAME-n is not present.

IN-DISPOSITION

Input MIME Content-Disposition to match for conversion.

IN-DPARAMETER-NAME-n

Specifies the name of the Input MIME Content-Disposition parameter that must match for conversion; The n = 0, 1, 2.... is used to optionally pair the specified parameter name requirement with a value required by using IN-DPARAMETER-VALUE-n with the same value of n.

IN-DPARAMETER-VALUE-n

Specifies the value required of the input MIME Content-Disposition parameter whose name is specified in the corresponding IN-DPARAMETER-NAME-n. The conversion specified by this entry is performed only if the input body part has the Content-Disposition parameter specified by the corresponding IN-DPARAMETER-NAME-n and its value matches the value of this parameter. Wildcards allowed.

IN-DPARAMETER-DEFAULT-n

Default value to use if the input MIME Content-Disposition parameter specified by the corresponding IN-DPARAMETER-NAME-n is not present.

IN-DESCRIPTION

Input MIME Content-Description to match for conversion.

IN-SUBJECT

Input Subject from enclosing MESSAGE/RFC822 part.

TAG

Input tag, as set by a mail list CONVERSION_TAG parameter.

Part 2: Output Parameters (Specify the body part’s post-conversion output settings.)

OUT-TYPE

Output MIME type if it is different than the input type. 

OUT-SUBTYPE

Output MIME subtype if it is different than the input subtype. 

OUT-PARAMETER-NAME-n

Specifies the name of a content-type parameter which will be set on the output body part.

OUT-PARAMETER-VALUE-n

Output MIME Content-Type parameter value corresponding to OUT-PARAMETER-NAME-n.

PARAMETER-COPY-n

Specifies the name of a content-type parameter which should be copied from the input body part to the output body part.

OUT-DISPOSITION

Output MIME Content-Disposition if it is different than the input MIME Content-Disposition.

OUT-DPARAMETER-NAME-n

Output MIME Content-Disposition parameter name; n=0, 1, 2...

OUT-DPARAMETER-VALUE-n

Output MIME Content-Disposition parameter value corresponding to OUT-DPARAMETER-NAME-n.

DPARAMETER-COPY-n

A list of the Content-Disposition: parameters to copy from the input body part’s Content-Disposition: parameter list to the output body part’s Content-Disposition: parameter list; n = 0, 1, 2,... Takes as argument the name of the MIME parameter to copy, as matched by an IN-PARAMETER-NAME-n clause. Wildcards may be used in the argument. In particular, an argument of * means to copy all the original Content-Disposition: parameters.

OUT-DESCRIPTION

Output MIME Content-Description if it is different than the input MIME Content-Description.

OUT-MODE

Mode in which to read and store the converted file. This should be BLOCK (binaries and executables) or TEXT.

OUT-ENCODING

Encoding to apply to the converted file when the message is reassembled. 

Part 3: Action Parameters (Specify an action to take on a message part.)

COMMAND

Command to execute to perform conversion. Command to execute to perform conversion. This parameter is required; if no command is specified, the entry is ignored. Use / to specify paths, not \. Example: command="D:/tmp/mybat.bat"

DELETE

0 or 1. If this flag is set, the message part is deleted. (If this is the only part in a message, then a single empty text part is substituted.) 

RELABEL

RELABEL=1 will relabel the MIME label to whatever is specified by the Output parameters. Relabel=0 does nothing. Usually relabelling is done on mislabeled parts (example: from Content-type: application/octet-stream to Content-type: application/msword) so users can “doubleclick” to open a part, rather than having to save the part to a file and open it with a program.

SERVICE-COMMAND

SERVICE-COMMAND=command will execute a site-supplied procedure that will operate on entire MIME message (MIME headers and content body part). Also, unlike other CHARSET-CONVERSION operations or conversion channel operations, the service-command are expected to do their own MIME disassembly, decoding, re-encoding, and reassembly. Note that this flag causes an entry to be ignored during conversion channel processing; SERVICE-COMMAND entries are instead performed during character set conversion processing. Use / to specify paths, not \. Example: command="D:/tmp/mybat.bat"

Part 4: Information Passing Parameters (Used to pass information to and from the site-supplied program.)

DPARAMETER-SYMBOL-n

Environment variable into which the Content-disposition parameter value, if present, will be stored; n = 0, 1, 2,... Each DPARAMETER-SYMBOL-n is extracted from the Content-Disposition: parameter list in order (n=0 is first parameter, n=2 second, etc.) and placed in the specified environment variable prior to executing the site-supplied program.

PARAMETER-SYMBOL-n

Specifies the name of a content-type parameter which, if present in the input body part, its value will be stored in an environment variable of the same name. If the parameter does not exist in the input body part, the environment variable will not exist in the process. For example, if you specify parameter-symbol-0=foo, and there's a content type parameter foo with value bar, you end up with an environment variable foo with value bar.

Environment variable into which the Content-Type parameter value, if present, will be stored; n = 0, 1, 2... Each PARAMETER-SYMBOL-n is extracted from the Content-Type: parameter list in order (n=0 is first parameter, n=2 second, etc.) and placed in an environment variable of the same name prior to executing the site-supplied program. Takes as argument the variable name into which the MIME parameter to convert, as matched by an IN-PARAMETER-NAME-n clause.

MESSAGE-HEADER-FILE

Writes all, part, or none of the original header of a message to the file specified by the environmental variable MESSAGE_HEADERS. If set to 1, the original header of the immediately enclosing body part are written to the file specified by the environmental variable MESSAGE_HEADERS. If set to 2, the original header of the message as a whole (the outermost message header) are written to the file.

ORIGINAL-HEADER-FILE

0 or 1. If set to 1, the original header of the enclosing MESSAGE/RFC822 part (not just the body part) are written to the file represented by the environmental variable ORIGINAL_HEADERS.

OVERRIDE-HEADER-FILE

0 or 1. If set to 1, then MIME header lines are read by the conversion channel from the environmental variable OUTPUT_HEADERS, overriding the original header lines in the enclosing MIME part.

OVERRIDE-OPTION-FILE

If OVERRIDE-OPTION-FILE=1, the conversion channel reads options from the OUTPUT_OPTIONS environmental variable.

PART-NUMBER

Dotted integers: a. b. c... The part number of the MIME body part.

13.5.6 Automatic Arabic Character Set Detection

A new auto_ef program was added to automatically detect Arabic character sets.

You can call the auto_ef program from the conversion channel to automatically detect and label most unlabeled or incorrectly labeled text messages in Arabic character sets. These unlabeled or mislabeled messages are usually sent from Yahoo or Hotmail in Arabic.

Without the correct character set labeling, many mail clients cannot display the messages correctly.

If a message has MIME content-type headers, the auto_ef program examines and processes only those with text/plain content type. If the message is not labeled with a MIME content-type header, then auto_ef adds a text/plain content-type unconditionally.

To activate or enable this program, you must:

ProcedureTo Automatically Detect Arabic Character Sets

  1. Edit your mappings file in the msg-svr-base/config directory to enable a conversion channel for the source and destination channel of your choosing. To enable a conversion channel for all mail coming in from the Internet to your local users, add a section to your mappings file similar to the following:

    CONVERSIONS
    
       IN-CHAN=tcp*;OUT-CHAN=ims-ms;CONVERT YES

    Note that the IN and OUT channels depend on your configuration. If you are deploying on a relay MTA, you must modify the channels to fit your configuration. For example,

    IN-CHAN=tcp*;OUT-CHAN=tcp*;CONVERT YES

    Or, you could turn it on for all channels as follows:

    IN-CHAN=*;OUT-CHAN=*;CONVERT YES

  2. Create a conversions file in the msg-svr-base/config directory that is owned and readable by the current version of Messaging Server user, and that contains the following:

    !
    in-channel=*; out-channel=*;
      in-type=text; in-subtype=*;
      parameter-copy-0=*; dparameter-copy-0=*;
      original-header-file=1; override-header-file=1;
      command="msg-svr-base
    /lib/arabicdetect.sh"
    !
  3. Compile your MTA configuration with the following command:

    msg-svr-base/sbin/imsimta cnbuild

  4. Restart with the command:

    msg-svr-base/sbin/imsimta restart

13.6 Character Set Conversion and Message Reformatting

This section describes character set, formatting, and labelling conversions performed internally by the MTA. Note that some of the examples in this section use old or obsolete technology like DEC VMS, or the d channels. Although these technologies are old or obsolete, this does not make the examples DEC- or d channel-specific. The examples are still valid in describing how the conversion technology works. We will update the examples in a later release.

One very basic mapping table in Messaging Server is the character set conversion table. The name of this table is CHARSET-CONVERSION. It is used to specify what sorts of channel-to-channel character set conversions and message reformatting should be done.

On many systems there is no need to do character set conversions or message reformatting and therefore this table is not needed. Situations arise, however, where character conversions must be done. For example, sites running Japanese OpenVMS may need to convert between DEC Kanji and the ISO-2022 Kanji currently used on the Internet. Another possible use of conversions arises when multinational characters are so heavily used that the slight discrepancies between the DEC Multinational Character Set (DEC-MCS) and the ISO-8859-1 character set specified for use in MIME may become an issue, and actual conversion between the two may therefore be needed.

The CHARSET-CONVERSION mapping table can also be used to alter the format of messages. Facilities are provided to convert a number of non-MIME formats into MIME. Changes to MIME encodings and structure are also possible. These options are used when messages are being relayed to systems that only support MIME or some subset of MIME. And finally, conversion from MIME into non-MIME formats is provided in a small number of cases.

The MTA will probe the CHARSET-CONVERSION mapping table in two different ways. The first probe is used to determine whether or not the MTA should reformat the message and if so, what formatting options should be used. (If no reformatting is specified, the MTA does not bother to check for specific character set conversions.) The input string for this first probe has the general form:

IN-CHAN=in-channel;OUT-CHAN=out-channel;CONVERT

Here in-channel is the name of the source channel (where the message comes from) and out-channel is the name of the destination channel (where the message is going). If a match occurs the resulting string should be a comma-separated list of keywords. Table 13–7 lists the keywords.

Table 13–7 CHARSET-CONVERSION Mapping Table Keywords

Keyword 

Description 

Always

Force conversion even the message is going to be passed through the conversion channel before going to out-channel.

Appledouble

Convert other MacMIME formats to Appledouble format. 

Applesingle

Convert other MacMIME formats to Applesingle format. 

BASE64

Switch MIME encodings to BASE64. This keyword only applies to message parts that are already encoded. Messages with Content-transfer-encoding: 7BIT or 8bit do not require any special encoding and therefore this BASE64 option will have no effect on them. 

Binhex

Convert other MacMIME formats, or parts including Macintosh type and Mac creator information, to Binhex format. 

Block

Extract just the data fork from MacMIME format parts. 

Bottom

“Flatten” any message/rfc822 body part (forwarded message) into a message content part and a header part. 

Delete

“Flatten” any message/rfc822 body part (forwarded message) into a message content part, deleting the forwarded headers. 

Level

Remove redundant multipart levels from message. 

Macbinary

Convert other MacMIME formats, or parts including Macintosh type and Macintosh creator information, to Macbinary format. 

No

Disable conversion. 

QUOTED-PRINTABLE

Switch MIME encodings to QUOTED-PRINTABLE. 

Record,Text

Line wrap text/plain parts at 80 characters. 

Record,Text= n

Line wrap text/plain parts at n characters. 

RFC1154

Convert message to RFC 1154 format. 

Top

“Flatten” any message/rfc822 body part (forwarded message) into a header part and a message content part. 

UUENCODE

Switch MIME encodings to X-UUENCODE. 

Yes

Enable conversion. 

13.6.1 Character Set Conversion

If the MTA probes and finds that the message is to be reformatted, it will proceed to check each part of the message. Any text parts are found and their character set parameters are used to generate the second probe. Only when the MTA has checked and found that conversions may be needed does it ever perform the second probe. The input string in this second case looks like this:

IN-CHAN=in-channel;OUT-CHAN=out-channel;IN-CHARSET=in-char-set

The in-channel and out-channel are the same as before, and the in-char-set is the name of the character set associated with the particular part in question. If no match occurs for this second probe, no character set conversion is performed (although message reformatting, for example, changes to MIME structure, may be performed in accordance with the keyword matched on the first probe). If a match does occur it should produce a string of the form:

OUT-CHARSET=out-char-set

Here out-char-set specifies the name of the character set to which the in-char-set should be converted. Note that both of these character sets must be defined in the character set definition table, charsets.txt, located in the MTA table directory. No conversion will be done if the character sets are not properly defined in this file. This is not usually a problem since this file defines several hundred character sets; most of the character sets in use today are defined in this file. See the description of the imsimta chbuild (UNIX and NT) utility for further information on the charsets.txt file.

If all the conditions are met, the MTA will proceed to build the character set mapping and do the conversion. The converted message part will be relabelled with the name of the character set to which it was converted.

The charset-conversion mapping has been extended to provide several additional capabilities:


Example 13–2 Converting ISO-8859-1 to UTF-8 and back

Suppose that ISO-8859-1 is used locally, but this needs to be converted to UTF-8 for use on the Internet. In particular, suppose the connection to the Internet is via the tcp_local and tcp_internal and ims-ms are where internal messages originate and are delivered. The CHARSET-CONVERSION table shown below brings such conversions about. Note that each IN-CHAN entries must be on a single line. The backslash (\) is used to signify this.


CHARSET-CONVERSION

 IN-CHAN=tcp_internal;OUT-CHAN=tcp_local;CONVERT               Yes
 IN-CHAN=tcp_local;OUT-CHAN=tcp_internal;CONVERT               Yes
 IN-CHAN=tcp_local;OUT-CHAN=ims-ms;CONVERT                     Yes
 IN-CHAN=*;OUT-CHAN=*;CONVERT                                  No
 IN-CHAN=tcp_internal;OUT-CHAN=tcp_local;IN-CHARSET=ISO-8859-1 OUT-CHARSET=UTF-8
 IN-CHAN=tcp_local;OUT-CHAN=tcp_internal;IN-CHARSET=UTF-8 OUT-CHARSET=ISO-8859-1
 IN-CHAN=tcp_local;OUT-CHAN=ims-ms;IN-CHARSET=UTF-8       OUT-CHARSET=ISO-8859-1


Example 13–3 Converting EUC-JP to ISO-2022-JP and Back

The CHARSET-CONVERSION table shown below specifies a conversion between local usage of EUC-JP and the ISO 2022 based JP code.


CHARSET-CONVERSION

  IN-CHAN=ims-ms;OUT-CHAN=ims-ms;CONVERT                  No
  IN-CHAN=tcp_internal;OUT-CHAN=ims-ms;CONVERT            No
  IN-CHAN=tcp_internal;OUT-CHAN=tcp_internal;CONVERT      No
  IN-CHAN=tcp_internal;OUT-CHAN=*;CONVERT                 Yes
  IN-CHAN=*;OUT-CHAN=ims-ms;CONVERT                       Yes
  IN-CHAN=*;OUT-CHAN=tcp_internal;CONVERT                 Yes
  IN-CHAN=tcp_internal;OUT-CHAN=*;IN-CHARSET=EUC-JP      OUT-CHARSET=ISO-2022-JP
  IN-CHAN=*;OUT-CHAN=ims-ms;IN-CHARSET=ISO-2022-JP        OUT-CHARSET=EUC-JP
  IN-CHAN=*;OUT-CHAN=tcp_internal;IN-CHARSET=ISO-2022-JP  OUT-CHARSET=EUC-JP

13.6.2 Message Reformatting

As described above, the CHARSET-CONVERSION mapping table is also used to effect the conversion of attachments between MIME and several proprietary mail formats.

The following sections give examples of some of the other sorts of message reformatting which can be affected with the CHARSET-CONVERSION mapping table.

13.6.2.1 Non-MIME Binary Attachment Conversion

Mail in certain non-standard (non-MIME) formats; for example, mail in certain proprietary formats or mail from the Microsoft Mail (MSMAIL) SMTP gateway is automatically converted into MIME format if CHARSET-CONVERSION is enabled for any of the channels involved in handling the message. If you have a tcp_local channel then it is normally the incoming channel for messages from a Microsoft Mail SMTP gateway, and the following will enable the conversion of messages delivered to your local users:

CHARSET-CONVERSION  

  IN-CHAN=tcp_local;OUT-CHAN=ims-ms;CONVERT         Yes

You may also wish to add entries for channels to other local mail systems. For instance, an entry for the tcp_internal channel:

CHARSET-CONVERSION

  IN-CHAN=tcp_local;OUT-CHAN=l;CONVERT              Yes
  IN-CHAN=tcp_local;OUT-CHAN=tcp_internal;CONVERT   Yes

Alternatively, to cover every channel you can simply specify OUT-CHAN=* instead of OUT-CHAN=ims-ms. However, this may bring about an increase in message processing overhead as all messages coming in the tcp_local channel will now be scrutinized instead of just those bound to specific channels.

More importantly, such indiscriminate conversions might place your system in the dubious and frowned upon position of converting messages—not necessarily your own site’s—which are merely passing through your system, a situation in which you should merely be acting as a transport and not necessarily altering anything beyond the message envelope and related transport information.

To convert MIME into the format Microsoft Mail SMTP gateway understands, use a separate channel in your MTA configuration for the Microsoft Mail SMTP gateway; for example, tcp_msmail, and put the following in the mappings. file:

CHARSET-CONVERSION  

  IN-CHAN=*;OUT-CHAN=tcp_msmail;CONVERT        RFC1154

13.6.2.2 Relabelling MIME Headers

Some user agents or gateways may emit messages with MIME headers that are less informative than they might be, but that nevertheless contain enough information to construct more precise MIME headers. Although the best solution is to properly configure such user agents or gateways, if they are not under your control, you can instead ask the MTA to try to reconstruct more useful MIME headers.

If the first probe of the CHARSET-CONVERSION mapping table yields a Yes or Always keyword, then the MTA will check for the presence of a conversions file. If a conversions file exists, then the MTA will look in it for an entry with RELABEL=1 and if it finds such an entry, the MTA will then perform any MIME relabelling specified in the entry. See 13.5.3 To Control Conversion Processing for information on conversions file entries.

For example, the combination of a CHARSET-CONVERSION table such as:


CHARSET-CONVERSION  

  IN-CHAN=tcp_local;OUT-CHAN=tcp_internal;CONVERT            Yes

and MTA conversion file entries of


out-chan=ims-ms; in-type=application; in-subtype=octet-stream; 
  in-parameter-name-0=name; in-parameter-value-0=*.ps; 
  out-type=application; out-subtype=postscript;   
  parameter-copy-0=*; relabel=1 

out-chan=ims-ms; in-type=application; in-subtype=octet-stream; 
  in-parameter-name-0=name; in-parameter-value-0=*.msw; 
  out-type=application; out-subtype=msword; 
     parameter-copy-0=* relabel=1

will result in messages that arrive on the tcp_local channel and are routed to the ims-ms channel, and that arrive originally with MIME labelling of application/octet-stream but have a filename parameter with the extension ps or msw, being relabelled as application/postscript or application/msword, respectively. (Note that this more precise labelling is what the original user agent or gateway should have performed itself.) Such a relabelling can be particularly useful in conjunction with a MIME-CONTENT-TYPES-TO-MR mapping table, used to convert such resulting MIME types back into appropriate MRTYPE tags, which needs precise MIME labelling in order to function optimally; if all content types were left labelled only as application/octet-stream, the MIME-CONTENT-TYPES-TO-MR mapping table could only, at best, unconditionally convert all such to one sort of MRTYPE.

With the above example and MIME-CONTENT-TYPES-TO-MR mapping table entries including

APPLICATION/POSTSCRIPT        PS 
APPLICATION/MSWORD              MW

a labelling coming in as, for example,

Content-type: application/octet-stream; name=stuff.ps

would be relabelled as

Content-type: application/postscript

and then converted into an MRTYPE tag PS to let Message Router know to expect PostScript.

Sometimes it is useful to do relabelling in the opposite sort of direction, “downgrading” specific MIME attachment labelling to application/octet-stream, the label for generic binary data. In particular, “downgrading” specific MIME labelling is often used in conjunction with the convert_octet_stream channel keyword on the mime_to_x400 channel (PMDF-X400) or xapi_local channel (PMDF-MB400) to force all binary MIME attachments to be converted to X.400 bodypart 14 format.

For instance, the combination of a CHARSET-CONVERSION mapping table such as

CHARSET-CONVERSION

    IN-CHAN=*;OUT-CHAN=mime_to_x400*;CONVERT Yes

and PMDF conversions file entries of

out-chan=mime_to_x400*; in-type=application; in-subtype=*;
   out-type=application; out-subtype=octet-stream; relabel=1
 
out-chan=mime_to_x400*; in-type=audio; in-subtype=*; 
   out-type=application; out-subtype=octet-stream; relabel=1 

out-chan=mime_to_x400*; in-type=image; in-subtype=*; 
   out-type=application; out-subtype=octet-stream; relabel=1 

out-chan=mime_to_x400*; in-type=video; in-subtype=*; 
   out-type=application; out-subtype=octet-stream; relabel=1

will result in downgrading various specific MIME attachment labelling to the generic application/octet-stream labelling (so that convert_octet_stream will apply) for all messages going to mime_to_x400* channels.

13.6.2.3 MacMIME Format Conversions

Macintosh files have two parts, a resource fork that contains Macintosh specific information, and a data fork that contains data usable on other platforms. This introduces an additional complexity when transporting Macintosh files, as there are four different formats in common use for transporting the Macintosh file parts. Three of the formats, Applesingle, Binhex, and Macbinary, consist of the Macintosh resource fork and Macintosh data fork encoded together in one piece. The fourth format, Appledouble, is a multipart format with the resource fork and data fork in separate parts. Appledouble is hence the format most likely to be useful on non-Macintosh platforms, as in this case the resource fork part may be ignored and the data fork part is available for use by non-Macintosh applications. But the other formats may be useful when sending specifically to Macintoshes.

The MTA can convert between these various Macintosh formats. The CHARSET-CONVERSION keywords Appledouble, Applesingle, Binhex, or Macbinary tell the MTA to convert other MacMIME structured parts to a MIME structure of multipart/appledouble, application/applefile, application/mac-binhex40, or application/macbinary, respectively. Further, the Binhex or Macbinary keywords also request conversion to the specified format of non-MacMIME format parts that do nevertheless contain X-MAC-TYPE and X-MAC-CREATOR parameters on the MIME Content-type: header. The CHARSET-CONVERSION keyword Block tells the MTA to extract just the data fork from MacMIME format parts, discarding the resource fork; (since this loses information, use of Appledouble instead is generally preferable).

For example, the following CHARSET-CONVERSION table would tell the MTA to convert to Appledouble format when delivering to the VMS MAIL mailbox or a GroupWise postoffice, and to convert to Macbinary format when delivering to the Message Router channel:

CHARSET-CONVERSION
   IN-CHAN=*;OUT-CHAN=l;CONVERT              Appledouble 
   IN-CHAN=*;OUT-CHAN=wpo_local;CONVERT      Appledouble 
   IN-CHAN=*;OUT-CHAN=tcp_internal;CONVERT   Macbinary

The conversion to Appledouble format would only be applied to parts already in one of the MacMIME formats. The conversion to Macbinary format would only be applied to parts already in one of the MacMIME formats, or non-MacMIME parts which included X-MAC-TYPE and X-MAC-CREATOR parameters on the MIME Content-type: header.

When doing conversion to Appledouble or Block format, the MAC-TO-MIME-CONTENT-TYPES mapping table may be used to indicate what specific MIME label to put on the data fork of the Appledouble part, or the Block part, depending on what the Macintosh creator and Macintosh type information in the original Macintosh file were. Probes for this table have the form format|type|creator|filename where format is one of SINGLE, BINHEX or MACBINARY, where type and creator are the Macintosh type and Macintosh creator information in hex, respectively, and where filename is the filename.

For example, to convert to Appledouble when sending to the ims-ms channel and when doing so to use specific MIME labels for any MS Word or PostScript documents converted from MACBINARY or BINHEX parts, appropriate tables might be:


CHARSET-CONVERSION 

  IN-CHAN=*;OUT-CHAN=ims-ms;CONVERT     Appledouble


MAC-TO-MIME-CONTENT-TYPES 

! PostScript 
    MACBINARY|45505346|76677264|*     APPLICATION/POSTSCRIPT$Y 
    BINHEX|45505346|76677264|*        APPLICATION/POSTSCRIPT$Y 
! Microsoft Word 
    MACBINARY|5744424E|4D535744|*     APPLICATION/MSWORD$Y 
    BINHEX|5744424E|4D535744|*        APPLICATION/MSWORD$Y

Note that the template (right hand side) of the mapping entry must have the $Y flag set in order for the specified labelling to be performed. Sample entries for additional types of attachments may be found in the file mac_mappings.sample in the MTA table directory.

If you wish to convert non-MacMIME format parts to Binhex or Macbinary format, such parts need to have X-MAC-TYPE and X-MAC-CREATOR MIME Content-type: parameter values provided. Note that MIME relabelling can be used to force such parameters onto parts that would not otherwise have them.

13.6.3 Service Conversions

The MTA’s conversion service facility may be used to process with site-supplied procedures a message so as to produce a new form of the message. Unlike either the sorts of CHARSET-CONVERSION operations discussed above or the conversion channel, which operate on the content of individual MIME message parts, conversion services operate on entire MIME message parts (MIME headers and content) as well as entire MIME messages. Also, unlike other CHARSET-CONVERSION operations or conversion channel operations, conversion services are expected to do their own MIME disassembly, decoding, re-encoding, and reassembly.

Like other CHARSET-CONVERSION operations, conversion services are enabled through the CHARSET-CONVERSION mapping table. If the first probe of the CHARSET-CONVESION mapping table yields a Yes or Always keyword, then the MTA will check for the presence of an MTA conversions file. If a conversions file exists, then the MTA will look in it for an entry specifying a SERVICE-COMMAND, and if it finds such an entry, execute it. The conversions file entries should have the form:


in-chan=channel-pattern; 
  in-type=type-pattern; in-subtype=subtype-pattern; 
  service-command=command

Of key interest is the command string. This is the command that should be executed to perform a service conversion (for example, invoke a document converter). The command must process an input file containing the message text to be serviced and produce as output a file containing the new message text. On UNIX, the command must exit with a 0 if successful and a non-zero value otherwise.

For instance, the combination of a CHARSET-CONVERSION table such as

CHARSET-CONVERSION

IN-CHAN=bsout_*;OUT-CHAN=*;CONVERT Yes

and an MTA conversions file entry on UNIX of


in-chan=bsout_*; in-type=*; in-subtype=*; 
service-command="/pmdf/bin/compress.sh compress $INPUT_FILE $OUTPUT_FILE"

will result in all messages coming from a BSOUT channel being compressed.

Environment variables are used to pass the names of the input and output files as well as the name of a file containing the list of the message's envelope recipient addresses. The names of these environment variables are:

The values of these three environment variables may be substituted into the command line by using standard command line substitution: that is, preceding the variable's name with a dollar character on UNIX. For example, when INPUT_FILE and OUTPUT_FILE have the values a.in and a.out, then the following declaration on UNIX:


in-chan=bsout_*; in-type=*; in-subtype=*; 
 service-command="/pmdf/bin/convert.sh $INPUT_FILE $OUTPUT_FILE"

executes the command

/pmdf/bin/convert.sh a.in a.out