13.5.6 Automatic Arabic Character Set Detection

A new auto_ef program was added to automatically detect Arabic character sets.

You can call the auto_ef program from the conversion channel to automatically detect and label most unlabeled or incorrectly labeled text messages in Arabic character sets. These unlabeled or mislabeled messages are usually sent from Yahoo or Hotmail in Arabic.

Without the correct character set labeling, many mail clients cannot display the messages correctly.

If a message has MIME content-type headers, the auto_ef program examines and processes only those with text/plain content type. If the message is not labeled with a MIME content-type header, then auto_ef adds a text/plain content-type unconditionally.

To activate or enable this program, you must:

To Automatically Detect Arabic Character Sets

1. Edit your mappings file in the msg-svr-base/config directory to enable a conversion channel for the source and destination channel of your choosing. To enable a conversion channel for all mail coming in from the Internet to your local users, add a section to your mappings file similar to the following:

   CONVERSIONS
   
      IN-CHAN=tcp*;OUT-CHAN=ims-ms;CONVERT YES

   Note that the IN and OUT channels depend on your configuration. If you are deploying on a relay MTA, you must modify the channels to fit your configuration. For example,

   IN-CHAN=tcp*;OUT-CHAN=tcp*;CONVERT YES

   Or, you could turn it on for all channels as follows:

   IN-CHAN=*;OUT-CHAN=*;CONVERT YES

2. Create a conversions file in the msg-svr-base/config directory that is owned and readable by the current version of Messaging Server user, and that contains the following:

   !
   in-channel=*; out-channel=*;
     in-type=text; in-subtype=*;
     parameter-copy-0=*; dparameter-copy-0=*;
     original-header-file=1; override-header-file=1;
     command="msg-svr-base
   /lib/arabicdetect.sh"
   !

3. Compile your MTA configuration with the following command:

   msg-svr-base/sbin/imsimta cnbuild

4. Restart with the command:

   msg-svr-base/sbin/imsimta restart