Chapter 14 Integrating Spam and Virus Filtering Programs Into Messaging Server

This chapter describes how to integrate and configure spam and virus filtering software with Messaging Server. The spam/virus filtering technology described in this chapter is more powerful than the technology provided by the conversion channel (see 13.5 The Conversion Channel). Messaging Server supports Symantec Brightmail AntiSpam, SpamAssassin, Milter, and anti-spam/anti-virus programs which support Internet Content Adaptation Protocol (ICAP, RFC 3507), specifically Symantec AntiVirus Scan Engine.

In this chapter, references to anti-spam or spam filtering features also mean, when applicable, anti-virus or virus filtering features. Some products offer both (Brightmail), while others may offer only spam filtering (SpamAssassin) or only virus filtering (Symantec AntiVirus Scan Engine). Note also that spam is used generically in configuration parameters.

This chapter is divided into the following sections:

• 14.1 Integrating Spam Filtering Programs Into Messaging Server—Theory of Operations

• 14.2 Deploying and Configuring Third Party Spam Filtering Programs

• 14.3 Using Symantec Brightmail Anti-Spam

• 14.4 Using SpamAssassin

• 14.5 Using Symantec Anti-Virus Scanning Engine (SAVSE)

• 14.6 Using ClamAV

• 14.7 Support for Sieve Extensions

• 14.8 Using Milter

• 14.9 Cloudmark Anti-Abuse Client

• 14.10 Other Anti-Spam and Denial-of-Service Technologies

14.1 Integrating Spam Filtering Programs Into Messaging Server—Theory of Operations

From the perspective of Messaging Server, anti-spam solutions operate in much the same way:

1. Messaging Server sends a copy of a message to the spam filtering software.

2. The spam filtering software analyzes the message and returns a verdict of spam or not spam. Some programs, like SpamAssassin may also return a spam score, which is a numerical rating of the probability of the message being spam.

3. Messaging Server reads the verdict and takes a Sieve action on the message (see 14.2.3 Specifying Actions to Perform on Spam Messages).

Spam filtering programs interact with the MTA through a protocol. The protocol may be a standard as in ICAP-based programs such as Symantec AntiVirus Scan Engine, proprietary as in Brightmail, or simply non-standard as in SpamAssassin. Each protocol requires software hooks to interface with the MTA. Brightmail and SpamAssassin were the first two spam filtering programs that could be integrated with messaging server. The MTA now supports the programs that use ICAP.

14.2 Deploying and Configuring Third Party Spam Filtering Programs

There are five actions required to deploy third-party filtering software on Messaging Server:

1. Determine which spam filtering programs you wish to deploy, and how many servers on which to deploy them. Messaging Server allows you to filter incoming messages with up to eight different spam/virus programs. These programs can be run on separate systems, on the same system as Messaging Server in a single system deployment, or on the same system as the MTA in a two-tier deployment. The number of servers required depends on the message load, the hardware performance, and other factors. Refer to your spam filtering software documentation or representative for guidelines on determining the hardware requirements at your site.

2. Install and configure the spam filtering software. Refer to your spam filtering software documentation or representative for this information.

3. Load and configure the filtering client library. This involves specifying the client libraries and configuration files in the MTA option.dat file, and also setting the desired options in the filtering software’s configuration files. See 14.2.1 Loading and Configuring the Spam Filtering Software Client Library

4. Specify what messages get filtered. Messages can be filtered by user, domain, or channel. See 14.2.2 Specifying the Messages to Be Filtered.

5. Specify what happens to Spam. Spam can be discarded, filed into a folder, tagged on the subject line, and so on. See 14.2.3 Specifying Actions to Perform on Spam Messages

Previous versions of Messaging Server only supported the Brightmail filtering technology and so keywords and options had names, such as sourcebrightmail or Brightmail_config_file. These keywords and options have been changed to more generic names such as sourcespamfilter or spamfilter_config_file. The previous Brightmail names are retained for compatibility.

14.2.1 Loading and Configuring the Spam Filtering Software Client Library

Each spam filtering program is expected to provide a client library file and configuration file for Messaging Server. Loading and configuring the client library involves two things:

• Specifying the spam filtering software library path (spamfilterX_library) and configuration file (spamfilterX_config_file) in the option.dat file. In addition to these options, there are a number of others used to specify spam filtering LDAP attributes, as well as the Sieve actions to use on spam messages.

• Specifying the desired options in the spam filtering software configuration files. Each spam filtering program has a different configuration file and configuration options. These are described in the spam filtering software sections, as well as in the filtering software documentation. See 14.3 Using Symantec Brightmail Anti-Spam and 14.5 Using Symantec Anti-Virus Scanning Engine (SAVSE)

14.2.1.1 Specifying the Spam Filtering Software Library Paths

Messaging Server can call up to eight different filtering systems for your messages. For example, you can run your messages through both the Symantec AntiVirus Scan Engine and SpamAssassin. Each filtering software is identified by a number from 1 to 8. These numbers appear as part of the various spam filter options, LDAP attributes, and channel keywords; an X is used as a filter identification number. For example, sourcespamfilterXoptin or spamfilterX_config_file. If the identifying number is omitted from the keyword or option name it defaults to 1.

The following option.dat settings specify Messaging Server to filter messages through both Symantec AntiVirus Scan Engine and SpamAssassin:

spamfilter1_library=Symantec_Library_File
spamfilter1_config_file=Symantec_Config_File
spamfilter2_library=SpamAssassin_Library_File
spamfilter2_config_file=SpamAssassin_Config_File

When using other options or keywords to configure the system, use the corresponding number at the end of the option or keyword. For example, sourcespamfilter2optin would refer to SpamAssassin. sourcespamfilter1optin would refer to Symantec AntiVirus Scan Engine. It is not necessary to use numbers sequentially. For example, if you want to temporarily disable the Symantec AntiVirus Scan Engine, you can just comment out the spamfilter1_library configuration file.

14.2.2 Specifying the Messages to Be Filtered

Once the spam filtering software is installed and ready to run with Messaging Server, you need to specify what messages to filter. Messaging Server can be configured to filter messages by user, domain, or channel. Each of these scenarios is described in the following sections:

• To Specify User-level Filtering

• 14.2.2.1 User-level Filtering Example

The expression optin means that a user, domain or channel is selected to receive mail filtering.

To Specify User-level Filtering

It may be desirable to specify filtering on a per-user basis. For example, if spam or virus filtering is offered as a premium service to ISP customers, you can specify which users receive this and which don’t. The general steps for user filtering are as follows:

1. Specify the user LDAP attributes that activate the spam filtering software.

   Set the LDAP_OPTINX options in option.dat. Example:

   LDAP_OPTIN1=SymantecAV LDAP_OPTIN2=SpamAssassin

   ---

   Note –

   By default, the attributes like SymantecAV or SpamAssassin do not exist in the schema. Whatever new attributes you use, you will need to add them to your directory schema. See the appropriate Directory Server documentation for instructions.

   ---

2. Set filter attributes in the user entries that receive spam filtering.

   The values for the filter attributes are multi-valued and depend on the server. Using the example shown in Step 1, the entries are:

   SymantecAV: virus SpamAssassin: spam

   For a program like Brightmail, which can filter both viruses and spam, the valid values are spam and virus. When used as a multi-valued attribute, each value requires a separate attribute entry. For example, if the filter attribute for Brightmail was set to Brightmail, the entries are:

   Brightmail: spam Brightmail: virus

14.2.2.1 User-level Filtering Example

This example assumes that Brightmail is used. It also assumes that LDAP_OPTIN1 was set to Brightmail in the option.dat file. The user, Otis Fanning, has the Brightmail attribute set to spam and virus in his user entry. His mail is filtered by Brightmail for spam and viruses. 14.2.2.1 User-level Filtering Example shows the Brightmail user entry for Otis Fanning.

Example 14–1 Example LDAP User Entry for Brightmail

dn: uid=fanning,ou=people,o=sesta.com,o=ISP
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
objectClass: inetUser
objectClass: ipUser
objectClass: inetMailUser
objectClass: inetLocalMailRecipient
objectClass: nsManagedPerson
objectClass: userPresenceProfile
cn: Otis Fanning
sn: fanning
initials: OTF
givenName: Otis
pabURI: ldap://ldap.siroe.com:389/ou=fanning,ou=people,o=sesta.com,o=isp,o=pab
mail: Otis.Fanning@sesta.com
mailAlternateAddress: ofanning@sesta.com
mailDeliveryOption: mailbox
mailHost: manatee.siroe.com
uid: fanning
dataSource: iMS 5.0 @(#)ims50users.sh 1.5a 02/3/00
userPassword: password
inetUserStatus: active
mailUserStatus: active
mailQuota: -1
mailMsgQuota: 100
Brightmail: virus
Brightmail: spam

If Symantec AntiVirus Scan Engine and SpamAssassin were used, the entry would look like this:

SymantecAV: virus
SpamAssassin: spam

See 14.3 Using Symantec Brightmail Anti-Spam, 14.4 Using SpamAssassin or 14.5 Using Symantec Anti-Virus Scanning Engine (SAVSE)

To Specify Domain-level Filtering

You can specify which domains receive filtering. An example of this feature would be if anti-spam or anti-virus filtering were offered as a premium service to ISP domain customers. The general steps for specifying domain filtering is as follows:

1. Specify the domain LDAP attributes that activates the filtering software.

   Set the LDAP_DOMAIN_ATTR_OPTINX options in option.dat. Example:

   LDAP_DOMAIN_ATTR_OPTIN1=SymantecAV LDAP_DOMAIN_ATTR_OPTIN2=SpamAssassin

   ---

   Note –

   By default, the attributes like SymantecAV or SpamAssassin do not exist in the schema. Whatever new attributes you use, you will need to add them to your directory schema. See the appropriate Directory Server documentation for instructions.

   ---

2. Set filter attributes in the domain entries that receive spam filtering.

   The values for the filter attributes are multi-valued and depend on the server. Using the example shown in Step 1, the entries would be as follows:

   SymantecAV: virus SpamAssassin: spam

   For a program like Brightmail which can filter both viruses and spam, the valid values are spam and virus. When used as a multi-valued attribute, each value requires a separate attribute value entry. For example, if LDAP_DOMAIN_ATTR_OPTIN1 was set to Brightmail, the entries would be:

   Brightmail: spam Brightmail: virus

Domain-level Filtering Example

This example assumes that Brightmail is used. It also assumes that LDAP_DOMAIN_ATTR_OPTIN1 was set to Brightmail in the option.dat file. The Brightmail attribute is set to spam and virus in the sesta.com domain entry in the DC tree for Sun LDAP Schema 1. For Sun LDAP Schema 2 you also set Brightmail in the domain entries that receive spam filtering.

All mail sent to sesta.com is filtered for spam and viruses by Brightmail. A Domain-level Filtering Example is shown below.

Example 14–2 Example LDAP Domain Entry for Brightmail

dn: dc=sesta,dc=com,o=internet
objectClass: domain
objectClass: inetDomain
objectClass: mailDomain
objectClass: nsManagedDomain
objectClass: icsCalendarDomain
description: DC node for sesta.com hosted domain
dc: sesta
inetDomainBaseDN: o=sesta.com,o=isp
inetDomainStatus: active
mailDomainStatus: active
mailDomainAllowedServiceAccess: +imap, pop3, http:*
mailRoutingHosts: manatee.siroe.com
preferredMailHost: manatee.siroe.com
mailDomainDiskQuota: 100000000
mailDomainMsgQuota: -1
mailClientAttachmentQuota: 5
Brightmail: spam
Brightmail: virus
 

If Symantec AntiVirus Scan Engine and SpamAssassin were used, the entry would look similar to like this:

SymantecAV: virus
SpamAssassin: spam

See 14.3 Using Symantec Brightmail Anti-Spam, 14.4 Using SpamAssassin or 14.5 Using Symantec Anti-Virus Scanning Engine (SAVSE) for more examples and details.

To Specify Channel-level Filtering

Filtering by source or destination channel provides greater flexibility and granularity for spam filtering. For example, you may wish to filter in these ways:

• Only messages from a specific MTA relay to a backend message store

• All incoming mail from a specific MTA.

• All outgoing mail from a specific MTA.

• Incoming and outgoing mail from a specific MTA.

Messaging Server allows you to specify filtering by source or destination channel. The mechanism for doing this are the channel keywords described in 12.12.5 Spam Filter Keywords. The following example demonstrates how to set up channel-level filtering.

1. Add a rewrite rule in the imta.cnf file for all inbound SMTP servers that send messages to a backend message store host. Example:

   msg_store1.siroe.com $U@msg_store1.siroe.com

2. Add a channel corresponding to the rewrite rule with the destinationspamfilterXoptin keyword. Example:

   tcp_msg_store1 smtp subdirs 20 backoff "pt5m" "pt10" "pt30" \ "pt1h" "pt2h" "pt4h" maxjobs 1 pool IMS_POOL \ fileinto $U+$S@$D destinationspamfilter1optin spam msg_store1.siroe.com

Channel-level Filtering Examples

These examples assume a filtering program specified by the number 1. See 12.12.5 Spam Filter Keywords for the keywords available for spam filtering.

To Filter from an MTA Relay to a Backend Message Store

This example filters all mail for spam and viruses from an MTA relay to a backend message store called msg_store1.siroe.com

1. Add a rewrite rule in the imta.cnf file that sends messages to a backend message store host. Example:

   msg_store1.siroe.com   $U@msg_store1.siroe.com

2. Add a channel corresponding to that rewrite rule with the destinationspamfilterXoptin keyword. Example:

   tcp_msg_store1 smtp subdirs 20 backoff "pt5m" "pt10" "pt30" "pt1h" \
   "pt2h" "pt4h" maxjobs 1 pool IMS_POOL fileinto $U+$S@$D \
   destinationspamfilter 1optin spam,virus
   msg_store1.siroe.com

   Example 2. Filter for spam all incoming mail passing through your MTA (Typically, all incoming messages pass through the tcp_local channel):

   tcp_local smtp mx single_sys remotehost inner switchchannel \
   identnonelimited subdirs 20 maxjobs 7 pool SMTP_POOL \
   maytlsserver maysaslserver saslswitchchannel tcp_auth \
   sourcespamfilter1optin spam
   tcp-daemon

   Example 3. Filter all outgoing mail to the Internet passing through your MTA. (Typically, all messages going out to the Internet pass through the tcp_local channel.)

   tcp_local smtp mx single_sys remotehost inner switchchannel \
   identnonelimited subdirs 20 maxjobs 7 pool SMTP_POOL \
   maytlsserver maysaslserver saslswitchchannel tcp_auth \
   destinationspamfilter1optin spam tcp-daemon

   Example 4. Filter all incoming and outgoing mail passing through your MTA:

   tcp_local smtp mx single_sys remotehost inner switchchannel \
   identnonelimited subdirs 20 maxjobs 7 pool SMTP_POOL \
   maytlsserver maysaslserver saslswitchchannel tcp_auth \
   sourcespamfilter1optin spam destinationspamfilter1optin spam
   tcp-daemon

   Example 5. Filter all mail destined to the local message store in a two-tiered system without using user optin:

   ims-ms smtp mx single_sys remotehost inner switchchannel \
   identnonelimited subdirs 20 maxjobs 7 pool SMTP_POOL \
   maytlsserver maysaslserver saslswitchchannel tcp_auth \
   destinationspamfilter1optin spam
   tcp-daemon

   Example 6. Filter all incoming and outgoing mail for spam and viruses (this presumes that your software filters both spam and viruses):

   tcp_local smtp mx single_sys remotehost inner switchchannel \
   identnonelimited subdirs 20 maxjobs 7 pool SMTP_POOL \
   maytlsserver maysaslserver saslswitchchannel tcp_auth \
   destinationspamfilter1optin spam,virus sourcespamfilter1optin \spam,virus 
   tcp-daemon

14.2.3 Specifying Actions to Perform on Spam Messages

Spam filtering programs analyze messages and return a verdict of spam or not spam to current version of Messaging Server. Messaging Server then takes action on the message. Actions are specified using the Sieve mail filtering language. Possible actions are to discard the message, file it into a folder, add a header, add a tag to the subject line, and so on. Complex Sieve scripts with if-then-else statements are also possible.

Refer to the Sieve specification 3028 for the complete Sieve syntax. Also see http//www.cyrusoft.com/sievettp://www.cyrusoft.com/sieve/

Sieve scripts are specified with the MTA spam filter options (option.dat) described in Table 14–1. The primary spam filter action options are SpamfilterX_null_action, which specifies the Sieve rule to execute when a null value is returned as the spam verdict value, and SpamfilterX_string_action, which specifies the Sieve rule to execute when a string is returned as the spam verdict.

Spam filtering programs typically return a string or a null value to the MTA to indicate that message is spam. Some programs also return a spam score—a number rating the probability of the message being spam or not. This score can be used as part of the action sequence. The following examples show how to specify actions on filtered messages. Each example assumes a filtering program specified by the number 1.

Example 1: File spam messages with a null verdict value to the file SPAM_CAN.

spamfilter1_null_action=data:,require "fileinto"; fileinto "SPAM_CAN”;

The same action can be performed on a spam message that returns a string:

spamfilter1_string_action=data:,require "fileinto"; fileinto "SPAM_CAN”;

Example 2: File spam messages with a returned verdict string into a file named after the returned verdict string (this is what $U does). That is, if the verdict string returned is spam, the message is stored in a file called spam.

spamfilter1_null_action=data:,require "fileinto"; fileinto "$U”;

Example 3: Discard spam messages with a string verdict value.

spamfilter1_string_action=data:,discard

The same action can be performed on a spam message that returns a null value:

spamfilter1_null_action=data:,require "fileinto"; fileinto "SPAM_CAN”;

Example 4. This line adds the header Spam-test: FAIL to each message determined to be spam by a string verdict value:

spamfilter1_string_action=data:,require ["addheader"];addheader "Spam-test: FAIL”;

Example 5. This line adds the string [PROBABLE SPAM] to the subject line of the spam messages returning a string:

spamfilter1_string_action=data:,addtag “[PROBABLE SPAM]”;

Example 6. This line assumes a string verdict value and files a spam message in the mailbox testspam if the header contains resent-from and User-1. If the message does not have that header, it files the message into spam.

spamfilter1_string_action=data:,require "fileinto"; \
  if header :contains ["resent-from"] ["User-1"] {  \
  fileinto "testspam"; \
  } else {  \
  fileinto "spam";};

Because verdict strings are configurable with most spam filter software, you can specify different actions depending on the returned string. This can be done with the matched pairs spamfilterX_verdict_n and spamfilterX_action_n options.

Example 7. These matched pair options discard spam messages with the returned verdict string of remove.

spamfilter1_verdict_0=remove
spamfilter1_action_0=data:,discard

Refer to the specific spam filtering software sections for instructions on how to specify the spam verdict string.

spamfilter1_verdict_0=reject
spamfilter1_action_0=data:,require "reject";
reject "Rejected by spam filter";

14.3 Using Symantec Brightmail Anti-Spam

The Brightmail solution consists of the Brightmail server along with realtime anti-spam and anti-virus rule updates downloaded to email servers. In addition to the sections below, refer to Configuring Brightmail with Sun Java System Messaging Server.

• 14.3.1 How Brightmail Works

• 14.3.2 Brightmail Requirements and Performance Considerations

• 14.3.3 Deploying Brightmail

• 14.3.4 Brightmail Configuration Options

14.3.1 How Brightmail Works

The Brightmail server is deployed at a customer site. Brightmail has email probes set around the Internet for detection of new spam. Brightmail technicians create custom rules to block this spam in realtime. These rules are downloaded to Brightmail servers, also in realtime. The Brightmail database is updated and Brightmail server runs this database filter against the email for the specified users or domains.

14.3.1.1 Brightmail Architecture

Figure 14–1 depicts the Brightmail architecture.

Figure 14–1 Brightmail and Messaging Server Architecture

When the Brightmail Logistics and Operations Center (BLOC) receives spam from email probes, operators immediately create appropriate spam filtering rules, which are downloaded to Brightmail customer machines. Similarly, the Symantec Security Response realtime virus rules are also sent from Brightmail. These rules are used by customer’s Brightmail servers to catch spam and viruses.

The MTA uses the Brightmail SDK to communicate with the Brightmail Server. The MTA dispatches messages based on the response from Brightmail. After the mail (1a) or (1b) is received by the MTA, the MTA sends the message to the Brightmail server (2). The Brightmail server uses its rules and data to determine if the message is a spam or virus (3), and returns a verdict to the MTA. Based on the verdict, the MTA either (4a) discards the message or files the message into a folder, or (4b) delivers it normally to the destination.

Since the Brightmail SDK is third party software, we do not include it in our installation kit. The Brightmail SDK and server software must be obtained from Brightmail Inc. The MTA has configuration settings to tell it whether and where to load the Brightmail SDK to enable Brightmail integration.

Once the SDK is loaded, Brightmail message processing is determined by several factors and levels of granularity (the term used by Brightmail to specify active processing is optin). This is specified by the following criteria:

• Whether the source or destination channel is enabled for Brightmail (imta.cnf)

• Whether there is a channel default for the services opted in (imta.cnf)

• Whether there is a per domain optin (LDAP)

• Whether there is per-user optin (LDAP)

For any particular message recipient, the optins and defaults above are combined, which means, if the channel default is already specified for both spam and virus, then there is no reason to bother with per-user optin. That is, if the system administrator decides to do spam and virus filtering for everyone, then there is no reason to expose to the user the ability to optin for spam or virus. There is no way to opt out of processing, that is, you can not say you do not want the service if a user is already optin via a system or domain optin. This also means that if you are optin for a service, and you have forwarded your mail to another address, that address would get the mail after the filtering has been performed on your behalf.

There are only two services offered, virus or spam detection. Brightmail also provides “content-filtering” service, but this functionality is provided using Sieve, so there is no added value to have Brightmail do the Sieve filtering.

When a message is determined to contain a virus, the Brightmail server can be configured to clean the virus and resubmit the cleaned message back to the MTA. (Due to some undesirable side effects caused by loss of information about the original message in a resubmitted cleaned message, we recommend you do not configure Brightmail to resubmit the cleaned message back to the MTA.) When the message is spam, the verdict back from the Brightmail along with the configuration in Brightmail allows the MTA to determine what happens to the message. The message can be discarded, filed into a folder, tagged as spam or virus on the subject line, passed to a Sieve rule, delivered normally in the INBOX, and so on.

The Brightmail servers can be located on the same system as the MTA, or it can be on a separate system. In fact, you can have a farm of Brightmail servers serving one or more MTAs. The Brightmail SDK uses the Brightmail configuration file to determine which Brightmail servers to use.

14.3.2 Brightmail Requirements and Performance Considerations

• Brightmail servers must run on the Solaris Operating System.

• If Brightmail does spam and virus checking, MTA message throughput can be reduced by as much 50%. To keep up with MTA throughput, you may need two Brightmail servers for each MTA.

• While SpamAssassin has the ability to perform different sorts of filtering on a user basis, it is unable to apply two different sets of filtering criteria to the same message at the same time. Thus, SpamAssassin only allows system-wide filtering. Customized filtering for individual users is not possible.

14.3.3 Deploying Brightmail

Perform the following steps to deploy Brightmail.

• Install and Configure Brightmail. Refer to the Brightmail software documentation or representative for installation and configuration information. Selected Brightmail configuration options are shown in 14.3.4 Brightmail Configuration Options, however the most complete and up-to-date information is in the Brightmail documentation.

• Load and Configure the Brightmail Client Library. This involves specifying the Brightmail client library, libbmiclient.so, and configuration, config, file to the MTA. See 14.2.1 Loading and Configuring the Spam Filtering Software Client Library

• Specify what messages to filter for spam. Messages can be filtered by user, domain, or channel. See 14.2.2 Specifying the Messages to Be Filtered

• Specify what actions to take on spam messages. Spam can be discarded, filed into a folder, tagged on the subject line, and so on. See 14.2.3 Specifying Actions to Perform on Spam Messages

• Set miscellaneous MTA filter configuration parameters as desired. See Table 14–1.

14.3.4 Brightmail Configuration Options

Selected Brightmail configuration file options are shown in Table 14–2. The most complete listing of Brightmail configuration file options can be obtained from Brightmail. Options and values are not case-sensitive.

14.4 Using SpamAssassin

This section consists of the following subsections:

• 14.4.1 SpamAssassin Overview

• 14.4.2 SpamAssassin/Messaging Server Theory of Operations

• 14.4.3 SpamAssassin Requirements and Usage Considerations

• 14.4.4 Deploying SpamAssassin

• 14.4.5 SpamAssassin Configuration Examples

• 14.4.6 Testing SpamAssassin

• 14.4.7 SpamAssassin Options

14.4.1 SpamAssassin Overview

Messaging Server supports the use of SpamAssassin, a freeware mail filter used to identify spam. SpamAssassin consists of a library written in Perl and a set of applications and utilities that can be used to integrate SpamAssassin into messaging systems.

SpamAssassin calculates a score for every message by performing a series of tests on the message header and body information. Each test succeeds or fails, and a verdict of true (spam) or false (not spam) is rendered. Scores are real numbers that can be positive or negative. Scores that exceed a specified threshold, typically 5.0, are considered to be spam. An example of a SpamAssassin result string is:

True ; 18.3 / 5.0

True indicates the message is spam. 18.3 is the SpamAssassin score. 5.0 is the threshold.

SpamAssassin is highly configurable. Tests may be added or removed at any time, and the scores of existing tests can be adjusted. This is all done through various configuration files. Further information on SpamAssassin can be found on the SpamAssassin web site.

The same mechanism used for calling out to the Brightmail spam and virus scanning library can be used to connect to the SpamAssassin spamd server. The module provided in Messaging Server is called libspamass.so.

14.4.2 SpamAssassin/Messaging Server Theory of Operations

spamd is the daemon version of SpamAssassin and can be invoked from the MTA. spamd listens on a socket for requests and spawns a child process to test the message. The child process dies after processing the message and sending back a result. In theory, the forking should be an efficient process because the code itself is shared among the children processes.

The client portion, spamc from the SpamAssassin installation, is not used. Instead, its function is done by a shared library called libspamass.so, which is part of the Messaging Server. libspamass.so is loaded the same way that the Brightmail SDK is loaded.

From the MTA’s point of view, you can almost transparently switch between SpamAssassin and Brightmail for spam filtering. It’s not completely transparent, because they do not have the same functions. For example, Brightmail can also filter for viruses, but SpamAssassin is only used to filter for spam. The result, or verdict, returned by the two software packages is also different. SpamAssassin provides a score, while Brightmail provides just the verdict name, so the configuration would also have some differences.

When using SpamAssassin with the MTA, only a score and verdict is returned from SpamAssassin. The message itself is not modified. That is, options such as adding headers and modifying subject lines must be done by Sieve scripts. In addition, the mode option allows you to specify the string that is returned to indicate the verdict. The string choices are null, default, SpamAssassin result string, or a verdict string. See 14.4.7 SpamAssassin Options for details.

14.4.3 SpamAssassin Requirements and Usage Considerations

• SpamAssassin is free. Go to http//www.spamassassin.org for software and documentation.

• SpamAssassin can be tuned and configured to provide very accurate detection of spam. The tuning is up to you and the SpamAssassin community. Messaging Server does not provide or enhance what SpamAssassin can do.

• While no specific numbers are available, SpamAssassin seems to reduce throughput more than Brightmail.

• SpamAssassin integrated with the MTA can be enabled for a user, a domain, or a channel.

• SpamAssassin can be configured to use other online databases such as Vipul Razor or Distributed checksum clearinghouse (DCC).

• Messaging Server does not supply an Secure Socket Layer (SSL) version of libspamass.so, however, it is possible to build SpamAssassin to use openSSL.

• Perl 5.6 or later is required.

14.4.3.1 Where Should You Run SpamAssassin?

SpamAssassin can run on a separate system of its own, on the same system as the Messaging Server in a single system deployment, or on the same system as the MTA in a two-tier deployment. If Local Mail Transfer Protocol (LMTP) is used between the MTA and the message store, the filtering must be invoked from the MTA. It cannot be invoked from the message store. When SMTP is used between the MTA and the message store, it can be invoked from either one, and it can run on either system or a separate third system.

If you want to use a farm of servers running SpamAssassin, you would have to use a load balancer in front of them. The MTA is configured with only one address for the SpamAssassin server.

14.4.4 Deploying SpamAssassin

Perform the following steps to deploy SpamAssassin:

• Install and configure SpamAssassin. Refer to the SpamAssassin software documentation for installation and configuration information. See also 14.4.7 SpamAssassin Options.

• Load and configure the SpamAssassin client library. This involves specifying the client library, libspamass.so, and configuration file to the MTA (you must create this file). See 14.2.1 Loading and Configuring the Spam Filtering Software Client Library

• Specify what messages to filter for spam. Messages can be filtered by user, domain, or channel. See 14.2.2 Specifying the Messages to Be Filtered

• Specify what actions to take on spam messages. Spam can be discarded, filed into a folder, tagged on the subject line, and so on. See 14.2.3 Specifying Actions to Perform on Spam Messages

• Set miscellaneous filter configuration parameters as desired. See Table 14–1

14.4.5 SpamAssassin Configuration Examples

This section describes some common SpamAssassin configuration examples:

• To File Spam to a Separate Folder

• To Add a Header Containing SpamAssassin Score to Spam Messages

• To Add the SpamAssassin Result String to the Subject Line

These examples use a number of options and keywords. Refer to 12.12.5 Spam Filter Keywords and Table 14–1.

To File Spam to a Separate Folder

This example tests messages arriving at the local message store and files spam into a folder called spam. The first three steps can be done in any order.

1. Create the SpamAssassin configuration file.

   The name and location of this file is specified in Step 2. A good name is spamassassin.opt. This file contains the following lines:

   host=127.0.0.1 port=2000 mode=0 verdict=spam debug=1

   host and port specify the name of the system where spamd is running and the port on which spamd listens for incoming requests. mode=0 specifies that a string, specified by verdict, is returned if the message is perceived as spam. debug=1 turns on debugging in the SpamAssassin library. See Table 14–3

2. Add the following lines to the option.dat file:

   ! for Spamassassin spamfilter1_config_file=/opt/SUNWmsgsr/config/spamassassin.opt spamfilter1_library=/opt/SUNWmsgsr/lib/libspamass.so spamfilter1_optional=1 spamfilter1_string_action=data:,require "fileinto"; fileinto "$U";

   spamfilter1_config_file specifies the SpamAssassin configuration file.

   spamfilter1_library specifies the SpamAssassin shared library.

   spamfilter1_optional=1 specifies that the MTA continue operation if there is a failure by spamd.

   spamfilter1_string_action specifies the Sieve action to take for a spam messages.

   In this example, spamfilter1_string_action is not necessary because the default value already is data:,require "fileinto"; fileinto "$U";. This line specifies that spam messages are sent to a folder. The name of the folder is the spam verdict value returned by SpamAssassin. The value returned by SpamAssassin is specified by the verdict option in spamassassin.opt. (See Step 1.) In this case, the folder name is spam.

3. Specify the messages to be filtered.

   To filter all messages coming into the local message store, change the imta.cnf file by adding the destinationspamfilterXoptin spam keywords on the ims-ms channel:

   ! ! ims-ms ims-ms defragment subdirs 20 notices 1 7 14 21 28 backoff "pt5m" "pt10m" "pt30m" "pt1h" "pt2h" "pt4h" maxjobs 4 pool IMS_POOL fileinto $U+$S@$D destinationspamfilter1optin spam ims-ms-daemon

4. Recompile the configuration and restart the server. Only the MTA needs to be restarted. You do not need to execute stop-msg.

   # imsimta cnbuild # imsimta restart

5. Start the spamd daemon. This is normally done with a command of the form:

   spamd -d

   spamd defaults to only accepting connections from the local system. If SpamAssassin and Messaging Server are running on different systems, this syntax is required:

   spamd -d -i listen_ip_address -A allowed_hosts

   where listen_ip_address is the address on which to listen and allowed_hosts is a list of authorized hosts or networks (using IP addresses) which can connect to this spamd instance.

   ---

   Note –

   0.0.0.0 can be used with -i listen_ip_address to have spamd listen on all addresses. Listening on all addresses is preferable because it spamfilterX_verdict_n avoids having to change command scripts when changing a system’s IP address.

   ---

To Add a Header Containing SpamAssassin Score to Spam Messages

This example adds the header Spam-test: result string to messages determined to be spam by SpamAssassin. An example header might be:

Spam-test: True ; 7.3 / 5.0

where Spam-test: is a literal and everything after that is the result string. True means that it is spam (false would be not spam). 7.3 is the SpamAssassin score. 5.0 is the threshold. This result is useful for setting up a Sieve filter that can file or discard mail above or between a certain score.

In addition, setting USE_CHECK to 0 returns the list of SpamAssassin tests that matched along with the verdict string. See USE_CHECK in Table 14–3.

1. Specify the messages to be filtered. This is described in Step 3 in To File Spam to a Separate Folder

2. Create the SpamAssassin configuration file.

   The name and location of this file is specified with spamfilter_configX_file (see next step). It consists of the following lines:

   host=127.0.0.1 port=2000 mode=1 field= debug=1

   host and port specify the name of the system where spamd is running and the port on which spamd listens for incoming requests. mode=1 specifies that the SpamAssassin result string is returned if the message is found to be spam. field= specifies a string prefix for the SpamAssassin result string. In this example, a prefix is not desired because we are specifying it in the Sieve script. debug=1 turns on debugging in the SpamAssassin library.

3. Add the following lines to the option.dat file:

   !for Spamassassin spamfilter1_config_file=/opt/SUNWmsgsr/config/spamassassin.opt spamfilter1_library=/opt/SUNWmsgsr/lib/libspamass.so spamfilter1_optional=1 spamfilter1_string_action=data:,require ["addheader"];addheader "Spam-test: $U";

   As in previous examples, the first three options specify the SpamAssassin configuration file, shared library, and to continue MTA operation if there is a failure by the shared library. The following line:

   spamfilter1_string_action=data:,require ["addheader"];addheader "Spam-test: $U";

   specifies that a header is added to spam messages. The header has the literal prefix Spam-text: followed by the string returned by SpamAssassin. Because mode=1 was specified in the previous step, the SpamAssassin result string is returned. For example: True; 7.3/5.0

4. Recompile the configuration, restart the server and start the spamd daemon.

   See 14.4.5 SpamAssassin Configuration Examples.

To Add the SpamAssassin Result String to the Subject Line

By adding the SpamAssassin result string to the Subject line, users can determine whether they wish to read a message with a SpamAssassin score. For example:

Subject: [SPAM True ; 99.3 / 5.0] Free Money At Home with Prescription Xanirex!

Note that setting USE_CHECK to 0 returns the list of SpamAssassin tests that matched along with the verdict string (see 14.4.7 SpamAssassin Options in 14.4.7 SpamAssassin Options). This list can be very long, so it is best to set USE_CHECK to 1.

1. Specify the messages to be filtered.

   See Step 3 in To File Spam to a Separate Folder

2. Create the SpamAssassin configuration file.

   This step is described in To File Spam to a Separate Folder. mode=1 specifies that the SpamAssassin result string is returned if the message is found to be spam.

   host=127.0.0.1 port=2000 mode=1 debug=1

   host and port specify the name of the system where spamd is running and the port on which spamd listens for incoming requests. mode=1 specifies that the SpamAssassin result string is returned if the message is spam. debug=1 turns on debugging in the SpamAssassin library.

3. Add the following lines to the option.dat file:

   !for Spamassassin spamfilter1_config_file=/opt/SUNWmsgsr/config/spamassassin.opt spamfilter1_library=/opt/SUNWmsgsr/lib/libspamass.so spamfilter1_optional=1 spamfilter1_string_action=data:,addtag “[SPAM detected: $U]”;

   As in previous examples, the first three options specify the SpamAssassin configuration file, shared library, and to continue MTA operation if there is a failure by the shared library. The following line

   spamfilter1_string_action=data:,addtag “[SPAM detected $U]”;

   specifies that a tag be added to the Subject: line. It has the literal prefix SPAM detected followed by the field string (default: Spam-Test) followed by “[result string]” returned by SpamAssassin. Because mode=1 was specified in 14.4.5 SpamAssassin Configuration Examples, the SpamAssassin result string is returned. Thus, a subject line looks something like this:

   Subject: [SPAM detected Spam-Test: True ; 11.3 / 5.0] Make Money!

   You can also use addheader and addtag together:

   spamfilter1_string_action=data:,require ["addheader"];addtag "[SPAM detected $U]";addheader "Spamscore: $U";

   to get a message like this:

   Subject: [SPAM detected Spam-Test: True ; 12.3 / 5.0] Vigaro Now!Spamscore: Spam-Test: True ; 12.3 / 5.0

   Set field= in spamassassin.opt to remove the default value of Spam-Test. A cleaner message is returned:

   Subject: [SPAM True ; 91.3 / 5.0] Vigaro Now!Spamscore: True ; 91.3 / 5.0

4. Recompile the configuration, restart the server and start the spamd daemon.

   See To File Spam to a Separate Folder.

To Filter Messages Based on SpamAssassin Score

This example shows how to filter messages based on a SpamAssassin score. It uses the spamadjust and spamtest Sieve filter actions. In this example, a header containing the SpamAssassin score is added to all messages. This header can be used by the SpamAssassin software administrator to tune SpamAssassin for improved spam email detection. If the message has a SpamAssassin score between 5 and 10, the message is filtered to a spam folder within the user's account. If the message has a SpamAssassin score greater then 10, the message is discarded. Note that by default SpamAssassin considers messages with a score of 5 and greater to be spam.

1. Specify the messages to be filtered.

   This is described in Step 3 of To File Spam to a Separate Folder.

2. Create the SpamAssassin configuration file.

   The name and location of this file is specified with spamfilter_configX_file (see next step). It consists of the following lines:

   debug=1 host=127.0.0.1 port=783 mode=2 field=

   host and port specify the name of the system where spamd is running and the port on which spamd listens for incoming requests. mode=2 specifies that the SpamAssassin result string is always returned regardless of the score. field= specifies a string prefix for the SpamAssassin result string. In this example, a prefix is not desired because we are specifying it in the Sieve script. debug=1 turns on debugging in the SpamAssassin library.

3. Add the following lines to the option.dat file

   ! For SpamAssassin spamfilter1_config_file=/opt/SUNWmsgsr/config/spamassassin.opt spamfilter1_library=/opt/SUNWmsgsr/lib/libspamass.so spamfilter1_optional=1 spamfilter1_string_action=data:, require ["addheader","spamtest"]; \ spamadjust "$U"; addheader "Spam-test: $U"

   As in previous examples, the first three lines specify the SpamAssassin configuration file, shared library, and to continue MTA operation if there is a failure by the shared library. The last two lines specify that the SpamAssassin score should be extracted from the return string from SpamAssassin ($U), which is used in the spamtest operation, and a spam score header should be added to all messages (for example, Spam-test: True; 7.3/5.0)

4. Create a channel level filter to process the email based on the spam score.

   Refer to To Create a Channel-level Filter. Add the following rule to that file:

   require ["spamtest","relational","comparator-i;ascii-numeric","fileinto"]; if spamtest :value "ge" :comparator "i;ascii-numeric" "10" {discard;} elsif spamtest :value "ge" :comparator "i;ascii-numeric" "5" {fileinto "spam";} else {keep;}

   The second line discards the spam email if the SpamAssassin score is greater or equal to 10. The third line files the email to the users "spam" folder if the score is greater or equal to 5. The last line else {keep;} keeps all messages which received a score less then 5.

5. Recompile the configuration, restart the server and start the spamd daemon

   See the final steps in To File Spam to a Separate Folder.

14.4.6 Testing SpamAssassin

To test SpamAssassin, first set debug=1 in the spamassassion.opt file. You do not have to turn on the channel-specific master_debug or slave_debug in the imta.cnf. Then send a test message to a test user. The msg-svr-base/data/log/tcp_local_slave.log* file should have lines similar to these:

15:15:45.44: SpamAssassin callout debugging enabled; config 
/opt/SUNWmsgsr/config/spamassassin.opt
15:15:45.44: IP address 127.0.0.1 specified
15:15:45.44: Port 2000 selected
15:15:45.44: Mode 0 selected
15:15:45.44: Field "Spam-Test: " selected
15:15:45.44: Verdict "spam" selected
15:15:45.44: Using CHECK rather than SYMBOLS
15:15:45.44: Initializing SpamAssassin message context
...
15:15:51.42: Creating socket to connect to SpamAssassin
15:15:51.42: Binding SpamAssassin socket
15:15:51.42: Connecting to SpamAssassin
15:15:51.42: Sending SpamAssassin announcement
15:15:51.42: Sending SpamAssassin the message
15:15:51.42: Performing SpamAssassin half close
15:15:51.42: Reading SpamAssassin status
15:15:51.67: Status line: SPAMD/1.1 0 EX_OK
15:15:51.67: Reading SpamAssassin result
15:15:51.67: Result line: Spam: False ; 1.3 / 5.0
15:15:51.67: Verdict line: Spam-Test: False ; 1.3 / 5.0
15:15:51.67: Closing connection to SpamAssassin
15:15:51.73: Freeing SpamAssassin message context

If your log file does not contain lines similar to these, or if spamd is not running, the following error message is returned in your SMTP dialog after the last period (.) is sent to the SMTP server.

452 4.4.5 Error writing message temporaries - Temporary scan failure: End message status = -1

In addition, if spamfilter1_optional=1 (highly recommended) is set in option.dat, the message is accepted, but not filtered. It is as if spam filtering was not enabled, and the following lines appear in tcp_local_slave.log*:

15:35:15.69: Creating socket to connect to SpamAssassin
15:35:15.69: Binding SpamAssassin socket
15:35:15.69: Connecting to SpamAssassin
15:35:15.69: Error connecting socket: Connection refused
15:35:15.72: Freeing SpamAssassin message context
         

The call to SpamAssassin happens after the SMTP server received the entire message (that is, after the last “.” is sent to the SMTP server), but before the SMTP server acknowledges to the sender that it accepted the message.

Another test is to send a sample spam message using sample-spam.txt from, for example, the Mail-SpamAssassin-2.60 directory. This message has the following special text string in it:

XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

The corresponding tcp_local_slave.log* contains something like this:

16:00:08.15: Creating socket to connect to SpamAssassin
16:00:08.15: Binding SpamAssassin socket
16:00:08.15: Connecting to SpamAssassin
16:00:08.15: Sending SpamAssassin announcement
16:00:08.15: Sending SpamAssassin the message
16:00:08.15: Performing SpamAssassin half close
16:00:08.15: Reading SpamAssassin status
16:00:08.43: Status line: SPAMD/1.1 0 EX_OK
16:00:08.43: Reading SpamAssassin result
16:00:08.43: Result line: Spam: True ; 1002.9 / 5.0
16:00:08.43: Verdict line: Spam-Test: True ; 1002.9 / 5.0
16:00:08.43: Closing connection to SpamAssassin
16:00:08.43: Mode 0 verdict of spam
16:00:08.43: Mode 0 verdict of spam
16:00:08.47: Freeing SpamAssassin message context

A corresponding entry in the mail.log_current file would look as follows. Note the +spam part of the destination address, which means the message is filed in the folder called spam:

15-Dec-2003 15:32:17.44 tcp_intranet ims-ms E 1 morchia@siroe.com rfc822;
morchia morchia+spam@ims-ms-daemon 15-Dec-2003 15:32:18.53 
ims-ms D 1 morchia@siroe.com rfc822;morchia morchia+spam@ims-ms-daemon

         

14.4.7 SpamAssassin Options

This section contains the SpamAssassin option table.

current-username|current-recipient-address|current-optin-string

14.4.7.1 The SpamAssassin mode Option

After processing a message, SpamAssassin determines whether a message is spam or not. mode allows you to specify the string that is returned to indicate the verdict. The string choices are null, default, SpamAssassin result string, or a verdict string specified with the verdict option. (Note that default is neither null, the SpamAssassin result string, nor the string specified by verdict, but some other non-configurable result string.) The mode operations are outlined in the table below.

The first column indicates whether the verdict option is set or not set. The second column indicates whether the message is spam or not. The mode columns indicate the string returned for the various modes. For example, if verdict is not set and mode is set to 0 and a message is not spam, a default string is returned. If the verdict is set to YO SPAM! and mode is set to 0 and a message is spam, the string YO SPAM! is returned.

14.5 Using Symantec Anti-Virus Scanning Engine (SAVSE)

In addition to describing how to deploy SAVSE this section can also be useful in deploying other ICAP-supported anti-spam/anti-virus programs. This section consists of the following subsections:

• 14.5.1 SAVSE Overview

• 14.5.2 SAVSE Requirements and Usage Considerations

• 14.5.3 Deploying SAVSE

• 14.5.4 SAVSE Configuration Example

• 14.5.5 SAVSE Options

14.5.1 SAVSE Overview

SAVSE is a TCP/IP server application and communication Application Programming Interface (API) that provides virus scanning services. Designed specifically to protect traffic served through, or stored on, network infrastructure devices, it detects and protects against viruses, worms, and Trojan horses in all major file types, including mobile code and compressed file formats. Refer to the Symantec website for detailed information

The current version of Messaging Server only supports the SAVSE scan function. It does not support the repair or delete functions.

14.5.2 SAVSE Requirements and Usage Considerations

This is a separately licensed product from Symantec.

Only the scan mode is supported, not the scan and repair or scan and delete mode in the SAVSE configuration.

14.5.2.1 Where Should You Run SAVSE?

SAVSE or another server that supports ICAP can run on a separate system of its own, on the same system as the Messaging Server in a single system deployment, or in a two-tier deployment on the same system as the MTA. If Local Mail Transfer Protocol (LMTP) is used between the MTA and the message store, the filtering must be invoked from the MTA. It cannot be invoked from the message store. When SMTP is used between the MTA and the message store, it can be invoked from either one, and it can run on either system or a separate third system.

If you want to use a farm of servers running SAVSE, you would have to use a load balancer in front of them. The MTA is configured with only one address for the SAVSE server.

14.5.3 Deploying SAVSE

Perform the following steps to deploy SAVSE.

• Install and configure the SAVSE. Refer to the Symantec software documentation for installation and configuration information. See also 14.5.5 SAVSE Options.

• Load and configure the SAVSE client library. This involves specifying the client library libicap.so and configuration file to the MTA (you must to create this file). See 14.2.1 Loading and Configuring the Spam Filtering Software Client Library

• Specify what messages to filter for viruses. Messages can be filtered by user, domain, or channel. See 14.2.2 Specifying the Messages to Be Filtered

• Specify what actions to take on virus messages. Viruses can be discarded, filed into a folder, tagged on the subject line, and so on. See 14.2.3 Specifying Actions to Perform on Spam Messages

• Set miscellaneous filter configuration parameters as desired. See Table 14–114.2.3 Specifying Actions to Perform on Spam Messages

14.5.4 SAVSE Configuration Example

The following example tests messages arriving at the local message store and discards messages with attached viruses. The first three steps can be done in any order.

To Configure SAVSE

1. Create the SAVSE configuration file.

   The name and location of this file is specified in the next step. The name used here is SAVSE.opt. An example of this file is shown below:

   host=127.0.0.1 port=1344 mode=0 verdict=virus debug=1

   host and port specify the name of the system where the SAVSE program is running and the port (1344 is the default for SAVSE) on which it listens for incoming requests. mode=0 specifies that a string, specified by verdict (in this case the word virus), will be returned if the message is perceived to contain a virus. debug=1 turns on debugging. See 14.5.5 SAVSE Options for a description of the ICAP configuration parameters.

2. Create an option.dat file. Example:

   ! for Symantex Anti-virus Scan Engine spamfilter1_config_file=/opt/SUNWmsgsr/config/SAVSE.opt spamfilter1_library=/opt/SUNWmsgsr/lib/libicap.so spamfilter1_optional=1 spamfilter1_string_action=data:,discard

   spamfilter1_config_files specifies the SAVSE configuration file.

   spamfilter1_library specifies the location of the SAVSE shared library.

   spamfilter1_optional=1 specifies that the MTA continue operation if there is a failure by the SAVSE program.

   spamfilter1_string_action specifies the Sieve action to take for a spam messages. This value specifies that messages with viruses are discarded. Since this is the default value, you don’t have to specify it unless you are changing the value.

3. Specify the messages to be filtered.

   To filter all messages coming into the local message store, change the imta.cnf file by adding the destinationspamfilter1optin spam keywords on the ims-ms channel:

   ! ! ims-ms ims-ms defragment subdirs 20 notices 1 7 14 21 28 backoff "pt5m" "pt10m" "pt30m" "pt1h" "pt2h" "pt4h" maxjobs 4 pool IMS_POOL fileinto $U+$S@$D destinationspamfilter1optin virus ims-ms-daemon

4. Recompile the configuration and restart the server. Only the MTA needs to be restarted. You do not need to execute stop-msg.

   # imsimta cnbuild # imsimta restart

5. Make sure SAVSE is started.

   It should have started automatically, but if not, the start command might looks something like this: /etc/init.d/symcscna start

14.5.4.1 Other Possible Configurations

Setting mode to 0 can be used with a spamfilterX_null_option to take some other action, such as filing messages in a particular folder when they are determined to be spam. For example:

spamfilter1_null_option=data:,require "fileinto"; fileinto "VIRUS";

Note that filing infected messages into a folder is not a good idea in most cases.

Setting mode to 1 can be used to start an action. For example, the spam result could be included in the reject message by setting mode to 1 and the spamfilterX_string_action option in the MTA to something like:

spamfilter1_string_action=data:,require "reject"; reject "Message contained a virus [$U]";

Like fileinto, using the reject action to deal with viruses is rarely a good idea because it sends the virus back to the sender.

You could also add a tag to the spam message header by adding a line to the option.dat file. Example:

spamfilter1_string_action=data:,addtag “[SPAM detected!]”;

Setting mode to 2 can be used where an action needs to be taken regardless of whether or not the message was determined to contain a virus. The addition of a header field that can subsequently be tested is an obvious application for mode 2:

spamfilterX_string_action=data:,require ["addheader"];addheader "$U"

14.5.5 SAVSE Options

The SAVSE option file is really a more generic ICAP option file. Its name and location is set by spamfilterX_config_file in option.dat. It consists of lines of the form option=value. The one required option is HOST. It must be set to the name of system where the ICAP filtering server is running. This option must be set even if the ICAP server is running on the local host. The option file is shown below.

14.5.5.1 The ICAP mode Option

After processing a message, ICAP anti-virus programs like SASVE determines whether a message has a virus or not. mode allows you to specify the string returned by the ICAP program indicating this verdict. The string choices are null, default, ICAP result string, or a verdict string (specified with the verdict option). Note that default is not null, the ICAP result string, nor the string specified by verdict, but some other non-configurable string returned by the program. The mode operations are outlined in the table below.

The first column indicates whether the verdict option is set or not set. The second column indicates whether the message contains a virus or not. The mode columns indicate the string returned for the various modes. For example, if verdict is not set and mode is set to 0 and a message does not have a virus, the ICAP program returns a default. If the verdict is set to WARNING VIRUS! and mode is set to 0 and a message does have a virus, the ICAP program returns the string WARNING VIRUS!

14.6 Using ClamAV

Messaging server supports the use of the popular and freely available third-party virus scanner ClamAV for the detection of virus- and Trojan horse- infected messages. Virus signatures used by ClamAV to detect newly created viruses can be automatically updated using the freshclam utility provided with the ClamAV software package.

Further information on ClamAV can be found at the ClamAV website.

• 14.6.1 ClamAV/Messaging Server Theory of Operations

• 14.6.2 ClamAV Requirements and Usage Considerations

• 14.6.3 Deploying ClamAV

• To Jettison Virus– or Trojan Horse– Infected Email Using ClamAV

• 14.6.4 Testing ClamAV

• 14.6.5 ClamAV Options

14.6.1 ClamAV/Messaging Server Theory of Operations

ClamAV integration in Messaging Server makes use of the clamd daemon that is provided as part of the ClamAV package. clamd is a multi-threaded process that listens on a socket for requests to process messages. After processing the message, it sends back a response and closes the connection. The client portion, clamdscan from the ClamAV installation, is not used. This function is done by a shared library called libclamav.so, which is part of Messaging Server.

libclamav.so is loaded the same way as the Brightmail SDK is loaded.

14.6.2 ClamAV Requirements and Usage Considerations

ClamAV can run on a separate system of its own, on the same system as the Messaging Server in a single system deployment, or on the same system as the MTA in a two-tier deployment. If Local Mail Transfer Protocol (LMTP) is used between the MTA and the message store, the filtering must be invoked from the MTA. It cannot be invoked from the message store. When SMTP is used between the MTA and the message store, it can be invoked from either one.

If you want to use a farm of servers running ClamAV, use a load balancer front of them. The MTA is configured with only one address for the ClamAV server.

Other considerations.

• ClamAV is free. Go to http://clamav.net for software and documentation.

• ClamAV integration with the MTA can be enabled for a user, a domain, or a channel.

• The ClamAV package provides a utility to regularly update virus-signatures. The utility is called freshclam. Refer to the ClamAV package documentation for further information.

• The libclamav.so library is included by default with Messaging Server 2006Q4 and above.

14.6.3 Deploying ClamAV

Perform the following steps to deploy ClamAV:

• Install and configure ClamAV. Refer to the ClamAV software documentation for installation and configuration information. See also 14.6.5 ClamAV Options.

• Load and configure the ClamAV client library. This involves specifying the client library, libclamav.soand a configuration file to the MTA (you must create this file). See 14.2.1 Loading and Configuring the Spam Filtering Software Client Library.

• Specify what messages to filter for spam. Messages can be filtered by user, domain, or channel. See 14.2.2 Specifying the Messages to Be Filtered.

• Specify what actions to take on virus messages. See 14.2.3 Specifying Actions to Perform on Spam Messages.

• Set miscellaneous filter configuration parameters as desired. See 14.6.5 ClamAV Options

To Jettison Virus– or Trojan Horse– Infected Email Using ClamAV

The following example jettisons all messages found to contain a virus or Trojan horse detected by ClamAV. The verdict string is not used.

1. Create the ClamAV configuration file.

   The name and location of this file is specified in Step 2. A good name is clamav.opt. This file contains the following lines:

   # more /opt/SUNWmsgsr/config/clamav.opt ! ClamAV Settings debug=1 host=127.0.0.1 port=3310 mode=1

   debug=1 turns on debugging in the ClamAV library.

   host and port specify the name of the system where clamd is running and the port on which clamd listens for incoming requests.

   mode=1 specifies that the ClamAV plug-in return the ClamAV result string as the verdict when a virus infected email is detected.

2. Modify the option.dat file.

   Add the following lines to the option.dat file:

   ! ClamAV settings spamfilter2_config_file=/opt/SUNWmsgsr/config/clamav.opt spamfilter2_library=/opt/SUNWmsgsr/lib/libclamav.so spamfilter2_string_action=data:,require ["jettison"]; jettison;

   spamfilter2_config_file specifies the ClamAV configuration file.

   spamfilter2_library specifies the ClamAV shared library.

   spamfilter2_string_action specifies the Sieve action to take for a virus infected email.

3. Specify the messages to be filtered.

   To filter all messages coming into the local message store, change the imta.cnf file by adding the destinationspamfilterXoptin virus keywords on the ims-ms channel:

   ! ! ims-ms ims-ms defragment subdirs 20 notices 1 7 14 21 28 backoff "pt5m" "pt10m" "pt30m" "pt1h" "pt2h" "pt4h" maxjobs 4 pool IMS_POOL fileinto $U+$S@$D destinationspamfilter2optin virus ims-ms-daemon

4. Recompile the configuration and restart the server.

   Only the MTA needs to be restarted. You do not need to execute stop-msg.

   # imsimta cnbuild # imsimta restart

5. Start the clamd daemon.

14.6.4 Testing ClamAV

To test ClamAV, first set debug=1 in the clamav.opt file. (You do not have to turn on the channel-specific master_debug or slave_debug in the imta.cnf.) Then send a file attachment to a test user which contains the EICAR virus string (http://www.eicar.org/anti_virus_test_file.htm). This string is designed to trigger virus scanners to recognize an email as virus-infected without having an actual virus attached:

X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*

Review the test logs. The msg-svr-base/data/log/tcp_local_slave.log* file should have lines similar to these:

10:39:00.85: ClamAV callout debugging enabled; 
config /opt/SUNWmsgsr/config/clamav.opt
10:39:00.85: IP address 127.0.0.1 specified 
10:39:00.85: Port 3310 selected 
10:39:00.85: Mode 1 selected 
10:39:00.85: Field "Virus-Test: " selected 
10:39:00.85: Verdict "" selected 
10:39:00.85: Initializing ClamAV message context
 ... 
10:39:00.85: Creating socket to connect to clamd server 
10:39:00.85: Binding clamd socket 
10:39:00.85: Connecting to clamd server 
10:39:00.85: Sending ClamAV STREAM request 
10:39:00.85: Retrieving ClamAV STREAM response 
10:39:00.85: STREAM response: PORT 2003 
10:39:00.85: Creating socket to connect to clamd server data port 
10:39:00.85: Binding clamd data socket 
10:39:00.85: Connecting to clamd server data port 
10:39:00.85: Sending ClamAV the message 
10:39:00.85: Closing ClamAV data connection 
10:39:00.85: Reading ClamAV result 
10:39:00.87: Result line: stream: Eicar-Test-Signature FOUND 
10:39:00.87: Scan result: Message is infected 
10:39:00.87: Verdict line: Virus-Test: True ; Eicar-Test-Signature 
10:39:00.87: Closing connection to ClamAV 
10:39:00.87: Mode 1 verdict of Virus-Test: True ; Eicar-Test-Signature 
10:39:00.87: Mode 1 verdict of Virus-Test: True ; Eicar-Test-Signature
 ... 
10:39:00.87: Freeing ClamAV message context  

If your log file does not contain lines similar to these, or if clamd is not running, the following error message is returned in your SMTP dialog after the last period (.) is sent to the SMTP server:

452 4.4.5 Error writing message temporaries - Error 
connecting to ClamAV server

14.6.5 ClamAV Options

The ClamAV option file is a typical messaging server-style option file consisting of lines of the form option=value. The one required option is HOST. It must be set to the name of the system where clamd is running. This option must be set even if clamd is running on the local host.

Further additional options are available for this options file are shown below.

Virus-Test: False 
Virus-Test: True ; Worm.Mydoom.I

14.7 Support for Sieve Extensions

In addition to the standard Sieve functions, Messaging Server provides support for a number extensions including addheader, addtag, spamtest and spamadjust. addheader and addtag are described in To Add a Header Containing SpamAssassin Score to Spam Messages and To Add the SpamAssassin Result String to the Subject Line.

These extensions gives administrators the ability to set different threshold values as well as the ability to set up white lists that override SpamAssassin verdicts. The two can even be combined to have different thresholds depending on who sent a particular message. spamadjust is a non-standard action. spamtest is described in ftp://ftp.isi.edu/in-notes/rfc3685.txt. Note that both can be used with Symantec Brightmail as well as SpamAssassin.

The internal separator character used to delimit multiple subject line tag additions for addtaghas been changed from space to vertical bar. This makes it possible to add a tag containing spaces, as some spam filters want to do. For example, previously, addtag "[Probable Spam]" was taken to mean addtag "[Probable"and add tag "spam]". Now it is seen as a single tag, "[Probable Spam]". This change prevents vertical bars from being used in tags.

spamtest can be used to compare SpamAssassin scores to specific values using the Sieve [RELATIONAL] extension with the "i;ascii-numeric" comparator. The SpamAssassin score is typically a real number, but spamtest forces the score into an integer value between 0 an 10 by first rounding the score to the nearest integer. Values below 0 are forced up to 0 while values above 10 are forced down to 10. Finally, the text string maintained by Messaging Server is appended to produce the test string that the spamtest test sees. :percent is supported by spamtest (see SIEVE Email Filtering: Spamtest and Virustest Extensions draft-ietf-sieve-spamtestbis-05). Note that spamtestcan be also be used with Brightmail.

spamadjust is used to adjust the current spam score. This action takes a single string argument which is scanned for a real numeric value. This value is used to adjust the current spam score. The entire string is also appended to the current score text string. In the example shown below, the string would be “undisclosed recipients.”

Multiple spamadjust actions are allowed; each one is added to the current score. Again, the score value always starts at 0. Signed numeric values are allowed, making it possible to lower the current score as well as raise it. There is no require clause for spamadjust; the spamtest extension should be listed instead.

For example, a possible use of spamadjust with a SpamAssassin MODE setting of 2 might be:

spamfilterX_string_action=data:,require ["spamtest"];spamadjust "$U";

A system-level Sieve filter could then modify the SpamAssassin score by checking for a particular type of header and, if found, adding 5 to the SpamAssassin score:

require "spamtest";  
if header :contains ["to", "cc", "bcc", "resent-to", "resent-cc", 
   "resent-bcc"]  ["<undisclosed recipients>", "undisclosed.recipients"]  
{spamadjust "+5 undisclosed recipients";}

Note that spamadjustcan be also be used with Brightmail.

Finally, a user-level Sieve script could test the resulting value, discard messages certain to be spam, file messages likely to be spam, and allow messages from addresses in the local domain to pass through:

require ["spamtest", "relational",  
"comparator-i;ascii-numeric", "fileinto"];                   
if anyof (address :matches "from" ["*@siroe.com", "*@*.siroe.com"]) 
    {keep;}                                                  
elsif spamtest :value "ge" :comparator "i;ascii-numeric" "8" 
    {discard;}                                               
elsif spamtest :value "ge" :comparator "i;ascii-numeric" "5" 
    {fileinfto "spam-likely";}                                
else                                                         
   {keep;}  

14.8 Using Milter

This section consists of the following subsections: 14.8 Using Milter

• 14.8.1 Milter Overview

• 14.8.2 Milter/Messaging Server Theory of Operations

• 14.8.3 Milter Requirements and Usage Considerations

• To Deploy Milter

14.8.1 Milter Overview

Milter is the short name for Sendmail Content Management API. It also refers to software written using this API. Milter provides a plug-in interface for third-party software to validate, modify, or block messages as they pass through the MTA. Milters can process a message's connection (IP) information, envelope protocol elements, message headers, and/or message body contents, and modify a message's recipients, headers, and body. Possible uses for filters include spam rejection, virus filtering, and content control. In general, Milter seeks to address site-wide filtering concerns in a scalable way. Originally designed for sendmail, Milters written for sendmail can now be used with Messaging Server, although some limitations may apply (see below). For more information on Milter refer to the Internet.

14.8.2 Milter/Messaging Server Theory of Operations

Milters control what action is to be performed on a message. Messaging Server controls what messages are to be acted upon by a Milter using the methods described in 14.2.2 Specifying the Messages to Be Filtered.

In sendmail, Milter consists of support code in sendmail itself and a separate libmilter library. Filter authors link their filters against libmilter to produce a server. Sendmail is then configured to connect to these Milter servers.

Messaging Server provides a library that emulates the sendmail side of the Milter interface. This allows Milters written for sendmail to be used with Messaging Server.

A few words of caution are in order here. The Milter protocol consists of a complex mixture of text and binary elements and is not well documented. Additionally, Milter semantics are tightly coupled with sendmail's way of processing messages. In particular, Milters can and usually do access a subset of the macros defined in the sendmail configuration. Messaging Server's Milter client library attempts to provide a reasonable set of sendmail macros, but it is entirely possible for a Milter to be written that depends on a specific aspect of sendmail configuration which is not currently implemented. The net result is that an arbitrary Milter pulled off the network may or may not work with this client library. If problems develop we'll try and resolve them, but we cannot possibly guarantee success with every Milter out there.

14.8.3 Milter Requirements and Usage Considerations

The Milter server can run on a separate system of its own, on the same system as Messaging Server, in a single system deployment, or, in a two-tier deployment, on the same system as the MTA. When LMTP is used between the MTA and the message store, the filtering must be invoked from the MTA, it can not be invoked from the message store. When SMTP is used between the MTA and the message store it can be invoked from either one, and it can run on either system or a third separate system.

Messaging Server supports connecting to multiple Milter servers. If you specify a domain name that translates to multiple IP addresses, the system will try all of them in order received from the DNS until one works. Some DNS servers support randomizing the order of addresses they return, so this provides a primitive load balancing/failover facility.

14.8.3.1 Supported Milter Message Modification Actions

The Milter interface currently supports the ability to add headers (SMFIF_ADDHDRS), change or delete headers (SMFIF_CHGHDRS), and quarantine messages (SMFIF_QUARANTINE). Changing the message body (SMFIF_CHGBODY), adding recipients (SMFIF_ADDRCPT) and deleting recipients (SMFIF_DELRCPT) are not supported at the present time.

14.8.3.2 Macros Provided by the Milter Interface

The following macros are currently defined by the Milter interface:

$j Text placed in the by clause of Received: header fields. In Messaging Server this is controlled by the RECEIVED_DOMAIN MTA option. If the option is not set, the official host on the local channel is used instead.

${client_addr} The IP address of the SMTP client, expressed as a dotted quad value. Only set when SMTP over TCP is being used.

$i Queue ID for the current message. Messaging Server generates a unique ID for each session; this ID is what appears in the $i macro.

${mail_addr} The MAIL FROM address for the current transaction.

${mail_host} The host part of the MAIL FROM address for the current transaction.

${rcpt_addr} A RCPT TO address for the current transaction.

${rcpt_host} The host part of the current RCPT TO address.

To Deploy Milter

Perform the following steps to deploy Milter.

1. Obtain and configure a Milter that will perform the actions you desire.

   Refer to specific Milter documentation for obtaining and configuring information.

2. Load and configure the Milter client library. (See 14.2.1 Loading and Configuring the Spam Filtering Software Client Library.)

   1. Specify the path to the client library, libmilter.so. Specify the path and name of the Milter configuration file.

      Example:

      spamfilter1_library=/opt/SUNWmsgsr/lib/libmilter.so spamfilter1_config_file=/opt/SUNWmsgsr/lib/milter.opt

   2. Create a Milter configuration file. with the desired options.

      The Milter option file consists of lines of the form option=value. The two required options are HOST and PORT. HOST must be set to the name of the system where the Milter server is running while PORT must be set to the port the Milter server is configured to listen on. Note that only TCP/IP connections are supported; UNIX domain sockets cannot be specified or used.

      Several additional options are also available in this options file:

      DEBUG (integer, default 0) — Enables or disables debug output from the Milter client library. The larger the value, the more debugging output will be produced. 0 produces no output. 1 provides basic debugging. 2 adds logging of TCP traffic. (Debug output from the Milter server is typically controlled by a setting on the command line used to start the server. Note that most Milters seem to only provide the ability to direct debug output to syslog.)

      TIMEOUT (integer, default 3600) — Specifies the timeout in hundredths of seconds for operations involving the Milter connection. Available in 6.3 and later versions.

      SOCKS_HOST (string, default "") — Specifies the name of an intermediate SOCKS server. If this option is specified the Milter connection is made through the specified SOCKS server and not directly.

      SOCKS_PORT (integer, default 1080) — Specifies the port the intermediate SOCKS server is running on.

      SOCKS_PASSWORD (string, default "") — Specifies a password to use in establishing the connection through the SOCKS server. Whether or not a username/password is required depends on the SOCKS server configuration.

3. Specify what messages to send to Milter.

   Messages can be filtered by user, domain, or channel. See 14.2.2 Specifying the Messages to Be Filtered.

4. Set the spamfilterX_string_action option in the option.dat file:

   spamfilterX_string_action=data:,$M

   This setting is used unconditionally, but must be in the MTA options file for Milter to work properly.

14.9 Cloudmark Anti-Abuse Client

Messaging Server also works with the Cloudmark Anti-Abuse Client. See Configuring Cloudmark with Sun Java System Messaging Server.

14.10 Other Anti-Spam and Denial-of-Service Technologies

Adding spam and virus filtering software to the system is the most effective way of reducing spam and viri from getting into user's mailboxes. However, Messaging Server provides a number of other techniques and methods to support spam filtering. These technologies are often used for purposes other than just spam filtering and so are spread throughout this book. Listed below are some sections describing anti-spam and denial-of-service technologies.

Anti-Spam Technologies:

• 14.10.1 Anti-Spam Technique: Delay Sending the SMTP Banner

• 12.12.6 Routing After Address Validation But Before Expansion

• Chapter 15, Handling Forged Email Using the Sender Policy Framework

• Chapter 18, Mail Filtering and Access Control

• 23.7 Configuring Client Access to POP, IMAP, and HTTP Services

• 12.4.2.6 DNS Domain Verification

• 12.5.9 Expansion of Multiple Addresses

• 18.14 To Create MTA-Wide Filters

• 18.7 Configuring SMTP Relay Blocking

Denial of Service Technologies:

• Chapter 19, Throttling Incoming Connections Using MeterMaid

• 12.9.2 Specifying Absolute Message Size Limits

• 18.3.6 To Limit Specified IP Address Connections to the MTA

• 27.4.1 Monitoring the Size of the Message Queues

• 27.4.3 Monitoring Inbound SMTP Connections

14.10.1 Anti-Spam Technique: Delay Sending the SMTP Banner

A useful spam-fighting strategy is to delay sending the SMTP banner for a brief time (half a second, say), then clear the input buffer, and finally send the banner. The reason this works is that many spam clients are not standards-compliant and starts spewing SMTP commands as soon as the connection is open, ignoring any responses the server sends. Spam clients that do this when this capability is enabled will lose the first few commands in the SMTP dialogue, rendering the remainder of the dialogue invalid.

This feature has now been implemented in Messaging Server. It can be enabled unconditionally by setting the BANNER_PURGE_DELAY SMTP channel option to the number of centiseconds to delay before purging and sending the banner. A value of 0 disabled both the delay and purge.

The PORT_ACCESS mapping can also be used to control this capability. Specifying $D in the template causes an additional argument to be read from the template result after the mandatory SMTP auth rulset and realm, and optional application information addition. This value must be an integer with the same semantics as the BANNER_PURGE_DELAY value. Note that any PORT_ACCESS mapping setting overrides the BANNER_PURGE_DELAY SMTP channel option.