Sun Java Communications Suite 5 Deployment Planning Guide

Chapter 14 Planning a Messaging Server Anti-Spam and Anti-Virus Strategy

Messaging Server provides many tools for dealing with unsolicited bulk email (UBE, or “spam”) and viruses. This chapter describes the various tools and strategies available for your use.

This chapter contains the following sections:

Anti-Spam and Anti-Virus Tools Overview

As more computers are connected to the Internet, and the ease of doing business online increases, the frequency of security incidents, including spam and viruses, continues to rise. You should plan your Messaging Server deployment to deal with these problems.

Mail traffic passing into, through, and out of Messaging Server can be separated into distinct channels according to various criteria. This criteria includes source and destination email addresses as well as source IP address or subnet. You can apply different processing characteristics to these different mail flows, or channels. Consequently, you can use different access controls, mail filters, processing priorities, and tools in different ways and combinations on these channels. For example, you can process mail originating from within your domain differently from mail originating from outside your deployment.

In addition to channel-based message flow classification, another useful classification is mailing list traffic. Traffic for a given mailing list can come into Messaging Server through a number of different channels and go back out through a number of different channels. When using mailing lists, you can find it helpful to think in terms of the list itself and not in terms of channels. Messaging Server recognizes this and enables many of the channel-specific spam fighting tools to also be applied in a mailing-list specific fashion.

The following summarizes the anti-spam and anti-virus tools you can use with Messaging Server:

Access Control. Rejects mail from known spam sources and enables control over who can send or receive email within the organization
Mailbox Filtering. Enables users to manage their own spam filters through a Web interface, controlling the nature of mail delivered to their mailboxes
Address Verification. Refuses mail with invalid originator addresses
Real-time Blackhole List. Refuses mail from recognized spam sources as identified by the Mail Abuse Protection System”s Real-time Blackhole List (MAPS RBL), a responsibly managed, dynamically updated list of known spam sources
Relay Blocking. Prevents abusers from using a mail system as a relay to send their spam to tens of thousands of recipients
Authentication Service. Enables password authentication in an SMTP server with the Simple Authentication and Security Layer (SASL) protocol
Sidelining. Silently sidelines or even deletes potential spam messages
Comprehensive Tracing. Uses reliable mechanisms for identifying a message’s source
Conversion Channel. Integrates with third-party anti-virus or anti-spam products
Milter. Provides a plug-in interface for third-party software to validate, modify, or block messages as they pass through the MTA.

You can use these tools individually or together. No one tool by itself will block all spam. However, taken together, these tools provide an effective means of combatting unauthorized use of your mail system. The following sections provide more details on these tools. For more information, see the Sun Java System Messaging Server 6.3 Administration Guide.

Rejecting Mail

Messaging Server has a general purpose mechanism that you can use to reject mail in accordance with a variety of criteria. This criteria includes the message source or destination email addresses, as well as source IP address. For example, you can use this mechanism to refuse mail from specific senders or entire domains (such as mail from spam@public.com). Should you have large lists of screening information, you can extend your lists with a database that stores the access criteria. While not UBE-related, this same access control mechanism is also suitable for maintaining a database of internal users who are or are not allowed to send mail out certain channels. For example, you can restrict on a per-user basis who can or cannot send or receive Internet mail.

See Access Controls for more information.

Mailbox Filtering

Messaging Server provides mail filters on a per-user, per-channel, and system-wide basis. Per-user channels can be managed from any web browser in Messenger Express. Using these filters, users can control what mail messages are delivered to their mailbox. For example, a user tired of “make money fast” UBE can specify that any message with such a subject be rejected. Mail filtering in Messaging Server is based on the Sieve filtering language (RFCs 3028 and 3685) developed by the Internet Engineering Task Force (IETF).

See Using Mailbox Filters for more information.

You can also implement content-based filtering or virus scanning through the use of third-party content filtering software, such as Brightmail and SpamAssassin. See Anti-Spam and Anti-Virus Considerations for more information.

Address Verification

UBE messages often use invalid originator addresses. The Messaging Server SMTP server can take advantage of this by reflecting messages with invalid originator addresses. If the originator's address does not correspond to a valid host name, as determined by a query to the DNS server, the message can be rejected. Note that a potential performance penalty can be incurred with such use of the DNS.

You enable address verification on a per-channel basis with the mailfromdnsverify channel keyword described in the Sun Java System Messaging Server 6.3 Administration Guide.

Real-time Blackhole List

The Mail Abuse Protection System’s Real-time Blackhole List (MAPS RBL) is a dynamically updated list of known UBE sources identified by source IP address. The Messaging Server SMTP server supports use of the MAPS RBL and can reject mail coming from sources identified by the MAPS RBL as originators of UBE. The MAPS RBL is a free service provided through the Internet DNS.

For more information, see:

http://mail-abuse.com

Use of the RBL by the Messaging Server SMTP server is enabled with the ENABLE_RBL option of the MTA Dispatcher.

Relay Blocking

A comprehensive UBE strategy should include both ways to prevent users from receiving UBE (access controls, mailbox filtering, address verification, RBL) as well as preventing users from unauthorized relay of mail from your system to other systems. This second method is called relay blocking. In its simplest form, relay blocking is achieved by enabling local users and systems to relay mail while rejecting relay attempts from non-local systems. Using IP addresses as the differentiator easily and securely makes this differentiation between local versus non-local. By default, Messaging Server enables relay blocking upon installation. See Configuring Anti-Relaying with Mapping Tables for more information.

Authentication Services

The Messaging Server SMTP server implements the Simple Authentication and Security Layer (SASL, RFC2222) protocol. SASL can be used with POP and IMAP clients to provide password-based access to your SMTP server. A typical usage for SASL is to permit mail relaying for external authenticated users. This solves the common problem posed by local users who use ISPs from home or while traveling. Such users, when connecting to your mail system, will have non-local IP addresses. Any relay blocking that takes into account only the source IP address will not permit these users to relay mail. This difficulty is overcome through the use of SASL, which enables these users to authenticate themselves. Once authenticated, the users are permitted to relay mail.

Sidelining Messages

The access control mechanisms discussed previously can also defer the processing of suspect messages for later, manual inspection. Or, rather than sideline, the mechanisms can change the destination address, thus routing the suspect mail to a specific mailbox or simply deleting it silently. This tactic is useful when UBE is being received from a known, fixed origin and outright rejection will only cause the abuser to change the point of origin. Similar features are available for Messaging Server mailing lists. Great care should exercised when silently deleting mail to ensure that valid senders are not affected.

Comprehensive Tracing

Messaging Server’s SMTP server discovers and records crucial origination information about every incoming mail message, including, for example, source IP address and the corresponding host name. All discovered information is recorded in the message’s trace fields (for example, the Received: header line) as well as in log files, if they are so configured. Availability of such reliable information is crucial in determining the source of UBE, which often has forged headers. Sites can use their own preferred reporting tools to access this information, which is stored as plain text.

Conversion Channel

The conversion channel is a very general purpose interface where you can invoke a script or another program to perform arbitrary body part processing of an email message. The conversion program hands off each MIME body part (not the entire message) to the program or script and can replace the body part with the output of the program or script. Conversion channels can be used to convert one file format to another (for example, text to PostScript), to convert one language to another, perform content filtering for company sensitive information, scan for viruses and replace them with something else.

Integration with Third-Party Products

Content-filtering software from third-party suppliers can be hooked in to your deployment through Messaging Server’s conversion channel. Channel keywords are used to enable mail filtering using anti-spam and anti-virus products, such as Brightmail or SpamAssassin. You can configure the MTA to filter for all messages or only those going from or to certain channels, or to set the granularity at a per-user level. A user can decide to use spam or virus filtering, or both. (SpamAssassin only filters for spam.)

An extensive Sieve support enables great flexibility to set the disposition of the message determined to be spam or virus. You can take the default action of discarding the virus and spam, or filing the spam into a special folder. But using Sieve, you can forward a copy of the message to some special account, add a custom header, or use the spamtest Sieve extension to take different action based on a rating returned by SpamAssassin.

Milter

Milter refers to the Sendmail Content Management API and also to software written using this API. Milter provides a plug-in interface for third-party software to validate, modify, or block messages as they pass through the MTA. In sendmail, milter consists of support code in sendmail itself and a separate milter library. Filter authors link their filters against this library to produce a server. Sendmail is then configured to connect to these milter servers. Messaging Server provides a library that emulates the sendmail side of the milter interface. Consequently, milters written for sendmail can also be used with Messaging Server. The milter server can run in a variety of configurations. It can run on a separate system of its own, on the same system as Messaging Server, in a single system deployment, or in a two-tier deployment. Messaging Server also supports connecting to multiple milter servers.

Anti-Spam and Anti-Virus Considerations

This section describes issues to keep in mind when planning your deployment to use anti-spam or anti-virus technologies.

Architecture Issues with Anti-Spam and Anti-Virus Deployments

The Messaging Server MTA can reside on the same system as the mail filtering system, such as Brightmail or SpamAssassin, or you can use separate systems. One of the advantages of separating the MTA from the mail filtering servers is that you can add more processing power for the filtering simply by adding more hardware and cloning the servers. While the system is capable and not overloaded, you can have the mail filtering server software collocated with the MTA.

In general, consider deploying a “farm” of Brightmail severs that the MTAs utilize to filter mail. You can configure MTAs to use a list of Brightmail server names, which essentially the MTAs will load balance on. (This load balancing functionality is provided by the Brightmail SDK.) The advantage of having the Brightmail server farm is that when you need more processing power, you can simply add more Brightmail servers.

Mail filtering products tend to be CPU-intensive. Creating an architecture that separates the MTA and the mail filtering products onto their own machines provides for better overall performance of the messaging deployment.

Note –

Because mail filtering servers tend to be CPU-intensive in nature, you could end up with an architecture consisting of more mail filtering systems than the MTA hosts they are filtering for.

In larger deployments, consider also creating inbound and outbound mail filtering pools of servers that are associated with the respective inbound and outbound MTA pools. You can also create a “swing” pool that can be utilized as either an inbound or outbound pool, in response to need in either area.

As with the rest of the deployment, you need to monitor the mail filtering tier. A threshold of 50 percent CPU utilization is a good rule of thumb to follow. Once this threshold has been met, you need to consider adding more capacity to the mail filtering tier.

Security Issues with Anti-Spam and Anti-Virus Deployments

When planning to deploy anti-spam or anti-virus technology, keep in mind that an incorrect deployment can defeat your security measures. Figure 14–1 shows an incorrect deployment of an anti-spam/anti-virus filter solution.

Figure 14–1 Incorrect Deployment of Anti-Spam/Virus Solution

This diagrams shows an incorrect deployment of an anti-spam/virus
solution.

Figure 14–2 shows a correct deployment of an anti-spam/virus filter solution.

Figure 14–2 Correct Deployment of Anti-Spam/Virus Solution

This diagram shows a correct deployment of an anti-spam/virus
solution.

The MTA performs certain functions well, including:

Rejecting messages as early as possible
Per-user configuration and policy
Email security and routing policy
Mail queue management

The anti-spam/virus filter is good at determining if an email is spam or has a virus, but is generally not nearly as good at doing the things expected of a good MTA. Thus, do not depend on an anti-spam/virus filter to do those things. Your deployment is more “correct” when the anti-spam/virus filter is well integrated with the MTA, which is the case with Messaging Server. Messaging Server spam filter plug-in support provides all the potential reasons to reject a message early and applies all reasons at the same time.

A robust MTA, such as Messaging Server's, contains security features (SSL/TLS, traffic partitioning by IP address, early address rejection to reduce denial-of-service attacks, connection throttling by IP address/domain, and so on), which are defeated when an anti-spam/virus filter is deployed in front. Furthermore, anti-spam/virus filters that communicate by using the SMTP protocol often do not follow the robustness requirements of SMTP and thus lose email when they shouldn't. A correct deployment should have the anti-spam/virus filter working in conjunction with a robust MTA.

Implementing an RBL

In general, implementing an RBL provides the most immediate benefit to reducing spam traffic. A good RBL implemented by your MTAs immediately reduces spam by a minimum of 10 percent. In some cases, this number could approach 50 percent.

You can use your RBL and Brightmail together. If Brightmail takes care of 95 out of 100 emails for a certain IP address within some amount of time you should add that IP address to your RBL. You can adjust the RBLs for Brightmail’s false positives when you do your Brightmail analysis. That makes the RBL much more proactive in handling a specific wave of spam.

Common Anti-Spam and Anti-Virus Deployment Scenarios

This section describes common deployment scenarios for Brightmail and SpamAssassin. See the Sun Java System Messaging Server 6.3 Administration Guide for more information.

Using Symantec Brightmail

There are several common deployment scenarios for Symantec Brightmail:

Processing incoming messages to the local message store (ims-ms channel)
Processing messages going out to the Internet (tcp-local channel)
Processing messages coming in from the Internet (tcp-local channel)
Processing messages going to a specific domain (per-domain option)
Processing messages going to specific users (per-user option)
Setting up Brightmail processing as a Class-of-Service Option

If Brightmail implements both spam and virus checking, MTA message throughput can be reduced by as much 50 percent. To keep up with MTA throughput, you might need two Brightmail servers for each MTA.

Using SpamAssassin

Messaging Server supports the use of SpamAssassin, a freeware mail filter used to identify spam. SpamAssassin consists of a library written in Perl and a set of applications and utilities that can be used to integrate SpamAssassin into messaging systems.

SpamAssassin calculates a score for every message. Scores are calculated by performing a series of tests on message header and body information. Each test either succeeds or fails, and the score is adjusted accordingly. Scores are real numbers and may be positive or negative. Scores that exceed a certain threshold (typically 5.0) are considered to be spam.

SpamAssassin is highly configurable. Tests can be added or removed at any time and the scores of existing tests can be adjusted. This is all done through various configuration files. Further information on SpamAssassin can be found on the SpamAssassin Web site:

http://www.spamassassin.org

The same mechanism used for connecting to the Brightmail spam and virus scanning library can be used to connect to the SpamAssassin spamd server.

Using Symantec AntiVirus Scan Engine (SAVSE)

Messaging Server supports the use of SAVSE. SAVSE is a TCP/IP server application and communications API that provides high-performance virus scanning. It is designed to protect traffic served through, or stored on, network infrastructure devices.

Developing an Anti-Spam and Anti-Virus Site Policy

When developing a policy for preventing spam and relaying, strike a balance between providing safety from spam and providing a site where emails are delivered in a timely fashion. The best policy is therefore to initially provide a core set of measures that do not take up too much processing time but trap the majority of spam. You can then define this core set of measures after stress testing the final architecture. Start with the initial measures below. Once you have deployed your system, monitor trapped and non-trapped spam to fine tune the system and replace or add new functions if required.

Use the following set of measures as a starting point for your site’s anti-spam and anti-virus policy:

Anti-relay should be provided by the ORIG_SEND_ACCESS settings. This is structured to enable only subscribers and partnership users access to deliver externally bound SMTP mail.
Use authentication services to validate roaming users. These users verify their identity before being allowed to route externally bound SMTP mail.
Implement subject line checking for common spam phrases using the system-wide mailbox filters.
Set a maximum number of recipients using the holdlimit keyword. This will have the effect of sidelining potential spam traffic. The initial value could be set at 50 recipients and should be monitored over a period of time to determine whether a higher or lower value is required.
Set up dummy accounts that are then manually used by the postmasters to encourage spam to these specific accounts to identify new spam sites.
A message in which a virus has been detected should not be returned to the original sender and should not be forwarded to the intended recipient. There is no value in this because most viruses generate their own mail with forged sender addresses. It has become very rare that such infected messages will have any useful content.
Send infected messages to an engine that harvests and catalogues information about the virus. You can then use such information to create threat reports for your system administrators about new virus and worm outbreaks.