Sun Java Communications Suite 5 Deployment Planning Guide

Protecting MTAs

Secure MTAs to protect processing resources and server availability. When messages are relayed from unauthorized users or large quantities of spam are delivered, response time is reduced, disk space is used up, and processing resources, which are reserved for end users, are consumed. Not only does spam waste server resources, it is also a nuisance for your end users.


Note –

Not only must you protect your deployment from external unauthorized users, but you might also have to protect your system from internal users as well.


The following table describes the most common threats to MTAs.

Table 13–1 Common MTA Security Threats

Threat  

Description  

UBE (Unsolicited Bulk Email) or spam 

Refers to the practice of sending electronic junk mail to millions of users.

Unauthorized relaying

Uses another company’s SMTP server to relay your email. Spammers often use this technique to cover their tracks. End-users might send complaints back to the sending relay, not to the spammer. 

Mail bombs 

Characterized by abusers who repeatedly send an identical message to a particular address. The goal is to exceed mailbox quotas with the message. 

Email spoofing

Creates email that appears to have originated from one source when it actually was sent from another source. 

Denial of service attacks

Prevents legitimate users of a service from using that service. For example, an attacker attempts to flood a network, thereby preventing legitimate network traffic. 

This section on MTA relays describes security options you can use in your deployment:

Access Controls

You can use access controls to reject messages from (or to) certain users at a system level. In addition, you can institute more complex restrictions of message traffic between certain users. Also, you might allow users to set up filters on their own incoming messages (including rejecting messages based on contents of the message headers).

If you want to control access with envelope-level controls, use mapping tables to filter mail. If you want to control access with header-based controls, or if users wish to implement their own personalized controls, use the more general mailbox filters approach with server-side rules.

Mapping Table Overview

You can control access to your mail services by configuring certain mapping tables. Many components of the MTA employ table lookup-oriented information. This type of table is used to transform, that is, map, an input string into an output string. Mapping tables are usually presented as two columns. The first (left-hand) column provides possible input strings against which to match (pattern), and the second (right-hand) column gives the resulting output string for which the input string is mapped (template).

The following table describes these mapping tables, which enable you to control who can or cannot send mail, receive mail, or both. See the Sun Java System Messaging Server 6.3 Administration Guide for more information.

Table 13–2 Access Control Mapping Tables

Mapping Table  

Description  

SEND_ACCESS

Used to block incoming connections based on envelope From: address, envelope To: address, source and destination channels. The To: address is checked after rewriting, alias expansion, and so on, have been performed.

ORIG_SEND_ACCESS

Used to block incoming connections based on envelope From: address, envelope To: address, source and destination channels. The To: address is checked after rewriting but before alias expansion.

MAIL_ACCESS

Used to block incoming connections based on combined information found in SEND_ACCESS and PORT_ACCESS tables: that is, the channel and address information found in SEND_ACCESS combined with the IP address and port number information found in PORT_ACCESS.

ORIG_MAIL_ACCESS

Used to block incoming connections based on combined information found in ORIG_SEND_ACCESS and PORT_ACCESS tables: that is, the channel and address information found in ORIG_SEND_ACCESS combined with the IP address and port number information found in PORT_ACCESS.

FROM_ACCESS

Used to filter mail based on envelope From: addresses. Use this table if the To: address is irrelevant.

PORT_ACCESS

Used to block incoming connections based on IP number. 

Figure 13–1 illustrates where mapping tables are activated in the mail acceptance process.

Figure 13–1 Mapping Tables and the Mail Acceptance Process

This diagram shows how pre-SMTP accept filtering is activated
in the mail acceptance process.

For all the network ports controlled by the MTA service dispatcher, a PORT_ACCESS rejection response, if warranted, takes place at the initial connection from a remote host. A FROM_ACCESS rejection occurs in response to the MAIL FROM: command, before the sending side can send the recipient information or the message data. A SEND_ACCESS or MAIL_ACCESS rejection occurs in response to a RCPT TO: command, before the sending side gets to send the message data. If an SMTP message is rejected, your Messaging Server never accepts or sees the message data, thus minimizing the overhead of performing such rejections. If multiple access control mapping tables exist, Messaging Server checks them all.


Note –

If the message is accepted, it can still be filtered by way of conversion channels and user defined filters.


Configuring Anti-Relaying with Mapping Tables

You can also use access control mappings to prevent people from relaying SMTP mail through your Messaging Server system. For example, someone might try to use your mail system to relay junk mail to thousands of mailboxes on your system or on other systems.

By default, Messaging Server prevents all SMTP relaying activity, including relaying by local POP and IMAP mail clients. If clients do not authenticate by using SMTP AUTH, as described in Enabling Authenticated SMTP, and attempt to submit messages to external addresses via Messaging Server’s SMTP server, their submission attempts are rejected. Thus, you will likely want to modify your configuration so that it recognizes your own internal systems and subnets from which relaying should always be accepted.

ProcedureTo Prevent Relaying From Outside Hosts

To prevent hosts that reside outside your domain from relaying to other hosts outside your domain:

  1. Split incoming mail into different channels. For example:

    • IP addresses within your domain go to the tcp_internal channel.

    • Authenticated sessions go to the tcp_auth channel.

    • All other mail is sent to the tcp_local channel.

  2. Recognize and allow mail from your POP and IMAP clients by using an INTERNAL_IP mapping table, fully explained in the chapter on Mail Filtering and Access Control in the Sun Java System Messaging Server 6.3 Administration Guide.

Using Mailbox Filters

A filter consists of one or more conditional actions to apply to a message. Messaging Server filters are stored on the server and evaluated by the server. They are sometimes called server-side rules (SSR).

You can create channel-level filters and MTA-wide filters to prevent the delivery of unwanted mail. You can also create filter templates and make them available to end users by using Messenger Express. End users use the templates to build personal mailbox filters to prevent delivery of unwanted mail message to their mailboxes. The server applies filters in the following priority. See the Sun Java System Messaging Server 6.3 Administration Guide for more information.

  1. Per-user filters

    Per-user filters apply to messages destined for a particular user’s mailbox. You can create filter templates and make them available to end users by using the Messenger Express client. End users use the templates to build personal server filters to manipulate the delivery of mail messages to their mailboxes. The filers reject unwanted messages, redirect mail, filter messages into mailbox folders, and so on.

    If a personal mailbox filter explicitly accepts or rejects a message, then filter processing for that message finishes.

    A filter template generalizes a Sieve script by replacing “hard-coded” elements of the Sieve script with prompts and input fields. A Java servlet is used to parse the Sieve templates and generate the user interface in the browser. When an end user supplies values in the input fields, the servlet takes those values and saves them in a Sieve script in the user’s directory profile entry. The prompts and input fields are presented to the end user through the Messenger Express interface.

    If the recipient user had no mailbox filter, or if the user’s mailbox filter did not explicitly apply to the message in question, Messaging Server next applies the channel-level filter.

  2. Channel-level filter

    Channel-level filters apply to each message enqueued to a channel. A typical use for this type of filter is to block messages going through a specific channel.

    To create a channel-level filter, you must write the filter using Sieve. See Chapter 18, Mail Filtering and Access Control, in Sun Java System Messaging Server 6.3 Administration Guide for specific instructions on creating filters with Sieve.

    If the channel-level filter explicitly accepts or rejects a message, then filter processing for that message finishes. Otherwise, Messaging Server next applies the MTA-wide filter, if there is one.

  3. MTA-wide filter

    MTA-wide filters apply to all messages enqueued to the MTA. A typical use for this type of filter is to block unsolicited bulk email or other unwanted messages regardless of the messages’ destinations.

    To create an MTA-wide filter, you must write the filter using Sieve. See Chapter 18, Mail Filtering and Access Control, in Sun Java System Messaging Server 6.3 Administration Guide for specific instructions on creating filters with Sieve.

    By default, each user has no mailbox filter. When a user accesses Messenger Express interface to create one or more filters, then their filters are stored in the LDAP Directory.

Conversion Channels and Third Party Filtering Tools

The conversion channel performs body-part-by-body-part conversions on messages through the MTA. This processing can be done by any site-supplied programs or command procedures. The conversion channel can do such things such as convert text or images from one format to another, scan for viruses, translate languages, and so forth. Various message types of the MTA traffic are selected for conversion, and specific processes and programs can be specified for each type of message body part. If you are looking to use the conversion channel with a virus scanning program, you can either disinfect, hold, or reject messages. A special conversion channel configuration is consulted to choose an appropriate conversion for each body part. For more information, see Chapter 13, Using Predefined Channels, in Sun Java System Messaging Server 6.3 Administration Guide.


Note –

Using specialized processing like a conversion channel puts additional load on your system. Be sure to account for it when you plan your sizing strategy.


With the conversion channel, you can use third-party anti-spam and anti-virus software solutions. You can also use the MTA API to create a channel to invoke a remote scanning engine. For more information on the MTA API, see the Sun Java System Messaging Server 6.3 Administration Reference.

In general, it is best that these third-party solutions are shielded from external sites and are only used on back-end or intermediate relays.

The Brightmail solution consists of the Brightmail server and real-time anti-spam and anti-virus (for service providers only) rule updates that are downloaded to your messaging servers. When the Brightmail Logistics and Operations Center (BLOC) receives spam from email probes, operators immediately create appropriate anti-spam rules. These rules are then downloaded to Brightmail customer machines. Similarly, the Symantec Security Response real-time virus rules are also sent from Brightmail. These rules are used by customer’s Brightmail servers to catch spam and viruses.

Messaging Server also supports the use of SpamAssassin, a freeware mail filter used to identify spam. SpamAssassin calculates a score for every message. Scores are calculated by performing a series of tests on message header and body information. Each test either succeeds or fails, and the score is adjusted accordingly. Scores are real numbers and may be positive or negative. Scores that exceed a certain threshold are considered to be spam.

For more information on configuring Brightmail and SpamAssassin for Messaging Server, see Chapter 14, Integrating Spam and Virus Filtering Programs Into Messaging Server, in Sun Java System Messaging Server 6.3 Administration Guide.

RBL Checking

The Mail Abuse Protection System’s Real-time Blackhole List (MAPS RBL) is a list of host and networks that are known to be friendly or neutral to abusers who use these hosts and networks to either originate or relay spam, or to provide spam support services.

You can configure your MTAs to compare incoming connections against the MAPS RBL. You can also use DNS-based databases used to determine incoming SMTP connections that might send unsolicited bulk mail.

For more information, see Chapter 18, Mail Filtering and Access Control, in Sun Java System Messaging Server 6.3 Administration Guide.

Client Access Filters

Messaging Server supports sophisticated access control on a service-by-service basis for POP, IMAP, and HTTP. The Messaging Server access-control facility is a program that listens at the same port as the TCP daemon it serves. The access-control facility uses access filters to verify client identity, and it gives the client access to the daemon if the client passes the filtering process.

If you are managing messaging services for a large enterprise or for a service provider, these capabilities can help you to exclude spammers and DNS spoofers from your system and improve the general security of your network.

As part of its processing, the Messaging Server TCP client access-control system performs (when necessary) the following analyses of the socket end-point addresses:

The system compares this information against access-control statements called filters to decide whether to grant or deny access. For each service, separate sets of Allow filters and Deny filters control access. Allow filters explicitly grant access; Deny filters explicitly forbid access.

When a client requests access to a service, the access-control system compares the client’s address or name information to each of that service’s filters—in order—using these criteria:

  1. The search stops at the first match. Because Allow filters are processed before Deny filters, Allow filters take precedence.

  2. Access is granted if the client information matches an Allow filter for that service.

  3. Access is denied if the client information matches a Deny filter for that service.

  4. If no match with any Allow or Deny filter occurs, access is granted. The exception is the case where there are Allow filters but no Deny filters, in which case lack of a match means that access is denied.

The filter syntax described here is flexible enough that you should be able to implement many different kinds of access-control policies in a simple and straightforward manner. You can use both Allow filters and Deny filters in any combination, even though you can probably implement most policies by using almost exclusively Allows or almost exclusively Denies.

Client access filters are particularly helpful if troublesome domains are a known quantity. While UBE filters must store and process every spam message, client access filters free Messaging Server from having to process any spammed messages. Because client access filters block mail from entire domains, this feature should be used with caution.

Note the following limitations to client access filters:

For more information on client access filters, see Chapter 23, Configuring Security and Access Control, in Sun Java System Messaging Server 6.3 Administration Guide.

Monitoring Your Security Strategy

Monitoring your server is an important part of your security strategy. To identify attacks on your system, monitor message queue size, CPU utilization, disk availability, and network utilization. Unusual growth in the message queue size or reduced server response time may identify some of these attacks on MTA relays. Also, investigate unusual system load patterns and unusual connections. Review logs on a daily basis for any unusual activity.