2 Planning Messaging Server Security

This chapter describes how to plan for and protect the various components of your Oracle Communications Messaging Server deployment.

Protecting Messaging Components in Your Deployment

This section describes how to secure components in your Messaging deployment.

Note:

With each component, you should use the chroot function to limit the number of available commands on each machine.

Protecting MTAs

Secure MTAs to protect processing resources and server availability. When messages are relayed from unauthorized users or large quantities of spam are delivered, response time is reduced, disk space is used up, and processing resources (which are reserved for end users) are consumed. Not only does spam waste server resources, it is also a nuisance for your end users.

Note:

Not only must you protect your deployment from external unauthorized users, but you might also have to protect your system from internal users as well.

Table 2-1 describes the most common threats to MTAs.

Table 2-1 Common MTA Security Threats

Threat	Description
UBE (Unsolicited Bulk Email) or spam	Refers to the practice of sending electronic junk mail to millions of users.
Unauthorized relaying	Uses another company's SMTP server to relay your email. Spammers often use this technique to cover their tracks. End-users might send complaints back to the sending relay, not to the spammer.
Mail bombs	Characterized by abusers who repeatedly send an identical message to a particular address. The goal is to exceed mailbox quotas with the message.
Email spoofing	Creates email that appears to have originated from one source when it was actually sent from another source.
Denial of service attacks	Prevents legitimate users of a service from using that service. For example, an attacker attempts to flood a network, thereby preventing legitimate network traffic.

This section on MTA relays describes the following security options you can use in your deployment:

Monitoring Your Security
Access Controls
Preventing Relaying From Outside Hosts
Conversion Channels and Third Party Filtering Tools
RBL Checking
Client Access Filters

Monitoring Your Security

Monitoring your server is an important part of your security strategy. To identify attacks on your system, you should monitor message queue size, CPU utilization, disk availability, and network utilization. Unusual growth in the message queue size or reduced server response time may identify some of these attacks on MTA relays. Also, investigate unusual system load patterns and unusual connections. Review logs on a daily basis for any unusual activity.

Access Controls

You can use access controls to reject messages from (or to) certain users at a system level. In addition, you can institute more complex restrictions of message traffic between certain users. Also, you might allow users to set up filters on their own incoming messages (including rejecting messages based on contents of the message headers).

If you want to control access with envelope-level controls, use mapping tables to filter mail. If you want to control access with header-based controls, or if users wish to implement their own personalized controls, use the more general mailbox filters approach with server-side rules.

Mapping Table Overview

You can control access to your mail services by configuring certain mapping tables. Many components of the MTA employ table lookup-oriented information. This type of table is used to transform, that is, map, an input string into an output string. Mapping tables are usually presented as two columns. The first (left-hand) column provides possible input strings against which to match (pattern), and the second (right-hand) column gives the resulting output string for which the input string is mapped (template).

Table 2-2 describes these mapping tables, which enable you to control who can or cannot send mail, receive mail, or both. See the Messaging Server System Administrator's Guide for more information.

Table 2-2 Access Control Mapping Tables

Mapping Table	Description
SEND_ACCESS	Used to block incoming connections based on envelope From: address, envelope To: address, source and destination channels. The To: address is checked after rewriting, alias expansion, and so on have been performed.
ORIG_SEND_ACCESS	Used to block incoming connections based on envelope From: address, envelope To: address, source and destination channels. The To: address is checked after rewriting but before alias expansion.
MAIL_ACCESS	Used to block incoming connections based on combined information found in SEND_ACCESS and PORT_ACCESS tables: that is, the channel and address information found in SEND_ACCESS combined with the IP address and port number information found in PORT_ACCESS.
ORIG_MAIL_ACCESS	Used to block incoming connections based on combined information found in ORIG_SEND_ACCESS and PORT_ACCESS tables: that is, the channel and address information found in ORIG_SEND_ACCESS combined with the IP address and port number information found in PORT_ACCESS.
FROM_ACCESS	Used to filter mail based on envelope From: addresses. Use this table if the To: address is irrelevant.
PORT_ACCESS	Used to block incoming connections based on IP number.

Figure 2-1 illustrates where mapping tables are activated in the mail acceptance process.

Figure 2-1 Mapping Tables and the Mail Acceptance Process

Description of ''Figure 2-1 Mapping Tables and the Mail Acceptance Process''

For all the network ports controlled by the MTA service dispatcher, a PORT_ACCESS rejection response, if warranted, takes place at the initial connection from a remote host. A FROM_ACCESS rejection occurs in response to the MAIL FROM: command, before the sending side can send the recipient information or the message data. A SEND_ACCESS or MAIL_ACCESS rejection occurs in response to a RCPT TO: command, before the sending side gets to send the message data. If an SMTP message is rejected, your Messaging Server never accepts or sees the message data, thus minimizing the overhead of performing such rejections. If multiple access control mapping tables exist, Messaging Server checks them all.

Note:

If the message is accepted, it can still be filtered by way of conversion channels and user defined filters.

Configuring Anti-Relaying with Mapping Tables

You can also use access control mappings to prevent people from relaying SMTP mail through your Messaging Server system. For example, someone might try to use your mail system to relay junk mail to thousands of mailboxes on your system or on other systems.

By default, Messaging Server prevents all SMTP relaying activity, including relaying by local POP and IMAP mail clients. If clients do not authenticate by using SMTP AUTH, as described in "Enabling Authenticated SMTP," and attempt to submit messages to external addresses via Messaging Server's SMTP server, their submission attempts are rejected. Thus, you will likely want to modify your configuration so that it recognizes your own internal systems and subnets from which relaying should always be accepted.

Preventing Relaying From Outside Hosts

To prevent hosts that reside outside your domain from relaying to other hosts outside your domain:

Split incoming mail into different channels. For example:
- IP addresses within your domain go to the tcp_intranet channel.
- Authenticated sessions go to the tcp_auth channel.
- All other mail is sent to the tcp_local channel.
Recognize and allow mail from your POP and IMAP clients by using an INTERNAL_IP mapping table, fully explained in the discussion on how to filter mail based on its source or header strings in the Messaging Server System Administrator's Guide.

Using Mailbox Filters

A filter consists of one or more conditional actions to apply to a message. Messaging Server filters are stored on the server and evaluated by the server. They are sometimes called server-side rules (SSR).

You can create channel-level filters and MTA-wide filters to prevent the delivery of unwanted mail. The server applies filters in the following priority. See the Messaging Server System Administrator's Guide for more information.

Per-user filters apply to messages destined for a particular user's mailbox. The filters reject unwanted messages, redirect mail, filter messages into mailbox folders, and so on. End users create these filters by using a client that supports the use of mail filters, such as Convergence.

If a personal mailbox filter explicitly accepts or rejects a message, then filter processing for that message finishes. A filter template generalizes a Sieve script by replacing hard-coded elements of the Sieve script with prompts and input fields. A Java servlet is used to parse the Sieve templates and generate the user interface in the browser. When an end user supplies values in the input fields, the servlet takes those values and saves them in a Sieve script in the user's directory profile entry. The prompts and input fields are presented to the end user through the client interface. If the recipient user had no mailbox filter, or if the user's mailbox filter did not explicitly apply to the message in question, Messaging Server next applies the channel-level filter.
Channel-level filters apply to each message enqueued to a channel. A typical use for this type of filter is to block messages going through a specific channel. To create a channel-level filter, you must write the filter using Sieve. For specific instructions on creating filters with Sieve, see the discussion on how to filter mail based on its source or header strings in the Messaging Server System Administrator's Guide. If the channel-level filter explicitly accepts or rejects a message, then filter processing for that message finishes. Otherwise, Messaging Server next applies the MTA-wide filter, if one exists.
MTA-wide filters apply to all messages enqueued to the MTA. A typical use for this type of filter is to block unsolicited bulk email or other unwanted messages regardless of the messages' destinations. To create an MTA-wide filter, you must write the filter using Sieve. For specific instructions on creating filters with Sieve, see the discussion on how to filter mail based on its source or header strings in the Messaging Server System Administrator's Guide. By default, each user has no mailbox filter. When a user accesses the Convergence interface to create one or more filters, then their filters are stored in the LDAP Directory.

Conversion Channels and Third Party Filtering Tools

The conversion channel performs body-part-by-body-part conversions on messages through the MTA. This processing can be done by any site-supplied programs or command procedures. The conversion channel can do such things as convert text or images from one format to another, scan for viruses, translate languages, and so forth. Various message types of the MTA traffic are selected for conversion, and specific processes and programs can be specified for each type of message body part. If you are looking to use the conversion channel with a virus scanning program, you can either disinfect, hold, or reject messages. A special conversion channel configuration is consulted to choose an appropriate conversion for each body part. For more information, see the discussion on how to use predefined channel definitions in the MTA in the Messaging Server System Administrator's Guide.

Note:

Using specialized processing like a conversion channel puts additional load on your system. Be sure to account for it when you plan your sizing strategy.

With the conversion channel, you can use third-party anti-spam and anti-virus software solutions. You can also use the MTA API to create a channel to invoke a remote scanning engine. For more information on the MTA API, see the Messaging Server MTA Developer's Reference available in the Messaging Server 8.0 documentation set.

In general, it is best that these third-party solutions are shielded from external sites and are only used on back-end or intermediate relays.

The Brightmail solution consists of the Brightmail server and real-time anti-spam and anti-virus (for service providers only) rule updates that are downloaded to your messaging servers. When the Brightmail Logistics and Operations Center (BLOC) receives spam from email probes, operators immediately create appropriate anti-spam rules. These rules are then downloaded to Brightmail customer machines. Similarly, the Symantec Security Response real-time virus rules are also sent from Brightmail. These rules are used by customer's Brightmail servers to catch spam and viruses.

Messaging Server also supports the use of SpamAssassin, a freeware mail filter used to identify spam. SpamAssassin calculates a score for every message. Scores are calculated by performing a series of tests on message header and body information. Each test either succeeds or fails, and the score is adjusted accordingly. Scores are real numbers and may be positive or negative. Scores that exceed a certain threshold are considered to be spam.

For more information on configuring Brightmail and SpamAssassin for Messaging Server, see the discussion on how to integrate and configure spam and virus filtering software in the Messaging Server System Administrator's Guide.

RBL Checking

The Mail Abuse Protection System's Real-time Blackhole List (MAPS RBL) is a list of hosts and networks that are known to be friendly or neutral to abusers who use these hosts and networks to either originate or relay spam, or to provide spam support services.

You can configure your MTAs to compare incoming connections against the MAPS RBL. You can also use DNS-based databases used to determine incoming SMTP connections that might send unsolicited bulk mail.

For more information, see the discussion on how to filter mail based on its source or header strings in the Messaging Server System Administrator's Guide.

Client Access Filters

Messaging Server supports sophisticated access control on a service-by-service basis for POP, IMAP, and HTTP. The Messaging Server access-control facility is a program that listens at the same port as the TCP daemon it serves. The access-control facility uses access filters to verify client identity and it gives the client access to the daemon if the client passes the filtering process.

If you are managing messaging services for a large enterprise or for a service provider, these capabilities can help you to exclude spammers and DNS spoofers from your system and improve the general security of your network.

As part of its processing, the Messaging Server TCP client access-control system performs (when necessary) the following analyses of the socket end-point addresses:

Reverse DNS lookups of both end points (to perform name-based access control)
Forward DNS lookups of both end points (to detect DNS spoofing)

The system compares this information against access-control statements called filters to decide whether to grant or deny access. For each service, separate sets of Allow filters and Deny filters control access. Allow filters explicitly grant access. Deny filters explicitly forbid access.

When a client requests access to a service, the access-control system compares the client's address or name information to each of that service's filters by using these criteria:

The search stops at the first match. Because Allow filters are processed before Deny filters, Allow filters take precedence.
Access is granted if the client information matches an Allow filter for that service.
Access is denied if the client information matches a Deny filter for that service.
If no match with any Allow or Deny filter occurs, access is granted. The exception is the case where there are Allow filters but no Deny filters, in which case lack of a match means that access is denied.

The filter syntax described here is flexible enough that you should be able to implement many different kinds of access-control policies in a simple and straightforward manner. You can use both Allow filters and Deny filters in any combination, even though you can probably implement most policies by using almost exclusively Allows or almost exclusively Denies.

Client access filters are particularly helpful if troublesome domains are a known quantity. While UBE filters must store and process every spam message, client access filters free Messaging Server from having to process any spammed messages. Because client access filters block mail from entire domains, this feature should be used with caution.

Note the following limitations to client access filters:

An SMTP client is required to log in before relaying a message.
Client access filters do not scale well for large deployments.

For more information on client access filters, see "Security and Access Control in Messaging Server."

Protecting the Message Store

The most important data in a messaging server is the user's mail in the Message Store. Note that the mail messages are stored as individual files, which are not encrypted. Consequently, access to the Message Store must be protected.

To secure the Message Store, restrict access to the machine where the store is installed. For information on passwords, see "Planning Messaging User Authentication."

Not only should you create password authentication to the store machine, you might also use tools like VPN access, ssh, or RBAC, which list valid users that are allowed to login to the machine.

In addition, a two-tiered architecture is recommended over a one-tiered architecture. Because the Message Store performs the most disk intensive work of any components in a messaging system, do not have filtering, virus scanning, and other disk-intensive security processes on the same machine. In a two-tiered architecture, you do not have to run UBE filters, anti-relay, and client access filters on the same machine as the Message Store, which can add load to your system. Instead, the MTAs handle that processing. In addition, user access to the store is limited through an MMP in a two-tiered deployment, potentially adding an extra security layer to the Message Store.

If you deploy a one-tiered architecture, be sure to account for the additional security processing and load (like SSL and virus scanning) that you will need. For more information, see the discussion on configuring Messaging Server to provide optimum performance, scalability, and reliability in the Messaging Server Installation and Configuration Guide.

For additional Message Store security processing, set disk quotas per user to limit disk usage. Also, use administrator alarms if free space thresholds are fast approaching their limits. Like the MTA, be sure to monitor the server state, disk space, and service response times. For more information, see the discussion on managing the Message Store in the Messaging Server System Administrator's Guide.

Protecting MMPs

Because the MMP serves as a proxy for the Message Store, it needs to protect access to end user data and guard against unauthorized access. User IDs and passwords provide basic authentication capabilities. In addition, you can use client access filters to limit user login to specific domains or IP address ranges. SMTP Authentication, or SMTP AUTH (RFC 2554) is the preferred method of providing SMTP relay server security. SMTP AUTH allows only authenticated users to send mail through the MTA. For more information, see "Enabling Authenticated SMTP."

Locate the MMP on a different machine (or under a different userID) in front of your POP or IMAP services. You can have front-end machines with just MMP and MTAs, and then have a physically secure network between those front-end machines, the mail stores, and the LDAP servers.

Special security considerations need to be given to Convergence access to the Message Store when your users are logging in from the Internet. In general, you want to make sure that the stores are separated from the outside world by a firewall. Like the MMP, the Webmail Server supports both unencrypted and encrypted (SSL) communication with mail clients.

Regular monitoring of log files can protect against unauthorized access.

Planning Messaging User Authentication

User authentication enables your users to log in through their mail clients to retrieve their mail messages. Methods for user authentication include:

Plain Text and Encrypted Password Login
Authentication with Simple Authentication and Security Layer (SASL)
Enabling Authenticated SMTP
Certificate-based Authentication with Secure Sockets Layer (SSL)

Plain Text and Encrypted Password Login

User IDs and passwords are stored in your LDAP directory. Password security criteria, such as minimum length, are determined by directory policy requirements. Password security criteria is not part of Messaging Server administration. Refer to the latest Directory Server documentation to understand directory server password policies:

http://docs.oracle.com/cd/E19656-01/index.html

An administrator can set a messaging configuration option to determine if plain passwords are allowed or if passwords must be encrypted. For more information, see the service.xxx.plaintextminciper (where xxx is HTTP, POP, or IMAP) option in the Messaging Server Reference. The RestrictPlainPasswords option provides the equivalent function for the MMP.

Both plain text and encrypted password login can be used with POP and IMAP user access protocols.

Authentication with Simple Authentication and Security Layer (SASL)

SASL (RFC 2222) provides additional authentication mechanisms for POP, IMAP, and SMTP user access protocols. Table 2-3 describes the Messaging Server SASL support for the user access protocols.

Table 2-3 SASL Authentication User Access Protocols Support Matrix

Protocol	Plain	Login	CRAM-MD5	Certificate	APOP
SMTP AUTH	Yes	Yes	Deprecated	Yes	No
POP	Yes	No	Deprecated	Yes	Deprecated
IMAP	Yes	No	Deprecated	Yes	No
HTTP	Yes	No	No	Yes	No

Note:

When using CRAM-MD5, passwords must be stored in plain text format in the LDAP directory server.
To use APOP, CRAM-MD5, or DIGEST-MD5, passwords must be stored in plain text format in the LDAP directory server.

If you use SASL, user name and passwords are not encrypted unless SSL is used for the session. (For more information on SSL, see "Encryption with SSL.") The SASL mechanisms, PLAIN and LOGIN, encode authentication information, but can be easily decoded if captured. Despite this limitation, SASL is useful because it can be combined with SMTP AUTH (described in "Enabling Authenticated SMTP") to allow only authenticated users to relay mail through your system. For example, legitimate users can authenticate to the SMTP server, and the SMTP server can then be configured to switch to a different channel. In this way, the message from an authenticated session can come from a different TCP channel than a user that did not authenticate. A message from a user in your internal network can also be switched to differentiate it from a message coming from other sources just based on the IP address of the incoming connection.

For more information on SASL, see "Security and Access Control in Messaging Server."

Enabling Authenticated SMTP

By default, the standard SMTP port (25) is for mail transfer only. Mail relay for submissions from external networks is disabled and authentication is disabled. By default, the standard SMTP submit port (587) is for mail submission and requires authenticated SMTP. As many mail user agents still use port 25 for submission by default, it might be useful to enable SMTP authentication on port 25 for those clients.

By default, users need not submit a password when they connect to the SMTP service of Messaging Server to send a message. You can, however, enable password login to SMTP in order to enable authenticated SMTP.

Authenticated SMTP (also referred to as SMTP AUTH) is an extension to the SMTP protocol. Authenticated SMTP allows clients to authenticate to the server. The authentication accompanies the message. The primary use of authenticated SMTP is to enable local users who are not in their office to submit mail without creating an open relay that others could abuse. The AUTH command is used by the client to authenticate to the server.

Authenticated SMTP provides security in sending messages with the SMTP protocol. To use authenticated SMTP, you do not need to deploy a certificate-based infrastructure. (Certificate authentication is described in "Certificate-based Authentication with Secure Sockets Layer (SSL)."

With authenticated SMTP, the client can indicate an authentication mechanism to the server and perform an authentication protocol exchange.

If you require SMTP AUTH for mail submission, turn on appropriate logging, so any mail abuse can be traced.

For more information on authenticated SMTP, see the conceptual description of the Messaging Server MTA in the Messaging Server System Administrator's Guide.

Certificate-based Authentication with Secure Sockets Layer (SSL)

Messaging Server uses the SSL protocol for encrypted communications and for certificate-based authentication of clients and servers. This section describes certificate-based SSL authentication. For information on SSL Encryptions, see "Encryption with SSL."

SSL is based on the concepts of public-key cryptography. Although TLS (Transport Layer Security) is functionally a superset of SSL, the names are used interchangeably.

At a high-level, a server which supports SSL needs to have a certificate, public key, private key, and security databases. This helps assure message authentication, privacy, and integrity.

Table 2-4 describes the SSL authentication support with each client access protocol. This table shows whether a secure session (STARTTLS) could be started up over an insecure channel and whether a separate secure channel (SSL on Separate Port) is provided.

Table 2-4 SSL Authentication Support Matrix

Protocol	STARTTLS	SSL on Separate Port
SMTP	Yes	Only for submission
POP	Yes	Yes
IMAP	Yes	Yes
POP over MMP	Yes	Yes
IMAP over MMP	Yes	Yes
SMTP over MMP	Yes	Yes
Webmail	No	Yes

The SMTP, POP, and IMAP protocols provide a way for the client and server to start communication without SSL, and then switch to it by using an equivalent STARTTLS command. The SMTP, POP, and IMAP servers can also be configured to use SSL on an alternate port, for clients which do not implement STARTTLS.

To authenticate with SSL, the mail client establishes an SSL session with the server and submits the user's certificate to the server. The server then evaluates if the submitted certificate is genuine. If the certificate is validated, the user is considered authenticated.

If you use SSL for authentication, you need to obtain a server certificate for your Messaging Server. The certificate identifies your server to clients and to other servers. Your server can also have any number of certificates of trusted Certificate Authorities (CAs) that it uses for client authentication.

Some protocols require use of the SASL EXTERNAL mechanism in conjunction with the SSL client certificate to move from un-authenticated to authenticated state.

For more information on SSL, see "Security and Access Control in Messaging Server."

Third-Party Authentication Server Support

This section contains the following topics:

Messaging Mutiplexor (MMP) Support
IMAP/POP/SMTP Support
Sample Code

Messaging Server provides support for third-party authentication servers by supporting a protocol designed to integrate third-party authentication services. This protocol is documented in the file authserver.txt, which is installed as part of the Messaging Server in the examples/tpauthsdk directory.

Support for third-party authentication servers addresses two primary problems:

If your computing infrastructure does not store passwords in LDAP, Messaging Server can be configured to query the authentication server you provide to verify passwords. This circumvents the need to replicate passwords from the third-party authentication service to the LDAP server used by Messaging Server for user information.

If you wish to use an authentication system that provides security or management capabilities not possible with traditional password authentication (such as Kerberos), Messaging Server can be configured to pass SASL mechanisms to the authentication server you provide for processing. While Messaging Server does not directly support Kerberos, it is now possible to write code that adds Kerberos capability to Messaging Server.

The examples/tpauthsdk directory also contains sample code for a third-party authentication server that can validate and modify authentication information provided by the Messaging Server.

Messaging Mutiplexor (MMP) Support

To enable third-party authentication in the MMP, the PreAuth configuration option must be enabled. For authentication methods other than PLAIN, you must specify the proxy authentication credentials (StoreAdmin and StoreAdminPass) and add the following configuration option to the MMP's ImapProxyAService.cfg and/or PopProxyAService.cfg file:

default:AuthenticationServer :56

For plain text logins, the MMP will first perform a normal user lookup in LDAP. Once the user is located, the MMP will connect to the host (localhost loopback) and port (56) specified in the AuthenticationServer option to authenticate the user. The MMP will pass the LDAP attributes inetUserStatus, mailUserStatus, uid and mailHost to the authentication server. You may also configure the MMP to look up additional LDAP attributes and pass them to the authentication server with the option:

default:AuthenticationLdapAttributes "attr1" "attr2" ...

Note that the authentication server should be running on the same server as the MMP and on a restricted port. The protocol used to communicate between the MMP and the third-party authentication server is not presently secured.

The MMP will advertise the additional SASL mechanisms provided by the authentication server via the standard server protocols (for example, through the IMAP capabilities). The authentication server can provide a SASL mechanism that the MMP implements natively. In this case, the authentication server mechanism will take precedence. To see a transcript of the communication between the MMP and authentication server process, add authserv to the default:debugkeys configuration setting for the MMP and set default:LogLevel to Debug. State transitions in the HULA authentication subsystem can also be logged via the hula debug key.

See the discussion about Configuring and Administering Multiplexor Services for details on configuring and administering MMP.

IMAP/POP/SMTP Support

To enable third-party authentication with IMAP/POP/SMTP, use the msconfig options auth.authenticationserver and auth.authenticationserver or the configutil options sasl.default.authenticationserver and sasl.default.authenticationldapattributes. These work as they do for the MMP except the authenticationattributes is a space-delimited list (quotes are not used). The local.debugkeys option provides functionality equivalent to the default:debugkeys option, but again as a space separated list.

To debug third-party authentication with store, set msconfig option base.debugkeys or configutil option local.debugkeys to include authserv (or authserv hula) and set msconfig option {imap/pop}.logfile.loglevel or configutil option logfile.loglevel to debug.

Sample Code

The third-party authentication sample code in the tpauth directory is largely suitable for a production environment (it uses a thread-pool model to handle a high volume of connections). However, the Messaging Server product team will not provide support for this sample code. The sample code is designed for use on a system which provides the standard Posix Threads API (for example, Oracle Solaris or Linux). The Makefile.sample is for use on Oracle Solaris.

Table 2-5 lists the contents of the tpauth directory.

Table 2-5 Contents of the tpauth Directory

File	Contents
README.txt	A link to this document.
authserver.txt	Third-Party Authentication Protocol Specification.
Makefile.sample	Use make -f Makefile.sample to build the sample code.
authserv.c	The core thread-pool protocol server implementation.
authserv.h	The API called by authserv.c to authenticate users.
sample.c	A very simple sample third-party authentication module using plain-text passwords. Third-parties may edit or replace this module to provide authentication services.
sample2.c	A simple CRAM-MD5 example demonstrating use of this interface for non-plain-text mechanisms.

Planning Message Encryption Strategies

This section describes encryption and privacy solutions. The following topics are covered:

Encryption with SSL
Signed and Encrypted S/MIME

Encryption with SSL

SSL functions as a protocol layer beneath the application layers of IMAP, HTTP, and SMTP. If transmission of messages between a Messaging Server and its clients and between the servers and other servers is encrypted, there is little chance for eavesdropping on the communications. If connecting clients and servers are authenticated, there is little chance for intruders to spoof them.

End-to-end encryption of message transmission requires the use of S/MIME. See the discussion on the basic configuration procedure to set up S/MIME for Convergence in the Convergence System Administrator's Guide and the discussion on administering S/MIME in the Messaging Server System Administrator's Guide.

Note:

The extra performance overhead in setting up an SSL connection can put a burden on the server. In designing your messaging installation and in analyzing performance, you need to balance security needs against server capacity.

If you use SSL for encryption, you can improve server performance by installing a hardware encryption accelerator. An encryption accelerator typically consists of a hardware board, installed permanently in your server machine, and a software driver.

The SSL connection process between client and server using HTTP/SSL (HTTPS) is as follows:

The client initiates contact using HTTPS. The client specifies which secret-key algorithms it can use.
The server sends its certificate for authentication and specifies which secret-key algorithm should be used. It will specify the strongest algorithm which it has in common with the client. If there is no match (for example, client is 40 bit only, server requires 128 bits), the connection will be refused. If the server has been configured to require client authentication, it will ask the client for its certificate at this point.
The client checks the validity of the server certificate to make sure that it has:
- Not expired
- A known signed Certification Authority
- A valid signature
- A host name on the certificate that matches the name of the server in the HTTPS request

SSL Ciphers

A cipher is the algorithm used to encrypt and decrypt data in the encryption process. Some ciphers are stronger than others, meaning that a message encrypted by a stronger cipher is more difficult for an unauthorized person to decrypt.

A cipher operates on data by applying a key to the data. Generally, the longer the key the cipher uses during encryption, the more difficult it is to decrypt the data without the proper decryption key.

When a client initiates an SSL connection with Messaging Server, the client lets the server know what ciphers and key lengths it prefers to use for encryption. In any encrypted communication, both parties must use the same ciphers. Because there are a number of cipher-and-key combinations in common use, a server should be flexible in its support for encryption. For more information on ciphers, see "Security and Access Control in Messaging Server."

Signed and Encrypted S/MIME

Secure/Multipurpose Internet Mail Extensions (S/MIME) provides a consistent way for email users to send and receive secure MIME data, using digital signatures for authentication, message integrity, and non-repudiation and encryption for privacy and data security. S/MIME version 3.1 (RFC 3851) is supported.

Several email clients support the S/MIME specification, including Microsoft Outlook and Mozilla mail.

You can deploy a secure mail solution by using Messaging Server and S/MIME. Convergence users who are set up to use S/MIME can exchange signed or encrypted messages with other users of Convergence, Microsoft Outlook, and Mozilla mail systems. A messaging proxy can provide an additional layer of security at the firewall to further protect information assets within Messaging Server.

For more information on S/MIME and Convergence, see the discussion on configuring and storing certificate information in Messaging Server and Directory Server and the discussion on the basic configuration procedure to set up S/MIME for Convergence in the Convergence System Administrator's Guide. For other clients that support S/MIME, see that client's documentation for information on S/MIME configuration.

Planning a Messaging Server Anti-spam and Anti-virus Strategy

Messaging Server provides many tools for dealing with unsolicited bulk email (UBE, or spam) and viruses. This information describes the various tools and strategies available for your use.

Anti-spam and Anti-virus Tools Overview

As more computers are connected to the Internet, and the ease of doing business online increases, the frequency of security incidents, including spam and viruses, continues to rise. You should plan your Messaging Server deployment to deal with these problems.

Mail traffic passing into, through, and out of Messaging Server can be separated into distinct channels according to various criteria. This criteria includes source and destination email addresses as well as source IP address or subnet. You can apply different processing characteristics to these different mail flows, or channels. Consequently, you can use different access controls, mail filters, processing priorities, and tools in different ways and combinations on these channels. For example, you can process mail originating from within your domain differently from mail originating from outside your deployment.

In addition to channel-based message flow classification, another useful classification is mailing list traffic. Traffic for a given mailing list can come into Messaging Server through a number of different channels and go back out through a number of different channels. When using mailing lists, you can find it helpful to think in terms of the list itself and not in terms of channels. Messaging Server recognizes this and enables many of the channel-specific spam fighting tools to also be applied in a mailing-list specific fashion.

The following summarizes the anti-spam and anti-virus tools you can use with Messaging Server:

"Access Control." Rejects mail from known spam sources and enables control over who can send or receive email within the organization.
"Mailbox Filtering." Enables users to manage their own spam filters through a Web interface, controlling the nature of mail delivered to their mailboxes.
"Address Verification." Refuses mail with invalid originator addresses.
"Real-time Blackhole List." Refuses mail from recognized spam sources as identified by the Mail Abuse Protection System's Real-time Blackhole List (MAPS RBL), a responsibly managed, dynamically updated list of known spam sources.
"Relay Blocking." Prevents abusers from using a mail system as a relay to send their spam to tens of thousands of recipients.
"Authentication Services." Enables password authentication in an SMTP server with the Simple Authentication and Security Layer (SASL) protocol.
"Sidelining Messages." Silently sidelines or even deletes potential spam messages.
"Comprehensive Tracing." Uses reliable mechanisms for identifying a message's source.
"Conversion Channel." Integrates with third-party anti-virus or anti-spam products.
"Milter." Provides a plug-in interface for third-party software to validate, modify, or block messages as they pass through the MTA.
"MeterMaid." Provides centralized metering and management of connections and transactions.
"memcached." Can provide the same functions as MeterMaid.

You can use these tools individually or together. No one tool by itself will block all spam. However, taken together, these tools provide an effective means of combating unauthorized use of your mail system. The following sections provide more details on these tools. For more information, see the Messaging Server System Administrator's Guide.

Access Control

Messaging Server has a general purpose mechanism that you can use to reject mail in accordance with a variety of criteria. This criteria includes the message source or destination email addresses, as well as source IP address. For example, you can use this mechanism to refuse mail from specific senders or entire domains (such as mail from spam@public.com). Should you have large lists of screening information, you can extend your lists with a database that stores the access criteria. While not UBE-related, this same access control mechanism is also suitable for maintaining a database of internal users who are or are not allowed to send mail out of certain channels. For example, you can restrict on a per-user basis who can or cannot send or receive Internet mail.

See "Access Controls" for more information.

Mailbox Filtering

Messaging Server provides mail filters on a per-user, per-channel, and system-wide basis. Per-user channels can be managed from any web browser in Convergence. Using these filters, users can control what mail messages are delivered to their mailbox. For example, a user tired of ”make money fast” UBE can specify that any message with such a subject be rejected. Mail filtering in Messaging Server is based on the Sieve filtering language (RFCs 3028 and 3685) developed by the Internet Engineering Task Force (IETF).

See "Using Mailbox Filters" for more information.

You can also implement content-based filtering or virus scanning through the use of third-party content filtering software, such as Cloudmark, Brightmail, and SpamAssassin.

See "Anti-spam and Anti-virus Considerations" for more information.

Address Verification

UBE messages often use invalid originator addresses. The Messaging Server SMTP server can take advantage of this by reflecting messages with invalid originator addresses. If the originator's address does not correspond to a valid host name, as determined by a query to the DNS server, the message can be rejected. Note that a potential performance penalty can be incurred with such use of the DNS.

You enable address verification on a per-channel basis with the mailfromdnsverify channel keyword described in the Messaging Server System Administrator's Guide.

Real-time Blackhole List

The Mail Abuse Protection System's Real-time Blackhole List (MAPS RBL) is a dynamically updated list of known UBE sources identified by source IP address. The Messaging Server SMTP server supports use of the MAPS RBL and can reject mail coming from sources identified by the MAPS RBL as originators of UBE. The MAPS RBL is a free service provided through the Internet DNS.

For more information, see:

http://mail-abuse.com

Use of the RBL by the Messaging Server SMTP server is enabled with the ENABLE_RBL option of the MTA Dispatcher.

Relay Blocking

A comprehensive UBE strategy should include ways to prevent users from receiving UBE access controls, mailbox filtering, address verification, and RBL, as well as preventing users from unauthorized relay of mail from your system to other systems. This second method is called relay blocking. In its simplest form, relay blocking is achieved by enabling local users and systems to relay mail while rejecting relay attempts from non-local systems. Using IP addresses as the differentiator easily and securely makes this differentiation between local versus non-local. By default, Messaging Server enables relay blocking upon installation.

See "Configuring Anti-Relaying with Mapping Tables" for more information.

Authentication Services

The Messaging Server SMTP server implements the Simple Authentication and Security Layer (SASL, RFC2222) protocol. SASL can be used with POP and IMAP clients to provide password-based access to your SMTP server. A typical usage for SASL is to permit mail relaying for external authenticated users. This solves the common problem posed by local users who use ISPs from home or while traveling. Such users, when connecting to your mail system, will have non-local IP addresses. Any relay blocking that takes into account only the source IP address will not permit these users to relay mail. This difficulty is overcome through the use of SASL, which enables these users to authenticate themselves. Once authenticated, the users are permitted to relay mail.

Sidelining Messages

The access control mechanisms discussed previously can also defer the processing of suspect messages for later, manual inspection. Or, rather than sideline, the mechanisms can change the destination address, thus routing the suspect mail to a specific mailbox or simply deleting it silently. This tactic is useful when UBE is being received from a known, fixed origin and outright rejection will only cause the abuser to change the point of origin. Similar features are available for Messaging Server mailing lists. Great care should be exercised when silently deleting mail to ensure that valid senders are not affected.

Comprehensive Tracing

Messaging Server's SMTP server discovers and records crucial origination information about every incoming mail message, including, for example, source IP address and the corresponding host name. All discovered information is recorded in the message's trace fields (for example, the Received: header line) as well as in log files, if they are so configured. Availability of such reliable information is crucial in determining the source of UBE, which often has forged headers. Sites can use their own preferred reporting tools to access this information, which is stored as plain text.

Conversion Channel

The conversion channel is a very general purpose interface where you can invoke a script or another program to perform arbitrary body part processing of an email message. The conversion program hands off each MIME body part (not the entire message) to the program or script and can replace the body part with the output of the program or script. Conversion channels can be used to convert one file format to another (for example, text to PostScript), to convert one language to another or perform content filtering for company sensitive information.

Integration with Third-party Products

Content-filtering software from third-party suppliers can be hooked in to your deployment through Messaging Server's conversion channel. Channel keywords are used to enable mail filtering using anti-spam and anti-virus products, such as Cloudmark, Brightmail, Proofpoint, or SpamAssassin. You can configure the MTA to filter for all messages or only those going from or to certain channels, or to set the granularity at a per-user level. A user can decide to use spam or virus filtering, or both. (SpamAssassin only filters for spam.)

An extensive Sieve support enables great flexibility to set the disposition of the message determined to be a virus or spam. You can take the default action of discarding the virus and spam, or filing the spam into a special folder. But using Sieve, you can forward a copy of the message to some special account, add a custom header, or use the spamtest Sieve extension to take a different action based on a rating returned by SpamAssassin.

Milter

Milter refers to the Sendmail Content Management API and also to software written using this API. Milter provides a plug-in interface for third-party software to validate, modify, or block messages as they pass through the MTA. In sendmail, Milter consists of support code in sendmail itself and a separate Milter library. Filter authors link their filters against this library to produce a server. Sendmail is then configured to connect to these Milter servers. Messaging Server provides a library that emulates the sendmail side of the Milter interface. Consequently, Milters written for sendmail can also be used with Messaging Server. The Milter server can run in a variety of configurations. It can run on a separate system of its own, on the same system as Messaging Server, in a single system deployment, or in a two-tier deployment. Messaging Server also supports connecting to multiple Milter servers.

MeterMaid

MeterMaid is a server that can provide centralized metering and management of connections and transactions. Functionally, MeterMaid can be used to limit how often a particular IP address can connect to the MTA. Limiting connections by particular IP addresses is useful for preventing excessive connections used in denial-of-service attacks.

memcached

memcached can provide the same functions as MeterMaid.

Anti-spam and Anti-virus Considerations

This section describes issues to keep in mind when planning your deployment to use anti-spam or anti-virus technologies.

Architecture Issues with Anti-spam and Anti-virus Deployments

The Messaging Server MTA can reside on the same system as the mail filtering system, such as Cloudmark, Brightmail, or SpamAssassin, or you can use separate systems. One of the advantages of separating the MTA from the mail filtering servers is that you can add more processing power for the filtering simply by adding more hardware and cloning the servers. While the system is capable and not overloaded, you can have the mail filtering server software collocated with the MTA.

In general, consider deploying a farm of Brightmail servers that the MTAs utilize to filter mail. You can configure MTAs to use a list of Brightmail server names, which essentially the MTAs will load balance on. (This load balancing functionality is provided by the Brightmail SDK.) The advantage of having the Brightmail server farm is that when you need more processing power, you can simply add more Brightmail servers.

Mail filtering products tend to be CPU-intensive. Creating an architecture that separates the MTA and the mail filtering products onto their own machines provides for better overall performance of the messaging deployment.

Note:

Because mail filtering servers tend to be CPU-intensive in nature, you could end up with an architecture consisting of more mail filtering systems than the MTA hosts they are filtering for.

In larger deployments, consider also creating inbound and outbound mail filtering pools of servers that are associated with the respective inbound and outbound MTA pools. You can also create a swing pool that can be utilized as either an inbound or outbound pool, in response to need in either area.

As with the rest of the deployment, you need to monitor the mail filtering tier. A threshold of 50 percent CPU utilization is a good rule of thumb to follow. Once this threshold has been met, you need to consider adding more capacity to the mail filtering tier.

Security Issues with Anti-spam and Anti-virus Deployments

When planning to deploy anti-spam or anti-virus technology, keep in mind that an incorrect deployment can defeat your security measures. Figure 2-2 shows an incorrect deployment of an anti-spam/anti-virus filter solution.

Figure 2-2 Incorrect Deployment of Anti-spam/virus Solution

Description of ''Figure 2-2 Incorrect Deployment of Anti-spam/virus Solution''

Figure 2-3 shows a correct deployment of an anti-spam/virus filter solution.

Figure 2-3 Correct Deployment of Anti-spam/virus Solution

Description of ''Figure 2-3 Correct Deployment of Anti-spam/virus Solution''

The MTA performs certain functions well, including:

Rejecting messages as early as possible
Per-user configuration and policy
Email security and routing policy
Mail queue management

The anti-spam/virus filter is good at determining if an email is spam or has a virus, but is generally not nearly as good at doing the things expected of a good MTA. Thus, do not depend on an anti-spam/virus filter to do those things. Your deployment is more correct when the anti-spam/virus filter is well integrated with the MTA, which is the case with Messaging Server. Messaging Server spam filter plug-in support provides all the potential reasons to reject a message early and applies all reasons at the same time.

A robust MTA, such as Messaging Server's, contains security features (SSL/TLS, traffic partitioning by IP address, early address rejection to reduce denial-of-service attacks, connection throttling by IP address/domain, and so on), which are defeated when an anti-spam/virus filter is deployed in front. Furthermore, anti-spam/virus filters that communicate by using the SMTP protocol often do not follow the robustness requirements of SMTP and thus lose email when they shouldn't. A correct deployment should have the anti-spam/virus filter working in conjunction with a robust MTA.

Implementing an RBL

In general, implementing an RBL provides the most immediate benefit to reducing spam traffic. A good RBL implemented by your MTAs immediately reduces spam by a minimum of 10 percent. In some cases, this number could approach 50 percent.

You can use your RBL and Brightmail together. If Brightmail takes care of 95 out of 100 emails for a certain IP address within some amount of time you should add that IP address to your RBL. You can adjust the RBLs for Brightmail's false positives when you do your Brightmail analysis. That makes the RBL much more proactive in handling a specific wave of spam.

Common Anti-spam and Anti-virus Deployment Scenarios

This section describes common deployment scenarios for Brightmail and SpamAssassin. See the Messaging Server System Administrator's Guide for more information.

Using Symantec Brightmail

There are several common deployment scenarios for Symantec Brightmail:

Processing incoming messages to the local message store (ims-ms channel)
Processing messages going out to the Internet (tcp-local channel)
Processing messages coming in from the Internet (tcp-local channel)
Processing messages going to a specific domain (per-domain option)
Processing messages going to specific users (per-user option)
Setting up Brightmail processing as a Class-of-Service Option

If Brightmail implements both spam and virus checking, MTA message throughput can be reduced by as much 50 percent. To keep up with MTA throughput, you might need two Brightmail servers for each MTA.

Using SpamAssassin

Messaging Server supports the use of SpamAssassin, a freeware mail filter used to identify spam. SpamAssassin consists of a library written in Perl and a set of applications and utilities that can be used to integrate SpamAssassin into messaging systems.

SpamAssassin calculates a score for every message. Scores are calculated by performing a series of tests on message header and body information. Each test either succeeds or fails, and the score is adjusted accordingly. Scores are real numbers and may be positive or negative. Scores that exceed a certain threshold (typically 5.0) are considered to be spam.

SpamAssassin is highly configurable. Tests can be added or removed at any time and the scores of existing tests can be adjusted. This is all done through various configuration files. Further information on SpamAssassin can be found on the SpamAssassin website:

http://spamassassin.apache.org/

The same mechanism used for connecting to the Brightmail spam and virus scanning library can be used to connect to the SpamAssassin spamd server.

Using Symantec AntiVirus Scan Engine (SAVSE)

Messaging Server supports the use of SAVSE. SAVSE is a TCP/IP server application and communications API that provides high-performance virus scanning. It is designed to protect traffic served through, or stored on, network infrastructure devices.

Developing an Anti-spam and Anti-virus Site Policy

When developing a policy for preventing spam and relaying, strike a balance between providing safety from spam and providing a site where emails are delivered in a timely fashion. The best policy is therefore to initially provide a core set of measures that do not take up too much processing time but trap the majority of spam. You can then define this core set of measures after stress testing the final architecture. Start with the initial measures below. Once you have deployed your system, monitor trapped and non-trapped spam to fine tune the system and replace or add new functions if required.

Use the following set of measures as a starting point for your site's anti-spam and anti-virus policy:

Anti-relay should be provided by the ORIG_SEND_ACCESS settings. This is structured to enable only subscribers and partnership users access to deliver externally bound SMTP mail.
Use authentication services to validate roaming users. These users verify their identity before being allowed to route externally bound SMTP mail.
Implement subject line checking for common spam phrases using the system-wide mailbox filters.
Set a maximum number of recipients using the holdlimit keyword. This will have the effect of sidelining potential spam traffic. The initial value could be set at 50 recipients and should be monitored over a period of time to determine whether a higher or lower value is required.
Set up dummy accounts that are then manually used by the postmasters to encourage spam to these specific accounts to identify new spam sites.
A message in which a virus has been detected should not be returned to the original sender and should not be forwarded to the intended recipient. There is no value in this because most viruses generate their own mail with forged sender addresses. It has become very rare that such infected messages will have any useful content.
Send infected messages to an engine that harvests and catalogues information about the virus. You can then use such information to create threat reports for your system administrators about new virus and worm outbreaks.