This chapter provides an overview of security methods, describes common security threats, and outlines the steps in analyzing your security needs.
This chapter contains the following sections:
For detailed product security information, see Chapter 13, Planning Messaging Server Security, Chapter 17, Planning Calendar Server Security, and Chapter 22, Planning Instant Messaging Security.
You manage security for a Communications Suite deployment by taking a “defense in depth” approach. By individually securing the network, hardware platform, operating system, and applications themselves, you make each layer of the architecture secure. Security includes hardening each layer by closing unnecessary network ports and access mechanisms. You also minimize the number of installed software packages so that only those packages required by the system are available. Finally, you secure and insulate the layers from unintended access within the network.
You can implement a Messaging Server proxy server to augment data security. A proxy server placed on the firewall with the Messaging Server behind it prevents attacks on the information on the Messaging Server.
Calendar Server provides a number of security levels to protect users against eavesdropping, unsanctioned usage, or external attack. The basic level of security is through authentication. Calendar Server uses LDAP authentication by default, but also supports the use of an authentication plugin for cases where an alternate means of authentication is desired. Integration with Access Manager enables Calendar Server to take advantage of its single sign-on capability.
Instant Messaging ensures the integrity of communications through its multiple authentication mechanisms and secure SSL connections. Integration with Portal Server and Access Manager bring additional security features, services-based provisioning access policy, user management, and secure remote access.
To ensure a completely secure environment, the deployment needs a time server to synchronize the internal clocks of the hosts being secured.
Creating a security strategy is one of the most important steps in planning your deployment. Your strategy should meet your organization’s security needs and provide a secure messaging environment without being overbearing to your users.
In addition, your security strategy needs to be simple enough to administer. A complex security strategy can lead to mistakes that prevent users from accessing their mail, or it can allow users and unauthorized intruders to modify or retrieve information that you don’t want them to access.
The five steps to developing a security strategy, as listed in RFC 2196, the Site Security Handbook, include:
Identifying what you are trying to protect.
For example, your list might include hardware, software, data, people, documentation, network infrastructure, or your organization’s reputation.
Determining what you are trying to protect it from.
For example: unauthorized users, spammers, or denial of service attacks.
Estimating how likely threats are to your system.
If you are a large service provider, your chances of security threats could be greater than a small organization. In addition, the nature of your organization could provoke security threats.
Implementing measures that will protect your assets in a cost-effective manner.
For example, the extra overhead in setting up an SSL connection can put a performance burden on your Messaging deployment. In designing your security strategy, you need to balance security needs against server capacity.
Continuously reviewing your strategy and make improvements each time a weakness is found.
Conduct regular audits to verify the efficiency of your overall security policy. You can do this by examining log files and information recorded by the SNMP agents. For more information on SNMP, refer to the Sun Java System Messaging Server 6.3 Administration Guide.
Your security strategy should also plan for:
Limit physical access to important parts of your infrastructure. For example, place physical limits on routers, servers, wiring closets, server rooms, or data centers to prevent theft, tampering, or other misuse. Network and server security become a moot point if any unauthorized person can walk into your server room an unplug your routers.
Limiting access to important operating system accounts and data is also part of any security strategy. Protection is achieved through the authentication and access control mechanisms available in the operating system.
In addition, you should install the most recent operating environment security patches and set up procedures to update the patches once every few months and in response to security alerts from the vendor.
Reduce potential risk of security breaches in the operating environment by performing the following, often termed “system hardening:”
Minimize the size of the operating environment installation. When installing a Sun server in an environment that is exposed to the Internet, or any untrusted network, reduce the Solaris software installation to the minimum number of packages necessary to support the applications to be hosted. Achieving minimization in services, libraries, and applications helps increase security by reducing the number of subsystems that must be maintained.
The Solaris Security Toolkit provides a flexible and extensible mechanism to minimize, harden, and secure Solaris systems. The primary goal behind the development of this toolkit is to simplify and automate the process of securing Solaris systems. For more information see:
Track and monitor file system changes. Within systems that require inclusion of security, a file change control and audit tool is indispensable as it tracks changes in files and detects possible intrusion. You can use a product such as Tripwire for Servers, or Solaris Fingerprint Database (available from SunSolve Online).
The recommended deployment configuration, to support both horizontal scalability and service security, is to place the access layer of the architecture behind a firewall. In a two-tiered architecture, use two firewalls, creating a DMZ. This enables access to the information delivery elements, the calendar and messaging front ends, while protecting the main service elements on the internal network behind a second firewall. Such a configuration also enables the access layer and data layer elements to be scaled independently, accommodating traffic and storage elements.
Limiting access to your network is an important part of your security strategy. Normally, overall access to networks is limited through the use of firewalls. However, email must be made available outside your site. SMTP is one such service.
To secure your network, you should:
Turn off all operating system-provided services that listen on ports that you do not use.
Replace telnet with sshd, if possible.
Place your application servers behind a packet filter, which drops external packets with an internal source IP address. A packet filter forbids all connections from the outside except for those ports that you explicitly specify.
Messaging Server offers the following sets of security features:
Protecting Messaging Components in Your Deployment
With this set of options, you can secure your MTA relays, message stores, Messenger Express mail clients, and multiplexing services. In addition, you’ll learn about third-party spam filter options.
Planning User Authentication
These options enable you to determine how your users will authenticate to your mail servers, preventing unauthorized users from gaining access to your system.
Understanding Security Misconceptions
Using this set of options, you can perform user authentication and protect the message itself by using authenticated SMTP and certificates for digital signatures, encryption, and Secure Sockets Layer (SSL).
See Chapter 13, Planning Messaging Server Security for more information.
The Communications Suite product portfolio provides features that ensure the security and integrity of business communications. Communications Suite offers extensive “built-in” security features, such as:
Authentication
Message and session encryption
Virus and spam protection
Archiving and auditing of communications
End-user configurable privacy options
Communications Suite supports security standards such as SSL/TLS, S/MIME, and SAML. SSL/TLS enables all communication between clients and servers to take place inside an encrypted session. Through integration with Portal Server, additional authentication mechanisms are available out-of-the-box, as well as single sign-on capabilities across the applications.
There is no SSL support between the Communications Express application in the web server and the Calendar Server cshttpd daemon.
If you want to implement public key data security, you must select mail clients that support public key infrastructure and key choice. Communications Suite products are capable, out of the box, of participating in the transmission and storage of messages so encrypted. Secure/Multipurpose Internet Mail Extension (S/MIME) is available on the Communications Express Mail client. Communications Express Mail users who are set up to use S/MIME can exchange signed or encrypted messages with other users of Communications Express Mail, Microsoft Outlook Express, and Mozilla mail systems.
The Webmail client from previous versions of Messaging Server is not capable of generating or decoding encrypted messages.
A more commonly used mechanism for data security protects the data across the wire only (that is, from client to server) by using SSL encryption on the connections used to transmit data between various messaging agents. This solution is not as complete as public key encryption, but is far easier to implement and is supported by many more products and service providers.
What problem does using SSL from client to server solve? An organization assumes that it controls its own corporate network and that data transmitted on that network is safe from non-employees. Mail sent to anyone from outside the corporate network using the corporation’s infrastructure transmits the data over an encrypted connection to the corporation’s network. Likewise, all mail picked up by a corporate user from outside the corporate network will be transmitted over an encrypted connection. Thus, if the enterprise’s assumption about the safety of the internal network is true, and its employees use only sanctioned servers for transmission between themselves and other employees, mail between employees is safe from external attack.
What problem doesn’t this solution solve? First of all, this approach does not protect the data from unintended viewing by non-intended recipients of the data who have access to the organization’s internal network. Secondly, there is no protection offered for data being transmitted between employees and their external partners, customers, or suppliers. The data travels across the public Internet in a completely insecure fashion.
However, this problem can be remedied by configuring SSL encryption between MTA routers at both the enterprise’s and customer’s network. This type of solution requires setup for each private connection you want to use. In so doing, you add an important additional layer of security with customer or partner data being sent or received via email. Using MTAs and SSL, companies can save money by using the public Internet as the transport, but force the MTAs to use SSL for their partners. This solution does not take into account other network traffic to and from partners. Nevertheless, mail is usually a large proportion of the traffic, and because companies can pay based on data transmitted, using the public Internet is usually cheaper.
You can implement SSL connections between server and clients, for example, from Messaging Server, and to other servers in your deployment as well (Web Server, Calendar Server, Directory Server). If desired, you can use two different Certificate Authorities (CAs), one for the server and one for the client.
In such a scenario, you can use one CA to issue server certificates, and another CA to issue client certificates. If you want the client to accept the server’s certificate as genuine, you will need to load the CA certificate for the server into the client’s certificate DB. If you want the server to accept the client as genuine, you will need to load the CA certificate for the client into the server’s certificate DB.
This section describes common misconceptions that are counterproductive to the security needs of your deployment.
Hiding Product Names and Versions
At best, hiding product names and versions hinders casual attackers. At worst, it gives a false sense of security that might cause your administrators to become less diligent about tracking real security problems.
In fact, removing product information and version numbers makes it more difficult for the vendor support organization to validate software problems as being that of their software or that of other software.
Hackers have little reason to be selective, particularly if there is a known vulnerability in SMTP servers, where they may attempt to access any SMTP server.
Hiding names of Internal Machines
Hiding internal IP addresses and machine names will make it more difficult to:
Trace abuse or spam
Diagnose mail system configuration errors
Diagnose DNS configuration errors
A determined attacker will have no problem discovering the machine names and IP addresses of machines once they find a way to compromise a network.
Turning off EHLO on the SMTP Server
With EHLO, the remote SMTP client determines if you have a limit and stops trying to send a message that exceeds the limit as soon as it sees this response. But, if you have to use HELO (because EHLO is turned off), the sending SMTP server sends the entire message data, then finds out that the message has been rejected because the message size exceeds the limits. Consequently, you are left with wasted processing cycles and disk space.
Network Address Translation
If you use NAT to provide a type of firewall, you do not have an end-to-end connection between your systems. Instead, you have a third node which stands in the middle. This NAT system acts as a middleman, causing a potential security hole.
For more information on designing a secure Communications Suite deployment, review the Computer Emergency Response Team (CERT) Coordination Center site: