23.5 Configuring Encryption and Certificate-Based Authentication

This section contains the following subsections:

• 23.5.1 Obtaining Certificates

• 23.5.2 To Enable SSL and Selecting Ciphers

• 23.5.3 To Set Up Certificate-Based Login

• 23.5.4 How to Optimize SSL Performance Using the SMTP Proxy

Messaging Server uses the Transport Layer Security (TLS) protocol, otherwise known as the Secure Sockets Layer (SSL) protocol, for encrypted communications and for certificate-based authentication of clients and servers. Messaging Server supports SSL versions 3.0 and 3.1. TLS is fully compatible with SSL and includes all necessary SSL functionality.

For background information on SSL, see Introduction to SSL in Managing Servers With iPlanet Console 5.0. SSL is based on the concepts of public-key cryptography, described in Introduction to Public-Key Cryptography in Managing Servers With iPlanet Console 5.0.).

If transmission of messages between a Messaging Server and its clients and between the server and other servers is encrypted, there is little chance for eavesdropping on the communications. If connecting clients are authenticated, there is little chance for intruders to impersonate (spoof) them.

SSL functions as a protocol layer beneath the application layers of IMAP4, HTTP, POP3, and SMTP. SMTP and SMTP/SSL use the same port; HTTP and HTTP/SSL require different ports; IMAP and IMAP/SSL, and POP and POP/SSL can use the same port or different ports. SSL acts at a specific stage of message communication, as shown in Figure 23–1, for both outgoing and incoming messages.

Figure 23–1 Encrypted Communications with Messaging Server

SSL provides hop-to-hop encryption, but the message is not encrypted on each intermediate server.

To enable encryption for outgoing messages, you must modify the channel definition to include the tls channel keywords, such as maytls, musttls, and so on. For more information, see the 12.4.8 Transport Layer Security Manual.

Keep in mind that the extra overhead in setting up an SSL connection can put a performance burden on the server. In designing your messaging installation and in analyzing performance, you may need to balance security needs against server capacity.

23.5.1 Obtaining Certificates

Whether you use SSL for encryption or for authentication, you need to obtain a server certificate for your Messaging Server. The certificate identifies your server to clients and to other servers. Another way of obtaining certificates is using msgcert command (described later in this section). Note that the old certutil command still works, but may be more complex to use and is not internationalized. For more information on certutil, see 23.5 Configuring Encryption and Certificate-Based Authentication and http://www.mozilla.org/projects/security/pki/nss/tools/certutil.html.

This section consists of the following subsections:

• 23.5.1.1 To Manage Internal and External Modules

• 23.5.1.2 Creating a Password File

• 23.5.1.3 Obtaining and Managing Certificates

• 23.5.1.4 About msgcert

• 23.5.1.5 Managing Certificates

• To Create a Messaging Server Certificate Database with a Default Self-signed Certificate

• To Manage Self-Signed Certificates

• 23.5.1.6 To Install Certificates of Trusted CAs

23.5.1.1 To Manage Internal and External Modules

A server certificate establishes the ownership and validity of a key pair, the numbers used to encrypt and decrypt data. Your server’s certificate and key pair represent your server’s identity. They are stored in a certificate database that can be either internal to the server or on an external, removable hardware card (smartcard).

Sun Java System servers access a key and certificate database using a module conforming to the Public-Key Cryptography System (PKCS) #11 API. The PKCS #11 module for a given hardware device is usually obtained from its supplier and must be installed into the Messaging Server before the Messaging Server can use that device. The pre-installed “Netscape Internal PKCS # 11 Module” supports a single internal software token that uses the certificate database that is internal to the server.

Setting up the server for a certificate involves creating a database for the certificate and its keys and installing a PKCS #11 module. If you do not use an external hardware token, you create an internal database on your server, and you use the internal, default module that is part of Messaging Server. If you do use an external token, you connect a hardware smartcard reader and install its PKCS #11 module.

In the following sections we will refer to the console or Directory Server Console. This refers to Directory Server prior to version 6. For version 6 or later, the graphical user interface is called the Directory Server Control Center. Refer to the latest Directory Server documentation (Sun Java System Directory Server Enterprise Edition 6.0 Administration Guide) for more information.

You can manage PKCS #11 modules, whether internal or external, through Console. To install a PKCS #11 module:

1. Connect a hardware card reader to the Messaging Server host machine and install drivers.

2. Use the modutil found in msg-svr-base/sbin to install the PKCS #11 module for the installed driver.

Installing Hardware Encryption Accelerators. If you use SSL for encryption, you may be able to improve server performance in encrypting and decrypting messages by installing a hardware encryption accelerator. An encryption accelerator typically consists of a hardware board, installed permanently in your server machine, plus a software driver. Messaging Server supports accelerator modules that follow the PKCS #11 API. (They are essentially hardware tokens that do not store their own keys; they use the internal database for that.) You install an accelerator by first installing the hardware and drivers as specified by the manufacturer, and then completing the installation—as with hardware certificate tokens—by installing the PKCS #11 module.

23.5.1.2 Creating a Password File

On most Sun Java System servers for which SSL is enabled, the administrator is prompted at startup to supply the password required to decrypt the key pair. On Messaging Server, however, to alleviate the inconvenience of having to enter the password multiple times (it is needed by at least three server processes), and to facilitate unattended server restarts, the password is read from a password file. Passwords themselves are generated when their certificate database is created using the msgcert generate_certdb command.

The password file is named sslpassword.conf and is in the directory msg-svr-base/config/. Entries in the file are individual lines with the format

moduleName:password

where moduleName is the name of the (internal or external) PKCS #11 module to be used, and password is the password that decrypts that module’s key pair. The password is stored as clear (unencrypted) text.

Messaging Server provides a default version of the password file, with the following single entry (for the internal module and default password):

Internal (Software) Token:netscape!

If you specify anything but the default password when you install an internal certificate, you need to edit the above line of the password file to reflect the password you specified. If you install an external module, you need to add a new line to the file, containing the module name and the password you specified for it.

Because the administrator is not prompted for the module password at server startup, it is especially important that you ensure proper administrator access control to the server and proper physical security of the server host machine and its backups.

23.5.1.3 Obtaining and Managing Certificates

Whether you use SSL for encryption or for authentication, you need to obtain a server certificate for your Messaging Server. The certificate identifies your server to clients and to other servers. The primary mechanism for obtaining and managing certificates is using the msgcert. However, if the Administration Server is installed, you can also use the Administration Console.

The rest of this section describes how to use msgcert.

23.5.1.4 About msgcert

msgcert allows you to generate a certificate request, add a certificate to the certificate database, list certificates in the database and so on. For detailed information enter the following at the command line:

msg-svr-base/sbin/msgcert --help

This is shown below.

# ./msgcert --help

Usage: msgcert SUBCMD [GLOBAL_OPTS] [SUBCMD_OPTS] [SUBCMD_OPERANDS]
Manages the Messaging Servers Certificate Database
The accepted values for SUBCMD are:

add-cert              Adds a certificate to the certificate database
add-selfsign-cert     Creates and adds a selfsign certificate to the 
                      certificate database
export-cert           Exports a certificate and its keys from the database
generate-certDB       Creates Messaging Server Databases cert8.db key3.db 
                      secmod.db and sslPassword
import-cert           Adds a new certificate and its keys to the cert database
import-selfsign-cert  Adds a new selfsign certificate and its keys to the 
                      cert database
list-certs            Lists all certificates in the Certificate database
remove-cert           Removes a certificate from the database
renew-cert            Renews a certificate
renew-selfsign-cert   Renews a selfsign certificate
request-cert          Generates a certificate request
show-cert             Displays a certificate

The accepted value for GLOBAL_OPTS is:-?, --help
                Displays SUBCMD help

NOTE: You must stop all the TLS or SSL-enabled servers before making any 
changes to the Certificate Database.

Each of the sub-commands shown above performs a specific certificate management function. Details about these sub-commands and their functions can be obtained by entering the following:

msgcert SUBCMD –help

The remainder of this section will describe some common certificate management procedures.

23.5.1.5 Managing Certificates

This section describes how to manage SSL certificates in Messaging Server To run SSL on Messaging Server, you must either use a self-signed certificate or a Public Key Infrastructure (PKI) solution which involves an external Certificate Authority (CA). For a PKI solution, you need a CA-signed server certificate which contains both a public and a private key. This certificate is specific to one Messaging Server. You also need a trusted CA certificate, which contains a public key. The trusted CA certificate ensures that all server certificates from your CA are trusted. This certificate is sometimes also called a CA root key or root certificate.

Configuring the Certificate Database Password

When managing certificates, you do not need to type a certificate password or specify the password file. You can simply pass the password as -W argument. Example:

echo "password22" > /tmp/certdbpwd
echo "password22" > /tmp/certdbpwd
# ./msgcert list-certs -W /tmp/certdbpwd

To Create a Messaging Server Certificate Database with a Default Self-signed Certificate

1. To create a Messaging Server Certificate Database run the following command:

   msgcert generate-certDB

   This reads the certificate database password from CERT_PW_FILE (Default: prompt for password)

2. You can view this certificate by using the command:

   msgcert show-cert Server-Cert

To Manage Self-Signed Certificates

If you are using certificates for test purposes, you can use self-signed certificates. In deployment configurations, you might prefer to use trusted Certificate Authority (CA) certificates. You can also use the Directory Server Admin Console to perform this task.

1. When you create the certificate database, a default self-signed certificate is automatically provided. If you want to use a self-signed certificate with non-default settings, use the msgcert add-selfsign-cert command. Example:

   msgcert add-selfsign-cert --name siroe --org comms --org-unit Messaging --city SantaClara --state ca --country us MySelfSigned-Cert

   A self-signed certificate is valid for three months.

2. When your self-signed certificate expires, renew the certificate with the command:

   msgcert renew-selfsign-cert cert_alias

23.5.1.6 To Install Certificates of Trusted CAs

Use ./msgcert add-cert to install the certificates of certificate authorities. A CA certificate validates the identity of the CA itself. The server uses these CA certificates in the process of authenticating clients and other servers.

If, for example, you set up your enterprise for certificate-based client authentication in addition to password-based authentication (see “Setting Up Certificate-Based Login” on page 157), you need to install the CA certificates of all CAs that are trusted to issue the certificates that your clients may present. These CAs may be internal to your organization or they may be external, representing commercial or governmental authorities or other enterprises. (For more details on the use of CA certificates for authentication, see Introduction to Public-Key Cryptography in in Managing Servers With iPlanet Console 5.0.)

When installed, Messaging Server initially contains CA certificates for several commercial CAs. If you need to add other commercial CAs or if your enterprise is developing its own CA for internal use (using Sun Java System Certificate Server), you need to obtain and install additional CA certificates.

The CA certificates automatically provided with Messaging Server are not initially marked as trusted for client certificates. You need to edit the trust settings if you want to trust client certificates issued by these CAs. For instructions, see 23.5.1 Obtaining Certificates.

The following procedure describes the process of requesting and installing CA-signed server and trusted CA certificates for use with Messaging Server.

To Request a CA-Signed Server Certificate

You can also use the Directory Server Admin Console to perform this task.

1. Generate a CA--signed server certificate request.

   msgcert request-cert [-W CERT_PW_FILE] {-S DN|--name NAME [--org ORG] [--org-unit ORG-UNIT] [--city CITY] [--state STATE] [--country COUNTRY] } [-F FORMAT] [-o OUTPUT_FILE]

   Below is an example of a request for a CA-signed server certificate. It returns the certificate in binary format:

   ./msgcert request-cert --name aqua --org siroe --org-unit Messaging -o my_ca_signed_request_cert

   To return the certificate in ASCII format use the command as follows:

   ./msgcert request-cert --name aqua --org siroe --org-unit Messaging -F ascii -o my_casigned_request_cert

   Certificate Authorities usually require all of the attributes that are shown in this example in order to completely identify the server. For a description of each attribute, enter ./msgcert request-cert --help. When you request a certificate by using msgcert request-cert, the resulting certificate request is a binary certificate request unless you specify ASCII as output format. If you specify ASCII, the resulting certificate request is a PKCS #10 certificate request in PEM format. PEM is the Privacy Enhanced Mail format specified by RFCs 1421 through 1424 and used to represent a base64-encoded certificate request in US-ASCII characters. The content of the request look similar to the following example:

   -----BEGIN NEW CERTIFICATE REQUEST----- MIIBdTCB3wIBADA2MRIwEAYDVQQLEwlNZXNzYWdpbmcxDjAMBgNVBAoTBXNpcm9l MRAwDgYDVQQDEwdhcXVhdGljMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDt KEh5Fnj/h9GEu18Da6DkJpcNShkwxanjnKs2883ZoUV5Sp4pN7U6Vfbh0414WXZh D26m3t81q9b9h47Klkf0pW1X3BB6LOjGOHSt2VoNBI8n3hJ6XiN2zYbrlLTgdKuo y0YrSG/kHFnqKghikag9O/Ox+cwD+mpjl2QnsPZgswIDAQABoAAwDQYJKoZIhvcN AQEEBQADgYEArqgWQIwNZDC2d3EZawI23Wj9o6Pyvu9J1rkb+NYgIEnNp9jugxqX F326N0ABLdHXXNX/2ZvC5TKOgS4RidTBM89N9xJvokmvRGfc+1x80uxy474YdNlZ s+nP8AYo9dW9mrLOammozx9HLPSVYNFp4FxekgV2n8QG7WC5rkN5bCE= -----END NEW CERTIFICATE REQUEST-----

2. Transmit the certificate request to your Certificate Authority, according to its procedures.

   The process for obtaining your Certificate Authority certificate differs depending on the certificate authority you use. Some commercial CAs provide a website that allows you to automatically download the certificate. Other CAs will email it to you upon request.

   After you have sent your request, you must wait for the CA to respond with your certificate. Response time for your request varies. For example, if your CA is internal to your company, the CA might only take a day or two to respond to your request. If your selected CA is external to your company, the CA could take several weeks to respond to your request.

3. Save the certificate you receive back from the Certificate Authority.

   You should back up your certificates in a safe location. If you ever lose the certificates, you can reinstall them by using your backup file. You can save them in text files. The PKCS #11 certificate in PEM format looks similar to the following example.

   -----BEGIN CERTIFICATE----- MIICjCCAZugAwIBAgICCEEwDQYJKoZIhKqvcNAQFBQAwfDELMAkGA1UEBhMCVVMx IzAhBgNVBAoGlBhbG9a2FWaWxsZGwSBXaWRnZXRzLCBJbmMuMR0wGwYDVQQLExRX aWRnZXQgTW3FrZXJzICdSJyBVczEpMCcGAx1UEAxgVGVzdCBUXN0IFRlc3QgVGVz dCBUZXN0IFlc3QgQ0EswHhcNOTgwMzEyMDIzMzUWhcNOTgwMzI2MDIzMpzU3WjBP MQswCYDDVQQGEwJVUzEoMCYGA1UEChMfTmV0c2NhcGUgRGlyZN0b3J5VIFB1Ymxp Y2F0aW9uczEWMB4QGA1UEAxMNZHVgh49dq2tLNvbjTBaMA0GCSqGSIb3DQEBAQUA A0kAMEYkCQCksMR/aLGdfp4m0OiGgijG5KgOsyRNvwGYW7kfW+8mmijDtZaRjYNj jcgpF3VnlbxbclX9LVjjNLC5737XZdAgEDozYwpNDARBglghkgBhvhCEAQEEBAMC APAwHkwYDVR0jBBgwFAU67URjwCaGqZHUpSpdLxlzwJKiMwDQYJKoZIhQvcNAQEF BQADgYEAJ+BfVem3vBOPBveNdLGfjlb9hucgmaMcQa9FA/db8qimKT/ue9UGOJqL bwbMKBBopsDn56p2yV3PLIsBgrcuSoBCuFFnxBnqSiTS7YiYgCWqWaUA0ExJFmD6 6hBLseqkSWulk+hXHN7L/NrViO+7zNtKcaZLlFPf7d7j2MgX4Bo= -----END CERTIFICATE-----

To Add the CA-Signed Server Certificate and Trusted CA Certificate

You can also use the Directory Server Admin Console to perform this task.

1. Add the CA--signed server certificate using the following command:

   msgcert add-cert cert_alias cert_file

   Where cert_alias is a name which you give to identify your certificate, and cert_file is the text file containing the PKCS #11 certificate in PEM format.

   For example, to install a CA-signed server certificate, you might use a command similar to:

   msgcert add-cert /my_cert/server-cert-file

   The certificate is now installed, but is not yet trusted. To trust the CA-signed server certificate, you must install the Certificate Authority certificate.

2. Add the trusted Certificate Authority certificate using the following command:

   msgcert add-cert -C cert_alias cert_file

   The -C option indicates that the certificate is a trusted Certificate Authority certificate.

   For example, to install a trusted certificate from a Certificate Authority, you might use the command:

   msgcert add-cert -C CA-cert /my_cert/ca-cert-file

3. Optionally, use the following command to verify your installed certificates:

   To list all server certificates, showing information such as alias and validity dates:

   msgcert list-certs

   The Messaging server will have a default certificate called Server-Cert when generated with ./msgcert generate-CertDB . The text Same as issuer indicates that the default certificate is a self-signed server certificate. For example:

   # ./msgcert list-certs Enter the certificate database password: Alias Valid from Expires on Self- Issued by Issued to signed ------------ ---------------- --------------- ------ --------------------- -------------- SelfSignedCrt 2006/07/28 12:58 2006/10/28 12:58 y CN=SFO,L=SC,ST=ca,C=us Same as issuer Server-Cert 2006/07/28 07:47 2006/10/28 07:47 y CN=perseids Same as issuer 2 certificates found

   To list trusted CA certificates:

   msgcert list-certs -C

   To view the details of a certificate, including the certificate expiry date:

   msgcert show-cert cert_alias

   For example, to show a self-signed certificate:

   # ./msgcert show-cert MySelfSigned-Cert Enter the certificate database password: Certificate: Data: Version: 3 (0x2) Serial Number: 00:83:35:37:94 Signature Algorithm: PKCS #1 MD5 With RSA Encryption Issuer: "CN=siroe,O=comms,OU=Messaging,L=SantaClara,ST=ca,C=us" Validity: Not Before: Fri Jul 28 19:58:31 2006 Not After : Sat Oct 28 19:58:31 2006 Subject: "CN=siroe,O=comms,OU=Messaging,L=SantaClara,ST=ca,C=us" Subject Public Key Info: Public Key Algorithm: PKCS #1 RSA Encryption RSA Public Key: Modulus: aa:9d:3d:23:b2:59:39:f3:77:c8:69:7f:b0:d1:ac:d2: 4e:81:c8:51:0f:27:6f:a1:21:4b:a9:27:46:d7:0f:b4: c8:44:86:32:5e:4f:2f:1c:2f:a9:b8:a3:49:b5:b8:ab: 51:a8:a5:ba:1c:e8:90:7d:46:67:f9:a7:44:c5:1d:24: e6:bd:e8:8f:07:b4:5a:68:41:b1:19:f2:ea:98:ba:25: 55:b8:ba:9c:af:bb:43:c3:c0:8f:14:a7:4c:2b:50:b4: ac:df:b5:cd:68:de:a6:14:9d:68:77:d3:8b:7f:de:c0: 5d:35:d7:55:8d:b5:c3:14:2a:60:a9:bf:de:96:90:a9 Exponent: 65537 (0x10001) Signature Algorithm: PKCS #1 MD5 With RSA Encryption Signature: 15:86:f1:cc:85:c9:08:0f:ff:d3:56:d8:e2:c8:ea:3c: 8e:45:36:be:8b:b0:7d:2f:e9:cd:e3:b4:ad:8c:70:59: c8:a5:14:da:9c:fa:7f:70:86:64:34:0b:21:ae:c4:28: d2:f5:94:5c:a6:78:0f:d9:fd:fc:c5:5e:37:49:25:a9: bc:12:59:cb:fb:4e:e9:d4:8a:8d:3d:41:12:ae:f1:7f: 8d:d3:10:ac:fb:33:51:5d:0c:1b:dc:23:5f:95:d5:6d: c6:1d:e5:ed:13:8b:16:41:89:5b:4d:de:c0:c7:56:a2: 48:82:38:32:5a:99:d5:21:20:c5:0d:5c:ea:0c:84:aa Fingerprint (MD5): EF:76:A3:6C:09:4E:BC:6B:87:76:A3:35:70:1F:B2:C4 Fingerprint (SHA1): BB:1C:20:4B:79:3A:F1:49:F0:83:FB:CC:9C:56:10:D3:06:97:AA:07 Certificate Trust Flags: SSL Flags: Valid CA Trusted CA User Trusted Client CA Email Flags: User Object Signing Flags: User

Renewing an Expired CA-Signed Server Certificate

When your CA-signed server certificate (public and private key) expires, you can renew it by using the following procedure. You can also use the Directory Server Admin Console to perform this task.

1. Obtain an updated CA-signed server certificate from your Certificate Authority.

2. Once you receive the updated certificate, install the certificate.

   msgcert renew-cert cert_alias cert_file

To Export and Import a CA-Signed Server Certificate

In some cases you might want to export a certificate so that you can later import the certificate, for example, to another host. You can also use the Directory Server Admin Console to perform this task.

1. Export the certificate.

   msgcert export-cert [-o OUTPUT_FILE] CERT_ALIAS

   For example:

   $ ./msgcert export-cert -o /tmp/first-certificate "First Certificate" $./msgcert export-cert -o /tmp/first-server-certificate Server-Cert Choose the PKCS#12 file password: Confirm the PKCS#12 file password: $ls /tmp first-server-certificate /tmp/first-certificate

2. Import the certificate.

   $ msgcert import-cert CERT_FILE

   For example, to import the certificate

   $ msgcert import-cert /tmp/first-server-certificate Enter the PKCS#12 file password: $

23.5.2 To Enable SSL and Selecting Ciphers

You can use Console to enable SSL and to select the set of encryption ciphers that Messaging Server can use in its encrypted communications with clients. You can also install the SSL certificate using the msgcert utility and run the appropriate configutil or edit the appropriate configuration files necessary to enable SSL for that particular service.

23.5.2.1 About Ciphers

A cipher is the algorithm used to encrypt and decrypt data in the encryption process. Some ciphers are stronger than others, meaning that a message they have scrambled is more difficult for an unauthorized person to unscramble.

A cipher operates on data by applying a key—a long number—to the data. Generally, the longer the key the cipher uses during encryption, the harder it is to decrypt the data without the proper decryption key.

When a client initiates an SSL connection with a Messaging Server, the client lets the server know what ciphers and key lengths it prefers to use for encryption. In any encrypted communication, both parties must use the same ciphers. Because there are a number of cipher-and-key combinations in common use, a server should be flexible in its support for encryption. Messaging Server can support up to 6 combinations of cipher and key length.

Table 23–2 lists the ciphers that Messaging Server supports for use with SSL 3.0. The table summarizes information that is available in more detail in the Introduction to SSL section of Managing Servers with iPlanet Console.

Unless you have a compelling reason for not using a specific cipher, you should support them all. However, note that export laws restrict the use of certain encryption ciphers in certain countries. Also, some client software produced before the relaxation of United States Export Control laws cannot use the higher strength encryption. Be aware that while the 40-bit ciphers might hinder the casual eavesdropper, they are not secure and therefore will not stop a motivated attack.

To enable SSL and select encryption ciphers, follow these command line steps:

To specify a certificate:

configutil -o encryption.rsa.nssslpersonalityssl -v certname

There is also a per-service configuration setting for the SSL server certificate nickname. The new configutil settings are as follows:

local.imta.sslnicknames for the SMTP and Submit servers; local.imap.sslnicknames for the IMAP server; local.pop.sslnicknames for the POP server; local.http.sslnicknames for web mail server.

They have the same meaning (and override) the encryption.rsa.nssslpersonalityssl setting. Specifically, this is a comma-separated list of NSS certificate nicknames. Although more than one nickname is permitted in the list, each nickname must refer to a different type of certificate (for example, an RSA cert and a DSS cert) so the setting will almost always be only one nickname. A nickname can be unqualified in which case the NSS software token or default token will be searched, or it can have the form security-module:nickname" in which case the specified security module will be searched for that nickname. This is needed for certificates stored in hardware tokens or places other than the default NSS database.

This does not permit the use of more than one NSS software token in the product. In particular, there is only one cert8.db, key3.db and secmod.db for IMAP, POP, SMTP and HTTP. NSS doesn't permit that.

To enable SSL encryption for outgoing messages, you must modify the channel definition to include the tls channel keywords, such as maytls, musttls, and so on. For more information, see the 12.4.8 Transport Layer Security Manual.

23.5.3 To Set Up Certificate-Based Login

In addition to password-based authentication, Sun Java System servers support authentication of users through examination of their digital certificates. In certificate-based authentication, the client establishes an SSL session with the server and submits the user’s certificate to the server. The server then evaluates whether the submitted certificate is genuine. If the certificate is validated, the user is considered authenticated.

To set up your Messaging Server for certificate-based login:

To Set Up Certificate-Based Login

1. Obtain a server certificate for your server. (For details, see 23.5.1 Obtaining Certificates

2. Run the Certificate Setup Wizard to install the certificates of any trusted certificate authorities that will issue certificates to the users your server will authenticate. (For details, see 23.5.1.6 To Install Certificates of Trusted CAs

   Note that as long as there is at least one trusted CA in the server’s database, the server requests a client certificate from each connecting client.

3. Turn on SSL. (For details, see 23.5.2 To Enable SSL and Selecting Ciphers

4. (Optional) Edit your server’s certmap.conf file so that the server appropriately searches the LDAP user directory based on information in the submitted certificates.

   Editing the certmap.conf file is not necessary if the email address in your users’ certificates matches the email address in your users’ directory entries, and you do not need to optimize searches or validate the submitted certificate against a certificate in the user entry.

   For details of the format of certmap.conf and the changes you can make, see the SSL chapter of Managing Servers with iPlanet Console.

   Once you have taken these steps, when a client establishes an SSL session so that the user can log in to IMAP or HTTP, the Messaging Server requests the user’s certificate from the client. If the certificate submitted by the client has been issued by a CA that the server has established as trusted, and if the identity in the certificate matches an entry in the user directory, the user is authenticated and access is granted (depending on access-control rules governing that user).

   There is no need to disallow password-based login to enable certificate-based login. If password-based login is allowed (which is the default state), and if you have performed the tasks described in this section, both password-based and certificate-based login are supported. In that case, if the client establishes an SSL session and supplies a certificate, certificate-based login is used. If the client does not use SSL or does not supply a certificate, the server requests a password.

23.5.4 How to Optimize SSL Performance Using the SMTP Proxy

Most sites should not use the SMTP proxy as it adds additional latency to the SMTP protocol. However, a large-scale site which makes heavy use of SSL to protect SMTP connections may wish to maximize their investment in SSL accelerator hardware by performing all SSL operations for all protocols on a server which does nothing other than SSL and proxy. The SMTP proxy allows SSL to be processed by a front end proxy server while the mail queues are on a separate MTA machine. This way hardware optimized for each task can be separately configured and purchased.

See Using the MMP SMTP Proxy in Sun Java Communications Suite 5 Deployment Planning Guide and 23.8 Enabling POP Before SMTP for instructions on how to install the SMTP Proxy.