How to Set SMTP to Use TLS

Language:

This procedure uses sample data to show you how to set up the certificates that enable sendmail to use TLS. For more information, see Support for Running SMTP With TLS in Version 8.13 of sendmail.

Before You Begin

You need to provide the following information to create certificates:

Country Name
State or Province Name
Locality Name
Organization Name
Organizational Unit Name
Common Name, which is the machine's fully qualified host name. For more information, see the check-hostname(8) man page.
Email Address

Become an administrator.
For more information, see Using Your Assigned Administrative Rights in Securing Users and Processes in Oracle Solaris 11.4.

Stop the sendmail service.

# svcadm disable -t network/smtp:sendmail

Set up the certificates that enable sendmail to use TLS.

Run the following commands:

# cd /etc/mail
# mkdir -p certs/CA
# cd certs/CA
# mkdir certs crl newcerts private
# echo "01" > serial
# cp /dev/null index.txt
# cp /etc/openssl/openssl.cnf .

Change the dir value in the openssl.cnf file from /etc/openssl to /etc/mail/certs/CA.

Use the openssl command-line tool to implement TLS.

The following example command generates interactive text.

# openssl req -new -x509 -keyout private/cakey.pem -out cacert.pem -days 365 \
-config openssl.cnf
Generating a 1024 bit RSA private key
.....................................++++++
.....................................++++++
writing new private key to 'private/cakey.pem'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) []:US
State or Province Name (full name) []:California
Locality Name (eg, city) []:Menlo Park
Organization Name (eg, company) [Unconfigured OpenSSL Installation]:Oracle
Organizational Unit Name (eg, section) []:Solaris
Common Name (eg, YOUR name) []:somehost.somedomain.example.com
Email Address []:someuser@example.com

req: Creates and processes certificate requests.
–new: Generates a new certificate request.
–x509: Creates a self-signed certificate.
–keyout private/cakey.pem: Assigns private/cakey.pem as the file name for the newly created private key.
–out cacert.pem: Assigns cacert.pem as the output file.
–days 365: Certifies the certificate for 365 days. The default value is 30.
–config openssl.cnf: Specifies openssl.cnf as the configuration file.

(Optional) If you need a new secure connection, make a new certificate and sign the new certificate with the certificate authority.

Make a new certificate.

For example:

# cd /etc/mail/certs/CA
# openssl req -nodes -new -x509 -keyout newreq.pem -out newreq.pem -days 365 \
-config openssl.cnf
Generating a 1024 bit RSA private key
..............++++++
..............++++++
writing new private key to 'newreq.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) []:US
State or Province Name (full name) []:California
Locality Name (eg, city) []:Menlo Park
Organization Name (eg, company) [Unconfigured OpenSSL Installation]:Oracle
Organizational Unit Name (eg, section) []:Solaris
Common Name (eg, YOUR name) []:somehost.somedomain.example.com
Email Address []:someuser@example.com

This command requires that you provide the same information that you provided in step 3c.

In this example, the certificate and private key are in the file newreq.pem.

Sign the new certificate with the certificate authority.

For example:

# cd /etc/mail/certs/CA
# openssl x509 -x509toreq -in newreq.pem -signkey newreq.pem -out tmp.pem
Getting request Private Key
Generating certificate request
# openssl ca -config openssl.cnf -policy policy_anything -out newcert.pem -infiles tmp.pem
Using configuration from openssl.cnf
Enter pass phrase for /etc/mail/certs/CA/private/cakey.pem:
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number: 1 (0x1)
        Validity
            Not Before: Jun 23 18:44:38 2005 GMT
            Not After : Jun 23 18:44:38 2006 GMT
        Subject:
            countryName               = US
            stateOrProvinceName       = California
            localityName              = Menlo Park
            organizationName          = Oracle
            organizationalUnitName    = Solaris
            commonName                = somehost.somedomain.example.com
            emailAddress              = someuser@example.com
        X509v3 extensions:
            X509v3 Basic Constraints: 
                CA:FALSE
            Netscape Comment: 
                OpenSSL Generated Certificate
            X509v3 Subject Key Identifier: 
                93:D4:1F:C3:36:50:C5:97:D7:5E:01:E4:E3:4B:5D:0B:1F:96:9C:E2
            X509v3 Authority Key Identifier: 
                keyid:99:47:F7:17:CF:52:2A:74:A2:C0:13:38:20:6B:F1:B3:89:84:CC:68
                DirName:/C=US/ST=California/L=Menlo Park/O=Oracle/OU=Solaris/\
                CN=someuser@example.com/emailAddress=someuser@example.com
                serial:00

Certificate is to be certified until Jun 23 18:44:38 2006 GMT (365 days)
Sign the certificate? [y/n]:y

1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
# rm -f tmp.pem

In this example, the file newreq.pem contains the unsigned certificate and private key. The file newcert.pem contains the signed certificate.

x509 utility: Displays certificate information, converts certificates to various forms, and signs certificate requests
ca applications: Signs certificate requests in a variety of forms and generates certificate revocation lists (CRLs)

Enable sendmail to use the certificates by adding the following lines to your .mc file:

define(`confCACERT_PATH', `/etc/mail/certs')dnl
define(`confCACERT', `/etc/mail/certs/CAcert.pem')dnl
define(`confSERVER_CERT', `/etc/mail/certs/MYcert.pem')dnl
define(`confSERVER_KEY', `/etc/mail/certs/MYkey.pem')dnl
define(`confCLIENT_CERT', `/etc/mail/certs/MYcert.pem')dnl
define(`confCLIENT_KEY', `/etc/mail/certs/MYkey.pem')dnl

For more information, see Configuration File Options for Running SMTP With TLS.

Rebuild and install the sendmail.cf file in your /etc/mail directory.
For more information, see Changing the sendmail Configuration.

Create symbolic links from the files you created with openssl to the files you defined in your .mc file.

# cd /etc/mail/certs
# ln -s CA/cacert.pem CAcert.pem
# ln -s CA/newcert.pem MYcert.pem
# ln -s CA/newreq.pem MYkey.pem

Deny read permission to group and others for MYkey.pem for more security.
```
# chmod go-r MYkey.pem
```
Use a symbolic link to install CA certs in the directory assigned to confCACERT_PATH.
```
# C=CAcert.pem
# ln -s $C `openssl x509 -noout -hash < $C`.0
```
For secure mail with other hosts, install their host certificates.
1. Copy the file defined by the other host's confCACERT option to /etc/mail/certs/host.domain.cert.pem.
  Replace host.domain with the other host's fully qualified host name.
2. Use a symbolic link to install CA certs in the directory assigned to confCACERT_PATH.
```
# C=host.domain.cert.pem
# ln -s $C `openssl x509 -noout -hash < $C`.0
```
  Replace host.domain with the other host's fully qualified host name.
Restart the sendmail service.
```
# svcadm enable network/smtp:sendmail
```

Example 1 Received: Mail Header

The following example shows a sample Received: header for secure mail with TLS.

Received: from his.example.com ([IPv6:2001:db8:3c4d:15::1a2f:1a2b])
        by her.example.com (8.13.4+Sun/8.13.4) with ESMTP id j2TNUB8i242496
        (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
        for <janepc@her.example.com>; Tue, 29 Mar 2005 15:30:11 -0800 (PST)
Received: from her.example.com (her.city.example.com [192.0.2.0])
        by his.example.com (8.13.4+Sun/8.13.4) with ESMTP id j2TNU7cl571102
        version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
        for <janepc@her.example.com>; Tue, 29 Mar 2005 15:30:07 -0800 (PST)

Note that the value for verify is OK, which means that the authentication was successful. For more information, see Macros for Running SMTP With TLS.

For more information, see the following OpenSSL man pages:

openssl(1)
req(1)
x509(1)
ca(1)