Sun ONE logo      Previous      Contents      Index      Next     

Sun ONE Application Server 7 Administrator's Guide to Security

Chapter 4
Administering SSL/TLS Encryption

Before beginning any of the tasks described in this section, you should already have a certificate and be familiar with the basic concepts of public-key cryptography, including encryption and decryption, public and private keys, digital certificates, and encryption protocols.

This section addresses the following topics:

For more information on encryption and related topics, see Introduction to SSL in the Sun ONE Directory Server document collection.

About Encryption

Encryption is the process of transforming information so it is unintelligible to anyone but the intended recipient; decryption is the process of transforming encrypted information so that it is intelligible again. The Sun ONE Application Server 7 supports the Secure Sockets Layer (SSL) and Transport Layer Security (TLS) encryption protocols.

A cipher is a cryptographic algorithm (a mathematical function), used for encryption or decryption. A cipher suite is a set of ciphers. The SSL and TLS protocols contain numerous cipher suites. Some ciphers are stronger and more secure than others. Generally speaking, the more bits a cipher uses, the harder it is to decrypt the data.

In any two-way encryption process, both parties must use the same ciphers. Because a number of ciphers are available, you need to enable your Sun ONE Application Server for those that are most commonly used in your environment.

SSL and TLS Protocols

The Sun ONE Application Server supports the Secure Sockets Layer (SSL) 3.0 and the Transport Layer Security (TLS) 1.0 protocols for encrypted communication. SSL and TLS are application independent, and higher-level protocols can be layered transparently on them.

SSL and TLS protocols support a variety of ciphers used to authenticate the server and client to each other, transmit certificates, and establish session keys. Clients and servers may support different cipher suites, depending on factors such as which protocol they support, company policies on encryption strength, and government restrictions on export of encrypted software. Among other functions, the SSL and TLS handshake protocols determine how the server and client negotiate which cipher suites they will use to communicate.

During a secure connection, the client and the server agree to use the strongest cipher they have both enabled for communication. You can choose ciphers from the SSL2, SSL3, and TLS protocols.

Note

Improvements to security and performance were made after SSL version 2.0; you should not use SSL 2.0 unless you have clients that are not capable of using SSL 3.0. Client certificates are not guaranteed to work with SSL 2.0 ciphers.

Public and Private Keys

The encryption cipher process alone isn’t enough to secure your server’s confidential information. A key must be used with the encrypting cipher to produce the actual encrypted result, or to decrypt previously encrypted information. The encryption process uses two keys to achieve this result: a public key and a private key. Information encrypted with a public key can be decrypted only with the associated private key. The public key is published as part of a certificate; only the associated private key is safeguarded.

For description of the various cipher suites, and more information about keys and certificates, see Introduction to SSL at the following location:

http://docs.sun.com/db/prod/3802#hic

Task Sequence

The task sequence for setting up basic security is:

  1. Install a certificate.

    2. Refer to "Administering Certificates".

  2. Enable encryption communication with LDAP.

    3. Refer to "Enabling SSL Communication with LDAP".

  3. Turn security on.

    4. Refer to "Turning Security On".

  4. Enable encryption protocols.

    5. Refer to "Enabling SSL and TLS".

  5. Set global security using SSL directives.

    6. "Configuring Security Globally".

Other administrative tasks associated with encryption are discussed in "Using External Encryption Modules", "Setting Strong Ciphers", and "Preventing Clients from Caching SSL Files".

Enabling SSL Communication with LDAP

After a server certificate has been installed, you are ready to activate encryption on the Sun ONE Application Server. It is important that you immediately configure your Admin Server to communicate with the LDAP database using SSL.

Note

This section applies only to HTTP server functionality. Setting SSL communication to LDAP here has no relation to the LDAP communication done by J2EE applications (which use LDAP as described in the Sun ONE Application Server Developer’s Guide).

To enable SSL, perform the following steps:

  1. Access App Server Instances and select the server instance in the left pane.
  2. Access Security in the left pane.
  3. Select Configure Directory Service.
  4. Click Yes next to Use Secure Sockets Layer (SSL) for connections?
  5. Click Save Changes.

    6. You are asked if you want to switch to the standard port.

  6. Click OK to change your port to the standard port for LDAP over SSL.
  7. Access App Server Instances and your server instance in the left pane, then click Apply Changes.
  8. Stop and start the server for changes to take effect.

Turning Security On

You must turn security on before you can configure any other security settings. The following sections describe the ways of turning security on:

Turning Security On When Creating as HTTP Listener

To turn security on when creating a new HTTP listener, perform the following steps:

  1. Access App Server Instances and select the server instance in the left pane.
  2. Access HTTP Server and HTTP Server Listeners in the left pane.

    3. The HTTP Listeners page is displayed.

  3. Click New.

    4. The listener settings page is displayed.

    Figure 4-1  HTTP Listeners Page
    This screen capture shows the HTTP Listeners setup page.

  4. Check Security Enabled under the SSL/TLS Settings to turn security on.
  5. Under Certificate Nickname, select the certificate. For example, Server-Cert.
  6. Entering the rest of the information for the new HTTP listener. Additional information on the fields is contained in the online help.

    7. Figure 4-2  HTTP Listener Security Information
    This screen capture displays the information fields for HTTP Listener settings

  7. Click OK.
  8. Access App Server Instances and your server instance in the left pane, then click Apply Changes.
  9. Stop and start the server for changes to take effect.

Turning Security On When Editing an HTTP Listener

To turn security on when editing an existing HTTP listener, perform the following steps:

  1. Access App Server Instances and select the server instance in the left pane.
  2. Access HTTP Server and HTTP Listeners in the left pane.
  3. Select a listener.

    4. The HTTP Listeners settings page is displayed.

    Note

    If you have an external module installed, the Manage Server Certificates page appears requiring the external module’s password before you can continue.

  4. Under the SSL/TLS Settings, check Security Enabled to turn security on.
  5. Under Certificate Nickname, select the certificate. For example, Server-Cert.
  6. Click Save.
  7. Access App Server Instances and your server instance in the left pane, then click Apply Changes.
  8. Stop and start the server for changes to take effect.

Enabling SSL and TLS

To protect the security of your application server, you should enable SSL by enabling the SSL2, SSL3, and TLS encryption protocols and selecting the various cipher suites.

The default settings allow the most commonly-used ciphers. Unless you have a compelling reason for not using a specific cipher suite, you should allow them all. For more information regarding specific ciphers, see Introduction to SSL at the following location:

Before enabling SSL and TLS, security must be turned on, and you must have at least one certificate installed.

To enable SSL and TLS, perform the following steps:

  1. Access App Server Instances and select the server instance in the left pane.
  2. Access HTTP Servers and HTTP Server Listeners in the left pane.
  3. Select the HTTP listener you want.

    4. The HTTP Listeners setting page is displayed.

    Note

    If you have an external module installed, the Manage Server Certificates page appears requiring the external module’s password before you can continue.

  4. Under the SSL/TLS Settings, check the appropriate boxes associated with SSL and TLS, including all the ciphers.

    5. Note

    Unless you have a compelling reason for not using a specific cipher suite, you should allow them all.

  5. For rollback:
    • TLS must also be enabled on the browser seeking access to your server:
    • For Netscape Navigator 6.0, check both TLS and SSL3.
    • For Microsoft Internet Explorer 5.0 and 5.5, use the TLS Rollback option.
    • For TLS Rollback, check TLS and make sure both SSL3 and SSL2 are disabled.
  6. Click Save.
  7. Access App Server Instances and your server instance in the left pane, then click Apply Changes.
  8. Stop and start the server for changes to take effect.

The init.conf file is automatically modified to show security on, and all virtual servers are automatically assigned the default security parameters.

After you have enabled SSL on a server, its URLs use https instead of http. URLs that point to documents on an SSL-enabled server have this format:

For example:

Configuring Security Globally

Installing an SSL-enabled server creates directive entries in the init.conf file for global security parameters. Security must be enabled for virtual server security settings to work. SSL properties for virtual servers can be found on a per-server basis in the ssl element of the server.xml file.

The Security directive globally enables or disables SSL by making certificates available to the server instance. If enabled, the user is prompted for the administrator password to access certificates, and so on).

Note

When you create a secure HTTP listener through the Administration interface, security is automatically turned on globally in the init.conf file. When you create a secure HTTP listener manually in the server.xml file, security must be turned on by editing the init.conf file.

The following topics address global security configuration:

SSL Configuration File Directives

To configure security globally, you must set values for the following SSL configuration file directives in the init.conf file:

SSLCacheEntries

Specifies the number of SSL sessions that can be cached. There is no upper limit.

Syntax

SSLCacheEntries number

If number is 0, the default value, which is 10000, is used.

SSLClientAuthDataLimit

Specifies the maximum amount of application data, in bytes, that is buffered during the client certificate handshake phase. The default value is 1048576 (1 MB).

SSLClientAuthTimeout

Specifies the number of seconds after which the client certificate handshake phase times out. The default value is 60.

SSLSessionTimeout

Controls SSL2 session caching.

Syntax

SSLSessionTimeout seconds

The seconds value is the number of seconds until a cached SSL2 session becomes invalid. If the SSLSessionTimeout directive is specified, the value of seconds is silently constrained to be between 5 and 100 seconds. The default value is 100.

SSL3SessionTimeout

Controls SSL3 session caching.

Syntax

SSL3SessionTimeout seconds

The seconds value is the number of seconds until a cached SSL3 session becomes invalid. The default value is 86400 (24 hours). If the SSL3SessionTimeout directive is specified, the value of seconds is silently constrained to be between 5 and 86400 seconds.

Setting Values for SSL Directives

To set values for your SSL configuration file directives, perform the following steps:

  1. Access App Server Instances and select the server instance in the left pane.
  2. Access HTTP Server and HTTP Listener in the left pane.
  3. Click the HTTP listener you want.

    4. The HTTP Listener settings page is displayed.

  4. In the SSL/TLS Settings section, check the Security Enabled checkbox.
  5. Under Certificate Nickname, select the certificate. For example, Server-Cert.
  6. Click Save.
  7. Select HTTP Server in the left pane.

    8. The HTTP Server page is displayed.

  8. Select the Advanced tab.

    9. The Advanced settings page is displayed.

    Figure 4-3  HTTP Server Advanced Settings Page
    This screen capture shows the HTTP Server advanced settings page.

  9. Choose the SSL link.

    10. The SSL directives table is displayed.

    Figure 4-4  HTTP Server Advanced SSL Settings Page
    This screen capture shows the HTTP Server advanced SSL settings page.

  10. Select the cipher suites and values.
  11. Click OK.
  12. Access App Server Instances and your server instance in the left pane, then click Apply Changes.
  13. Stop and start the server for changes to take effect.

Using External Encryption Modules

The Sun ONE Application Server supports the two methods for using external cryptographic modules (such as smart cards or token rings): Public Key Cryptography Standard (PKCS#11) and Federal Information Processing Standards (FIPS-140).

Note

You will need to add the PKCS #11 module before enabling the FIPS-140 encryption standard.

This section addresses the following topics:

Installing the PKCS11Module

The Sun ONE Application Server supports Public Key Cryptography Standard (PKCS) #11, which defines the interface used for communication between SSL and PKCS11 modules. PKCS11 modules are used for standards-based connectivity to SSL hardware accelerators. Imported certificates and keys for external hardware accelerators are stored in the secmod.db file, which is generated when the PKCS11 module is installed.

Starting the Server with an External Certificate

If you install a certificate for your server into an external PKCS11 module (for example, a hardware accelerator), the server will not be able to start using that certificate until you enable the HTTP listener to use that certificate.

The server always tries to start with the certificate named Server-Cert. However, certificates in external PKCS11 modules include one of the module’s token names in their identifier. For example, a server certificate installed on an external smartcard reader called smartcard0 would be named smartcard0:Server-Cert.

To start a server with a certificate installed in an external module, you’ll need to specify the certificate name. Using the Administration interface, request and install the certificate specifying the hardware cryptographic module to use.

After you install the certificate in the external hardware token, you can see it in the Manage Certificate page, but not in the HTTP Listener page due to the inherent limitation of the Administration interface.

At this point, you must use the command-line interface to edit the HTTP listener to select the certificate for SSL, change the port number, and so on.

  1. Edit the HTTP listener to select the certificate:

    2. /sun/appserver7/bin/asadmin create-ssl

    -user admin
    -password netscape
    -host qa280r-1.red.iplanet.com
    -port 8888
    -type http-listener
    -certname nobody@apprealm:Server-Cert
    -instance server1
    -ssl3enabled=true
    -ssl3tlsciphers +rsa_rc4_128_md5
    http-listener-1

    To find out the external certificate nickname, go to Manage Certificates page, and enter the key password for external tokens. The certificate name will be displayed, such as nobody@apprealm:Server-Cert.

  2. Enable security on the HTTP listener:

    3. /sun/appserver7/bin/asadmin set

    -user admin
    -password netscape
    -host qa280r-1.red.iplanet.com
    -port 8888
    server1.http-listener.http-listener-1.securityEnabled=true

  3. Change the port number of the HTTP listener:

    4. /sun/appserver7/bin/asadmin set

    -user admin
    -password netscape
    -host qa280r-1.red.iplanet.com
    -port 8888
    server1.http-listener.http-listener-1.port=443

  4. Apply the changes thus far:

    5. /sun/appserver7/bin/asadmin reconfig

    -u admin
    -w netscape
    -H qa280r-1.red.iplanet.com
    -p 8888
    server1

  5. Stop and start the server to enable the HTTP listener to listen on SSL.

Enabling FIPS-140 Standard

PKCS11 APIs enable communication with software or hardware modules that perform cryptographic operations. Once PKCS11 is installed on your Sun ONE Application Server, you can configure the server to be Federal Information Processing Standards (FIPS)-140 compliant.

Note

These libraries are included only in SSL version 3.0.

To enable FIPS-140, perform the following steps:

  1. Install the plug-in following the FIPS-140 instructions.
  2. In the Administration interface, access App Server Instances and select the server instance in the left pane.
  3. Access HTTP Server and HTTP Listeners in the left pane.
  4. Click the link for the HTTP listener you want.

    5. The HTTP Listeners page is displayed.

  5. In the SSL/TLS Settings section of the page, check Security Enabled if it is not already checked.
  6. Check SSL3 Enabled, if it is not already checked.
  7. Check all the SSL/TSL Ciphers, if they are not already checked.
  8. Click Save.
  9. Access App Server Instances and your server instance in the left pane, then click Apply Changes.
  10. Stop and start the server for changes to take effect.

Setting Strong Ciphers

The Strong Ciphers option presents a choice of 168, 128, or 56-bit secret key size for access or no restriction. You can specify a file to be served when the restriction is not met. If no file is specified, the Sun ONE Application Server returns a Forbidden status.

If you select a key size for access that is not consistent with the current cipher settings, the Sun ONE Application Server warns that you need to enable ciphers with larger secret key sizes.

The implementation of the key size restriction is based on an NSAPI PathCheck directive in the obj.conf file, rather than Service fn=key-toosmall. This directive is:

where:

PathCheck returns REQ_NOACTION if SSL is not enabled, or if the secret-keysize parameter is not specified. If the secret key size for the current session is less than the specified secret-keysize, the function returns REQ_ABORTED with a status of PROTOCOL_FORBIDDEN if bong-file is not specified, or else REQ_PROCEED, and the path variable is set to the bong-file filename. Also, when a key size restriction is not met, the SSL session cache entry for the current session is invalidated, so that a full SSL handshake will occur the next time the same client connects to the server.

Note

The Strong Ciphers option removes any Service fn=key-toosmall directives that it finds in an object when it adds a PathCheck fn=ssl-check.

To use the Strong Ciphers option, perform the following steps:

  1. Access App Server Instances and select the server instance in the left pane.
  2. Access HTTP Server in the left pane.
  3. Select Virtual Servers and click the virtual server you want.

    4. Figure 4-5  Virtual Server Settings Tabs
    This screen capture shows the virtual server settings tabs.

  4. Select the HTTP/HTML tab, then click the Strong Ciphers link.

    5. The Enforce Strong Security Requirements page is displayed.

    Figure 4-6  Virtual Server Strong Ciphers Settings Page
    This screen capture shows Enforce Strong Security Requirements (strong ciphers) page.

  5. Choose to edit:
    • From the drop down list
    • By clicking Browse
    • By clicking Wildcard
  6. Select the secret key size restriction:
    • 168 bit or larger
    • 128 bit or larger
    • 56 bit or larger
    • No restrictions
  7. Enter the file location of the message that will be used to reject access.
  8. Click OK.
  9. Access App Server Instances and your server instance in the left pane, then click Apply Changes.
  10. Stop and start the server for changes to take effect.

Preventing Clients from Caching SSL Files

You can prevent pre-encrypted files from being cached by a client by adding the following line inside the <HEAD> section of a file in HTML:



Previous      Contents      Index      Next     

Copyright 2003 Sun Microsystems, Inc. All rights reserved.