Chapter 8 Configuring OpenSSO Enterprise 8.0 Update 1 in FIPS Mode

This chapter describes how to configure Sun OpenSSO Enterprise 8.0 Update 1 in Federal Information Processing Standards (FIPS) mode. The following procedures use Sun Java System Web Server 7.0 as the OpenSSO Enterprise web container, with the NSS Certificate DB (certdb) as the key/certificate store.

• Before You Begin

• Configuring the NSS Database in FIPS Mode

• Enabling the FIPS-140 Standard for Sun Java System Web Server 7.0

• Configuring an OpenSSO Enterprise 8.0 Instance Using the Console

Before You Begin

• After you enable FIPS mode, the bootstrap file cannot be decrypted (CR 6835816). Therefore, before you enable FIPS mode, backup the CONFIG_DIR/bootstrap file. Then, after you enable FIPS mode, replace the bootstrap file with the backup copy.

• If Web Server 7.0 has the Java security manager enabled, add the following additional permissions to the server.policy file:

  permission java.security.SecurityPermission "insertProvider.Mozilla-JSS"; permission java.security.SecurityPermission "putProviderProperty.Mozilla-JSS"; permission java.security.SecurityPermission "removeProvider.Mozilla-JSS";

Configuring the NSS Database in FIPS Mode

Configure the NSS database in FIPS mode. For example, using the modutil command:

modutil -fips true -dbdir location-of-your-nss-database

For information about modutil, see http://www.mozilla.org/projects/security/pki/nss/tools/modutil.html.

To Enable the FIPS-140 Standard for Web Server 7.0

To enable the FIPS-140 Standard for Web Server 7.0, you must change the certdb password and enable FIPS mode as true. (By default, Web Server 7.0 sets the password to blank for its certdb.)

Set the password for the internal PKCS11 token using either the Web Server 7.0 Admin Console or CLI command.

Enabling the FIPS-140 Standard for Sun Java System Web Server 7.0

To enable the FIPS-140 Standard for Web Server 7.0, you must change the certdb password and enable FIPS mode as true. (By default, Web Server 7.0 sets the password to blank for its certdb.)

• To Enable the FIPS-140 Standard for Web Server 7.0

• To Set the Password Using the Web Server 7.0 Admin Console

• To Set the Password Using Web Server 7.0 CLI

• To Enable FIPS mode for Web Server 7.0 With modutil

• To Pull the Changes into the Admin Server

• To Test the FIPS Mode Change

To Enable the FIPS-140 Standard for Web Server 7.0

1. Set the password for the internal PKCS11 token using either the Web Server 7.0 Admin Console or CLI command.

To Set the Password Using the Web Server 7.0 Admin Console

1. Log in to the Admin Console.

2. Go to the configuration page in the Admin console.

3. Click the Certificates > PKCS11 Tokens tab.

4. Click the PKCS11 token name (default is internal).

5. Select the Token State checkbox.

6. Enter the password information.

7. Save your changes.

To Set the Password Using Web Server 7.0 CLI

1. Execute the wadm command. For example:

   wadm> set-token-pin -user=admin -password-file=admin.pwd -host=serverhost
   -port=8989 -config=config1 -token=internal

To Enable FIPS mode for Web Server 7.0 With modutil

1. Use modutil in the WS70_ROOT/bin directory. For example:

   modutil -fips true -dbdir location-of-your-nss-database
   

   By default, the NSS database is in the config directory for the Web Server 7.0 instance.

To Pull the Changes into the Admin Server

1. If you use certutil or modutil to modify files in the config directory, you must pull the changes into the Web Server 7.0 Admin Server. For example, using wadm:

   wadm pull-config -user=admin -password-file=_admin-pwfile_ -host=_server-host_ -port=8989 -config=config1 node1

To Test the FIPS Mode Change

1. After you enable FIPS mode for Web Server 7.0, confirm that FIPS is enabled by restarting server. You should see a new prompt for the certdb password. For example:

   > Please enter the PIN for the "NSS FIPS 140-2 Certificate DB" token:

Configuring an OpenSSO Enterprise 8.0 Instance Using the Console

To enable a single OpenSSO Enterprise 8.0 instance in FIPS mode, you must first configure the instance to use the JSS-based implementation class for encryption, Secure Random, SSL sockets, and the HTTPS Protocol Handler.

Before You Begin

• jss4.jar - The WS_INSTALL_DIR/lib/jss4.jar file must be compatible with the NSS version you are using. If necessary, download a compatible jss4.jar file and copy it to the WS_INSTALL_DIR/lib directory.

• Multiple OpenSSO Enterprise 8.0 instances - If you are configuring multiple OpenSSO Enterprise 8.0 instances that are part of a site, first add and configure all instances in the site in non-FIPS mode. Then, after all instances are added and configured for the site, configure the instances in FIPS mode.

To Configure an OpenSSO Enterprise 8.0 Instance Using the Console

1. Log in to the OpenSSO Enterprise Administration Console.

2. Click Configuration, Servers and Sites, and then the Server Name instance.

3. Click the Security tab.

4. Click the Inheritance Settings button.

5. Uncheck the Encryption class, FIPS Mode, and Secure Random Factory Class properties.

6. Click Save and then Back to Server Profile.

7. Change Encryption class to com.iplanet.services.util.JSSEncryption.

8. Change Secure Random Factory Class to com.iplanet.am.util.JSSSecureRandomFactoryImpl.

9. Check Yes for FIPS Mode.

10. Click Save and then the Advanced tab.

11. Change the com.iplanet.security.SSLSocketFactoryImpl property to com.iplanet.services.ldap.JSSSocketFactory.

12. Click Add and add following property with the value:

    opensso.protocol.handler.pkgs=com.iplanet.services.comm

13. Click Add and add following property with the value:

    com.iplanet.am.admin.cli.certdb.dir=path-to-FIPS-enabled-NSS-certdb

14. Click Save.

15. Restart the OpenSSO Enterprise server instance.