To enable a single OpenSSO Enterprise 8.0 instance in FIPS mode, you must first configure the instance to use the JSS-based implementation class for encryption, Secure Random, SSL sockets, and the HTTPS Protocol Handler.
Before You Begin
jss4.jar - The WS_INSTALL_DIR/lib/jss4.jar file must be compatible with the NSS version you are using. If necessary, download a compatible jss4.jar file and copy it to the WS_INSTALL_DIR/lib directory.
Multiple OpenSSO Enterprise 8.0 instances - If you are configuring multiple OpenSSO Enterprise 8.0 instances that are part of a site, first add and configure all instances in the site in non-FIPS mode. Then, after all instances are added and configured for the site, configure the instances in FIPS mode.
Log in to the OpenSSO Enterprise Administration Console.
Click Configuration, Servers and Sites, and then the Server Name instance.
Click the Security tab.
Click the Inheritance Settings button.
Uncheck the Encryption class, FIPS Mode, and Secure Random Factory Class properties.
Click Save and then Back to Server Profile.
Change Encryption class to com.iplanet.services.util.JSSEncryption.
Change Secure Random Factory Class to com.iplanet.am.util.JSSSecureRandomFactoryImpl.
Check Yes for FIPS Mode.
Click Save and then the Advanced tab.
Change the com.iplanet.security.SSLSocketFactoryImpl property to com.iplanet.services.ldap.JSSSocketFactory.
Click Add and add following property with the value:
opensso.protocol.handler.pkgs=com.iplanet.services.comm
Click Add and add following property with the value:
com.iplanet.am.admin.cli.certdb.dir=path-to-FIPS-enabled-NSS-certdb
Click Save.
Restart the OpenSSO Enterprise server instance.