RSA Certificate Issues on WebSphere

On some operating systems running WebSphere, root certificates of type RSA are not supported and will generate errors.

In some situations where the default signer certificate "root" from WebSphere is an RSA certificate type, you may see the following exception in the psemagent.log file during the PS Process Monitor discovery:

javax.net.ssl.SSLKeyException: RSA premaster secret error

This indicates that the agent's JDK on the specific operating system does not support RSA certificates.

To resolve RSA certificate issues:

  1. Create a self-signed certificate of type "DSA" using the keytool utility.

    For example, the following command creates a custom keystore "pskey" with one self-signed DSA certificate.

    $Agent_Home/jdk/bin/keytool -genkey -alias dsakey -keystore pskey 
-keyalg DSA -sigalg SHA1withDSA -storepass password -keypass password 
-storetype JKS -dname "cn=localhost"

    In the following steps, you will import this DSA certificate into WebSphere, and make it the default signer certificate for SSL communication.

  2. Login to the WebSphere administration console, and select Security, SSL certificate and key management, Key stores and certificates.

  3. Create a new keystore custompskey and import the above DSA certificate from that "pskey" keystore into this new keystore custompskey.

  4. Select SSL certificate and key management, SSL configurations, and create a new SSL configuration named customSSLConfiguration.

  5. Associate the SSL configuration (customSSLConfiguration) with the keystore custompskey and the DSA key within it.

  6. On the SSL certificate and key management screen click the Manage endpoint security configurations link and replace the "DefaultNodeSSLSettings" with "customSSLConfiguration" settings at the "node" scope level for both inbound and outbound routes.

  7. Verify the certificate by signing on to PeopleSoft.

    Sign on through the HTTPS port, and click "View Certificates" from the dialog box WebSphere presents for the certificate. It should show the DSA certificate imported in the previous steps.

  8. Copy the "pskey" keystore with the DSA certificate to a directory under $AgentHome, and set the pskey.properties file appropriately.

    Discovery of Process Monitor target should be successful.