Yes. Use the following Java 2 Platform, Standard Edition (J2SETM) properties to change the keystore password:
-Djavax.net.ssl.keyStorePassword=password -Djavax.net.ssl.trustStorePassword=password
Note that the keystore password must match the individual key passwords to perform operations on the keys, so you must change the keystore password with the property mentioned above, and then change the password to each key to match that password.