Using Security in CORBA Applications

Troubleshooting

This topic includes the following sections:

Notes: The problems in this topic pertain to using the SSL protocol and certificate authentication with CORBA applications.

The BEA Tuxedo CORBA Java client and BEA Tuxedo CORBA Java client ORB were deprecated in Tuxedo 8.1 and are no longer supported in Tuxedo 9.0.

All BEA Tuxedo CORBA Java client and BEA Tuxedo CORBA Java client ORB text references, associated code samples, etc. should only be used:

to help implement/run third party Java ORB libraries, and
for programmer reference only.

Technical support for third party CORBA Java ORBs should be provided by their respective vendors. BEA Tuxedo does not provide any technical support or documentation for third party CORBA Java ORBs.

Using ULOGS and ORB Tracing

In general, Object Request Brokers (ORBs) write important failures to the ULOG file. When using the CORBA C++ ORB, you can also enable ORB internal tracing which may provide information in addition to the information that appears in the ULOG file.

When looking at the ULOG file, note that remote ORB processes by default do not write data to the ULOG file in APPDIR.

On UNIX, the remote ORB writes information to a ULOG file in the current directory.
On Windows 2003, the remote ORB writes information to a ULOG file in the c:\ulog directory.

You can set the ULOGPFX environment variable to control the location of the ULOG file for remote ORBs (for example, you can set the location of the ULOG file to APPDIR so that all information is put in the same ULOG file). Set the ULOGPFX environment variable as follows:

Windows 2003

set ULOGPFX=%APPDIR%\ULOG

UNIX

setenv ULOGPFX $APPDIR/ULOG

To enable ORB tracing, complete the following steps:

Create a file named trace.dat in APPDIR. The contents of trace.dat should have all=on.

Use the following command to set the OBB_TRACE_INPUT environment variable to point to the trace.dat file before running the application:

set OBB_TRACE_INPUT=%APPDIR%\trace.dat
If you want ORB tracing sent to separate files, add the following line to the trace.dat file:
output=obbtrace%p.log
This command sends the trace output to files that are named after each running process. You may want to do this if you are using ORB tracing on UNIX to an NFS mounted drive. In this case, trace performance is slow due to the user log opening, writing, and closing the file for each trace statement.

CORBA::ORB_init Problems

The ORB_init routine does not perform internal ORB tracing so you will not see any trace output for invalid argument processing. Therefore, you need to double check the arguments that were passed to the ORB_init routine.

If a CORBA::BAD_PARAM exception occurs when executing the ORB_init routine, verify that all required arguments have values. Also, check that arguments which expect a value from a specific set of valid values have the correct value. Note that values for the arguments of the ORB_init routine are case sensitive.

If a CORBA::NO_PERMISSION exception occurs and an SSL argument was specified to the ORB_init routine, make sure the security license is enabled. Also, verify that the specified level of encryption does not exceed the encryption level supported by the security license.

If a CORBA::IMP_LIMIT exception occurs when executing the ORB_init routine, verify that the ORBport and ORBSecurePort system properties have the same value.

If a CORBA::Initialize exception occurs when executing the ORB_init routine, verify that the values for OrbId or configset are valid.

If Secure Sockets Layer (SSL) arguments are passed to the ORB_init routine, the ORB attempts to load and initialize the SSL protocol. If no SSL arguments are passed, the ORB does not attempt to initialize the SSL protocol.

The ORB is not aware of the new URL address formats for the Bootstrap object so if you specify a corbaloc or corbalocs URL address format, the ORB does not try to load the SSL protocol during the ORB_init routine.

If SSL arguments were specified to the ORB_init routine, check the following:

The specified values for the SSL arguments do not conflict with each other or other ORB arguments.
Whether or not the ORB is a native process. If the ORB is a native process, SSL arguments are not supported.
That the value specified for the maxCrypto system property is less than the value specified for the minCrypto system property. The values for the properties must be within the range appropriate for the license.
Application-controlled SSL configuration parameters that are not correct. The ORB_init routine does not perform digital certificate lookups check so look for missing or corrupted files that would case the dynamic libraries not to be loaded. Also, verify the dynamic libraries are loaded. The ORB trace function will provide information about whether or not the dynamic libraries are loaded.

If the problem persists, turn on ORB tracing. ORB tracing will log SSL failures that occur when the liborbssl dynamic library is loaded and initialized.

Password Authentication Problems

If the client application fails when using the corbalocs URL address format with password authentication, check the following:

The proper configuration steps were performed. See Configuring the SSL Protocoland Configuring Authentication for the list of the required configuration steps.
An initialization error occurred. Specify a valid SSL system property to the ORB_init routine, an error occurs if:

The IIOP Listener/Handler is not available. The ORB trace log will show failed connection attempts.
The IIOP Listener/Handler is available but it does not support the SSL protocol. The ULOG file will show that a non-GIOP message was received.
The IIOP Listener/Handler was available and configured for the SSL protocol but the SSL connection could not be established. This error can occur when the range of encryption strengths supported by the IIOP Listener/Handler and the range of encryption strengths required by the client application do not match.

The ULOG file will indicate that a non-GIOP message was received if the IIOP Listener/Handler was configured for the SSL protocol but the CORBA client application used a TOBJADDR object without the corbalocs prefix to indicate a secure connection.

Certificate Authentication Problems

If the client application fails when using the corbalocs URL address format with certificate authentication, check the following:

The proper configuration steps were performed. See Configuring the SSL Protocol and Configuring Authentication for the list of the required configuration steps.
Determine whether or not an initialization error occurred.
Specify a valid SSL system property to the ORB_init routine, an error occurs if:

The IIOP Listener/Handler is not available. The ORB trace log will show failed connection attempts.
The IIOP Listener/Handler is available but it does not support the SSL protocol. The ULOG file will show that a non-GIOP message was received.
The IIOP Listener/Handler was available and configured for the SSL protocol but the SSL connection could not be established. This error can occur when the range of encryption strengths supported by the IIOP Listener/Handler and the range of encryption strengths required by the client application do not match. The error can also occur when the client application does not trust the certificate chain of the IIOP Listener/Handler or the client application did not receive a certificate from the IIOP Listener/Handler. The error will be written to the ULOG file and the error will also show up in the ORB trace output.

If an error does not occur, the problem is in the authentication process and the ULOG file will contain one of the following error statements indicating the problem:

Couldn't connect to an LDAP server
Couldn't find a filter that matched the client certificate
The client certificate was not found in LDAP
The private key file could not be found
The passphrase used to open the private key is not correct
The public key from the client certificate did not match the private key

Additional certificate problems can also occur. See Tobj::Bootstrap:: resolve_initial_references Problems for more information about the types of certificate errors that can occur.

Note: At this point of the initialization process, the failure is not due to a problem in the IIOP Listener/Handler.

Tobj::Bootstrap::
resolve_initial_references Problems

If a failure occurs when performing a Tobj::Bootstrap::resolve_initial_references with the corbaloc or corbalocs URL address format, a CORBA::InvalidDomain exception is raised. This exception may mask CORBA::NO_PERMISSION or CORBA::COMM_FAILURE exceptions that are raised internally. Look at the ULOG file and turn on ORB tracing to get more details on the error. The following errors may occur:

If the IIOP Listener/Handler is not available, the ORB trace log will show failed connection attempts.
If the IIOP Listener/Handler is available but it does not support the SSL protocol, the ULOG file will show that a non-GIOP message was received.
If the IIOP Listener/Handler is available and configured for the SSL protocol but the SSL connection could not be established. An error can occur if the range of encryption strengths supported by the IIOP Listener/Handler and required by the client application do not match.
The IIOP Listener/Handler could not map a certificate to a username/password combination. Verify that the security level for the CORBA application is set to USER_AUTH and that the specified username matches the principal name passed into the authenticate call. Also, check that the username does not exceed the 30 character limit.

Additional certificate problems can occur. See Troubleshooting Tips for Digital Certificates on page 11-8 for more information about the types of certificate errors that can occur.

IIOP Listener/Handler Startup Problems

This section describes problems that can occur during the startup of the IIOP Listener/Handler.

If a failure occurs when starting the IIOP Listener/Handler, check the ULOG file for a description of the error. The IIOP Listener/Hander verifies that the values for the SSL arguments specified in the CLOPT parameters are valid. If any of the values are invalid, the appropriate error is recorded in the ULOG file. This check is similar to the argument checking done by the ORB.

The IIOP Listener/Handler will not start its processes unless the -m option is specified. The ISH is the process that actually loads and initializes the SSL libraries. If there is a problem loading and initializing the SSL libraries in the ISH process, the error will not be recorded in the ULOG file until the ISH process starts to handle incoming requests from client application.

If you suspect a problem with the startup of the IIOP Listener/Handler processes, check the ULOG file.

Configuration Problems

The following are miscellaneous tips to resolve the common configuration problems which may occur when using security:

The ORB -ORBpeerValidate command-line option and the -v option of the ISL command do not control the peer validation rules checking. This system property and option only control the checking of the host name specified in the peer certificate against the host name of the machine to which the principal was connected.
The only way to disable the peer validation rules on an installed kit is to create an empty file for %TUXDIR%\udataobj\security\certs\peer_val.rul. If you are writing a script that builds your CORBA application, you cannot register the peer_val.rul file in the script.
When enabling renegotiation intervals in the IIOP Listener/Handler, check that the option on the ISL command is -R not -r. If you use an -r, the IIOP Listener/Handler will use the SSL protocol but the renegotiation interval will not be used. In addition, the ULOG file will note that an unknown option was specified on the IIOP Listener/Handler.

Another way to determine if the IIOP Listener/Handler is performing renegotiations is to enable ORB tracing on the client side and check whether the cipher suite negotiation callback is being called the configured renegotiation interval. Note that the client application must be sending requests for in order for renegotiations to occur.

If you have defined the SECURITY parameter in the CORBA application's UBBCONFIG file to be APP_PW or greater and you have configured the IIOP Listener/Handler to use the SSL protocol but not mutual authentication, you must use password authentication with the corbalocs URL address format to communicate with the IIOP Listener/Handler. If you try to use certificate authentication, the IIOP Listener/Handler will not ask the principal for a certificate when establishing an SSL connection and the IIOP Listener/Handler is not able to map the identity of the principal to a BEA Tuxedo identity.

Problems with Using Callbacks Objects with the SSL Protocol

If you have a joint client/server application and the client portion of the joint client/server application specifies security requirements using either the corbalocs URL address format or by requiring credentials, you must use the -ORBsecurePort system property with the ORB_init routine to specify that a secure port be used.

If you do not specify the -ORBsecurePort system property, the server registration will fail with a CORBA::NO_PERMISSION exception. To verify this is the problem, enable ORB tracing and look for the following trace output:

TCPTransport::Listen: FAILURE: Attempt to listen on clear port while Credentials require SSL be used

If you want to use the SSL protocol with callback objects, the joint client/server application must use the SecurityLevel2::PrincipalAuthenticator::authenticate() method with certificate authentication. Otherwise, the joint client/server application does not have a certificate with which to identify itself to the IIOP Listener/Handler which in this case is the initiator of the SSL connection.

Troubleshooting Tips for Digital Certificates

In general, problems with digital certificates occur when:

One of the digital certificates in the certificate chain of the IIOP Listener/Handler is not from a certificate authority defined in the trust_ca.cer file. A problem can occur if any certificate authority in the trust_ca.cer file is invalid.
The name the IIOP Listener/Handler connected to the client application does not match the host name specified in digital certificates of the IIOP Listener/Handler when a host match is performed. The name of the IIOP Listener/Handler is specified in the CommonName attribute of the distinguish name of the IIOP Listener/Handler. The host name and the CommonName attribute must match exactly.

You can verify this error by setting the -ORBpeerValidate system property to none and executing the ORB_init routine again.

One of the digital certificates in the certificate chain of the IIOP Listener/Handler does not match the specified peer validation rules.
The digital certificate of the IIOP Listener/Handler is invalid. The digital certificate of the IIOP Listener/Handler becomes invalid when the digital certificate is tampered with, it expires, or the certificate authority that issued the digital certificate expires.

If a digital certificate is rejected for no explainable reason, complete the following steps:

Open the digital certificate in a viewer, for example, Microsoft Explorer.

Look at the KeyUsage and BasicConstraints properties of the digital certificate. A small yellow triangle with an exclamation mark indicates the property is critical. Any digital certificate with a property marked critical is rejected by the BEA Tuxedo software.

If the none of the properties of the digital certificate are critical, check the properties of the next digital certificate in the certificate chain. Perform this step until all the properties of all the digital certificates in the certificate chain have been verified.