Sun Java System Application Server Enterprise Edition 8.2 Performance Tuning Guide

ORB Settings

The Application Server includes a high performance and scalable CORBA Object Request Broker (ORB). The ORB is the foundation of the EJB Container on the server.

Overview

The ORB is primarily used by EJB components via:

RMI/IIOP path from an application client (or rich client) using the application client container.
RMI/IIOP path from another Application Server instance ORB.
RMI/IIOP path from another vendor’s ORB.
In-process path from the Web Container or MDB (message driven beans) container.

When a server instance makes a connection to another server instance ORB, the first instance acts as a client ORB. SSL over IIOP uses a fast optimized transport with high-performance native implementations of cryptography algorithms.

It is important to remember that EJB local interfaces do not use the ORB. Using a local interface passes all arguments by reference and does not require copying any objects.

How a Client Connects to the ORB

A rich client Java program performs a new initialContext() call which creates a client side ORB instance. This in turn creates a socket connection to the Application Server IIOP port. The reader thread is started on the server ORB to service IIOP requests from this client. Using the initialContext, the client code does a lookup of an EJB deployed on the server. An IOR which is a remote reference to the deployed EJB on the server is returned to the client. Using this object reference, the client code invokes remote methods on the EJB.

InitialContext lookup for the bean and the method invocations translate the marshalling application request data in Java into IIOP message(s) that are sent on the socket connection that was created earlier on to the server ORB. The server then creates a response and sends it back on the same connection. This data in the response is then un-marshalled by the client ORB and given back to the client code for processing. The Client ORB shuts down and closes the connection when the rich client application exits.

Monitoring the ORB

ORB statistics are disabled by default. To gather ORB statistics, enable monitoring with this asadmin command:

set serverInstance.iiop-service.orb.system.monitoringEnabled=true
reconfig serverInstance

Connection Statistics

The following statistics are gathered on ORB connections:

total-inbound-connections Total inbound connections to ORB.
total-outbound-connections Total outbound connections from ORB.

Use this command to get ORB connection statistics:

asadmin get --monitor
    serverInstance.iiop-service.orb.system.orb-connection.*

Thread Pools

The following statistics are gathered on ORB thread pools:

thread-pool-size Number of threads in ORB thread pool.
waiting-thread-count Number of thread pool threads waiting for work to arrive.

Use this command to get ORB thread pool statistics:

asadmin get --monitor
    serverInstance.iiop-service.orb.system.orb-thread-pool.*

Tuning the ORB

Tune ORB performance by setting ORB parameters and ORB thread pool parameters. You can often decrease response time by leveraging load-balancing, multiple shared connections, thread pool and message fragment size. You can improve scalability by load balancing between multiple ORB servers from the client, and tuning the number of connection between the client and the server.

The following table summarizes the tunable ORB parameters.

Table 3–5 Tunable ORB Settings


Path	ORB modules	Server settings
RMI/ IIOP from application client to application server	communication infrastructure, thread pool	steady-thread-pool-size, max-thread-pool-size, idle-thread-timeout-in-seconds
RMI/ IIOP from ORB to Application Server	communication infrastructure, thread pool	steady-thread-pool-size, max-thread-pool-size, idle-thread-timeout-in-seconds
RMI/ IIOP from a vendor ORB	parts of communication infrastructure, thread pool	steady-thread-pool-size, max-thread-pool-size, idle-thread-timeout-in-seconds
In-process	thread pool	steady-thread-pool-size, max-thread-pool-size, idle-thread-timeout-in-seconds

Tunable ORB Parameters

Tune the following ORB parameters using the Admin Console:

Thread Pool ID: Name of the thread pool to use.
Max Message Fragment Size: Messages larger than this number of bytes will be fragmented. In CORBA GIOPv1.2, a Request, Reply, LocateRequest and LocateReply message can be broken into multiple fragments. The first message is a regular Request or Reply message with more fragments bit in the flags field set to true. If inter-ORB messages are for the most part larger than the default size (1024 bytes), increase the fragment size to decrease latencies on the network.
Total Connections: Maximum number of incoming connections at any time, on all listeners. Protects the server state by allowing finite number of connections. This value equals the maximum number of threads that will actively read from the connection.
IIOP Client Authentication Required (true/false)

ORB Thread Pool Parameters

The ORB thread pool contains a task queue and a pool of threads. Tasks or jobs are inserted into the task queue and free threads pick tasks from this queue for execution. Do not set a thread pool size such that the task queue is always empty. It is normal for a large application’s Max Pool Size to be ten times the size of the current task queue.

The Application Server uses the ORB thread pool to:

Execute every ORB request.
Trim EJB pools and caches.

Thus, even when one is not using ORB for remote-calls (via RMI/ IIOP), set the size of the threadpool to facilitate cleaning up the EJB pools and caches.

Set ORB thread pool attributes under Configurations > config-name > Thread Pools > thread-pool-ID, where thread-pool-ID is the thread pool ID selected for the ORB. Thread pools have the following attributes that affect performance.

Minimum Pool Size: The minimum number of threads in the ORB thread pool. Set to the average number of threads needed at a steady (RMI/ IIOP) load.
Maximum Pool Size: The maximum number of threads in the ORB thread pool.
Idle Timeout: Number of seconds to wait before removing an idle thread from pool. Allows shrinking of the thread pool.
Number of Work Queues

In particular, the maximum pool size is important to performance. For more information, see Thread Pool Sizing.

Client ORB Properties

Specify the following properties as command-line arguments when launching the client program. You do this by using the following syntax when starting the Java VM:

-Dproperty=value

Controlling connections between client and server ORB

When using the default JDK ORB on the client, a connection is established from the client ORB to the application server ORB every time an initial context is created. To pool or share these connections when they are opened from the same process by adding to the configuration on the client ORB.

-Djava.naming.factory.initial=com.sun.appserv.naming.S1ASCtxFactory

Using multiple connections

Note –

The property com.sun.appserv.iiop.orbconnections is not supported in Sun Java System Application Server, version 8.x.

When using the context factory, (com.sun.appserv.naming.S1ASCtxFactory), you can specify the number of connections to open to the server from the client ORB with the property com.sun.appserv.iiop.orbconnections.

The default value is one. Using more than one connection may improve throughput for network-intense applications. The configuration changes are specified on the client ORB(s) by adding the following jvm-options:

-Djava.naming.factory.initial=com.sun.appserv.naming.S1ASCtxFactory
-Dcom.sun.appserv.iiop.orbconnections=value

Load Balancing

For information on how to configure RMI/IIOP for multiple application server instances in a cluster, Chapter 11, RMI-IIOP Load Balancing and Failover, in Sun Java System Application Server Enterprise Edition 8.2 High Availability Administration Guide.

When tuning the client ORB for load-balancing and connections, consider the number of connections opened on the server ORB. Start from a low number of connections and then increase it to observe any performance benefits. A connection to the server translates to an ORB thread reading actively from the connection (these threads are not pooled, but exist currently for the lifetime of the connection).

Thread Pool Sizing

After examining the number of inbound and outbound connections as explained above, tune the size of the thread pool appropriately. This can affect performance and response times significantly.

The size computation takes into account the number of client requests to be processed concurrently, the resource (number of CPUs and amount of memory) available on the machine and the response times required for processing the client requests.

Setting the size to a very small value can affect the ability of the server to process requests concurrently, thus affecting the response times since requests will sit longer in the task queue. On the other hand, having a large number of worker threads to service requests can also be detrimental because they consume system resources, which increases concurrency. This can mean that threads take longer to acquire shared structures in the EJB container, thus affecting response times.

The worker thread pool is also used for the EJB container’s housekeeping activity such as trimming the pools and caches. This activity needs to be accounted for also when determining the size. Having too many ORB worker threads is detrimental for performance since the server has to maintain all these threads. The idle threads are destroyed after the idle thread timeout period.

Examining IIOP Messages

It is sometimes useful to examine the IIOP messages passed by the Application Server. To make the server save IIOP messages to the server.log file, set the JVM option -Dcom.sun.CORBA.ORBDebug=giop. Use the same option on the client ORB.

The following is an example of IIOP messages saved to the server log. Note: in the actual output, each line is preceded by the timestamp, such as [29/Aug/2002:22:41:43] INFO (27179): CORE3282: stdout.

 ++++++++++++++++++++++++++++++
 Message(Thread[ORB Client-side Reader, conn to 192.18.80.118:1050,5,main]):
createFromStream: type is 4 <
 MessageBase(Thread[ORB Client-side Reader, conn to 192.18.80.118:1050,5,main]): 
Message GIOP version: 1.2
 MessageBase(Thread[ORB Client-side Reader, conn to 192.18.80.118:1050,5,main]): 
ORB Max GIOP Version: 1.2
 Message(Thread[ORB Client-side Reader, conn to 192.18.80.118:1050,5,main]): 
createFromStream: message construction complete.
 com.sun.corba.ee.internal.iiop.MessageMediator
(Thread[ORB Client-side Reader, conn to 192.18.80.118:1050,5,main]): Received message:
 ----- Input Buffer -----
 Current index: 0
 Total length : 340
 47 49 4f 50 01 02 00 04 0 0 00 01 48 00 00 00 05 GIOP.......H....

Note –

The flag -Dcom.sun.CORBA.ORBdebug=giop generates many debug messages in the logs. This is used only when you suspect message fragmentation.

In this sample output above, the createFromStream type is shown as 4. This implies that the message is a fragment of a bigger message. To avoid fragmented messages, increase the fragment size. Larger fragments mean that messages are sent as one unit and not as fragments, saving the overhead of multiple messages and corresponding processing at the receiving end to piece the messages together.

If most messages being sent in the application are fragmented, increasing the fragment size is likely to improve efficiency. On the other hand, if only a few messages are fragmented, it might be more efficient to have a lower fragment size that requires smaller buffers for writing messages.

Improving ORB Performance with Java Serialization

It is possible to improve ORB performance by using Java Serialization instead of standard Common Data Representation (CDR) for data for transport over the network. This capability is called Java Serialization over GIOP (General Inter-ORB Protocol), or JSG.

In some cases, JSG can provide better performance throughput than CDR. The performance differences depend highly on the application. Applications with remote objects having small amounts data transmitted between client and server will most often perform better using JSG.

To Enable Java Serialization

You must set this property on all servers that you want to use JSG.

In the tree component, expand the Configurations node.

Expand the desired node.

Select the JVM Settings node

In the JVM Settings page, choose the JVM Options tab.

Click Add JVM Option, and enter the following value:

-Dcom.sun.CORBA.encoding.ORBEnableJavaSerialization=true

Click Save

Restart the Application Server.

Using JSG for Application Clients

If an application uses standalone non-web clients (application clients), and you want to use JSG, you must also set a system property for the client applications. A common way to do this is to add the property to the Java command line used to start the client application, for example:

java -Dcom.sun.CORBA.encoding.ORBEnableJavaSerialization=true
 -Dorg.omg.CORBA.ORBInitialHost=gollum
 -Dorg.omg.CORBA.ORBInitialPort=35309
MyClientProgram