Skip Headers
Oracle® Application Server Performance Guide
10g Release 3 (10.1.3.1.0)

Part Number B28942-01
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
View PDF

3 Top Performance Areas

This chapter provides a description of top tuning areas for Oracle Application Server and includes the following sections:

3.1 Top Performance Areas

This section covers critical Oracle Application Server performance issues and provides a quickstart for tuning J2EE applications that run on OC4J. Table 3-1 lists a quick guide for performance considerations for Oracle Application Server.

Table 3-1 Top Performance Areas for Oracle Application Server Applications

Performance Area Description and Reference

Ensure Sufficient Hardware Resources

Oracle Application Server has minimum hardware requirements that enable you to use the product. You also need to run on hardware that supports the needs for your applications, including the database tier. For details on Oracle Application Server installation requirements, see the Requirements chapter in the Installation Guide for your platform.

See "Ensure Sufficient Hardware Resources" for more information on platform specific tools that can help you determine if your hardware resources are sufficient.

Ensure Sufficient Java Heap Size

To improve the performance of a J2EE application, you need to provide an adequate heap size for the JVM where the application runs. If the OC4J running your J2EE application does not have enough memory, performance can suffer due to the overhead required to manage limited memory.

See "Ensure Sufficient Java Heap for OC4J" for details on setting JVM heap size options.

Tune the JVM for Garbage Collection

To improve the performance of a J2EE application and ensure that your application performance is not being negatively impacted by JVM garbage collection, you need to manage the heap size and sometimes you also need to set JVM options that impact garbage collection frequency.

See "Tune the JVM Garbage Collection Options" for more information on garbage collection.

Reuse Database Connections

By default, OC4J data sources enable database connection pooling. Thus, OC4J manages database connections to avoid frequently reestablishing new connections. The Oracle Application Server data source facility provides options you can use to control the number of database connections maintained, and how long they are maintained.

See "Reuse Database Connections".

Specify Sufficient HTTP Connections

Tune the Oracle HTTP Server directives to set the level of concurrency by specifying the number of HTTP connections.

See "Specify Sufficient Oracle HTTP Server Connections".

Enable JDBC Statement Caching Option

By enabling statement caching to lower the overhead of repeated cursor creation and repeated statement parsing and creation, you can improve performance for applications using the database tier.

See "Enable Statement Caching for Data Sources".

Ensure the Database is Properly Tuned

For applications that access a database, ensure that your database is properly configured to support your application's requirements.

See "Verify Database Tuning".

Verify Logging Levels

You need to make sure that the logging level is not set higher than the default INFO level logging. If the logging setting does not match the default level, reset the logging level to the default for best performance.

See "Verify Logging Levels".

Reuse EJB Instances

Set the EJB options for creating and reusing instances to improve EJB performance.

See "Reuse EJB Instances".


3.1.1 Ensure Sufficient Hardware Resources

A most crucial performance area is ensuring that there are sufficient CPU, memory and network resources to support the user population and application requirements for your Oracle Application Server installation. You need to monitor resource utilization over an extended period to determine if you have occasional peaks of usage or whether a resource is consistently saturated. You also need to define the acceptable response times and throughputs for applications running at your site, for both peak and extended periods. Also, check the system while running your application under normal load and monitor operating system statistics, including, CPU, memory, disk, and network performance to determine if any hardware resource is saturated.

To check the CPU, memory, and disk performance you can use the following commands:

On Linux systems use the sar or mpstat command.

On Windows systems use the perfmon command.

To check network performance, you can use the following commands:

On Linux and Windows systems:

% netstat

On Windows systems, you can also use the Windows Task Manager to check network performance.

If any of the hardware resources are saturated, this could be due to one or more of the following:

  • The hardware resources are insufficient to run the application.

  • The system is not properly configured.

  • The application or database needs to be tuned.

For a consistently saturated resource, the solutions are to reduce load or increase resources. For peak traffic periods, if the increased response time is not acceptable the alternatives are to again increase resources or to determine if there is traffic that can be rescheduled to reduce the peak load, such as scheduling batch or background operations during slower periods. Oracle Application Server provides a variety of mechanisms to help you control resource concurrency; this can limit the impact of bursts of traffic. However, for a consistently saturated system, these mechanisms should be viewed as temporary solutions.

3.1.2 Ensure Sufficient Java Heap for OC4J

If you have sufficient memory available on your system and your application is memory intensive, increase the JVM heap size from the default value. While the amount of heap required varies based on the application and on the available memory, for most OC4J server applications, if you have sufficient memory, then Oracle recommends using an initial heap size of 512 Megabytes or larger.

You can improve performance by setting the initial heap size equal to the maximum heap size.

Take the following steps to change the heap size values for an OC4J instance:

  1. Navigate to the Home page for the OC4J instance.

  2. Click Administration.

  3. If necessary, expand the Properties section of the table by clicking the Expand icon. Then, click the Go to Task icon in the Server Properties row.

  4. In the Command Line Options area, change the value in the Maximum heap size and Initial heap size fields.

  5. Click Apply.

  6. Navigate to the Cluster Topology page, select the OC4J instance that you modified, and click Restart. On the Confirmation page, click Yes.

This specifies the following JVM options and changes the size of the heap allocated to the OC4J process in an OC4J instance.

If your Oracle Application Server topology includes more than one JVM on the same system, then to maximize performance, set the maximum heap size to accommodate application requirements and make sure that the total memory consumed by all of the JVMs running on the system does not exceed the memory capacity of your system.


See Also:

  • The JVM Metrics page in Application Server Control Console. This page is available from the OC4J home page, by clicking the Performance Secondary tab, and then, in the Related Links area, clicking JVM Metrics.

  • You can find detailed information about JVM options and their impact on performance on the JVM vendor's Web sites, such as

    http://java.sun.com/performance/reference/whitepapers/5.0_performance.html


3.1.3 Tune the JVM Garbage Collection Options

JVM garbage collection is an expensive and can have an impact on application performance; inefficient garbage collection can severely degrade application performance. Therefore, it is important to understand how applications create and destroy objects.

To tune the JVM garbage collection options you need to analyze garbage collections data and check for the frequency and type of garbage collections, the size of the memory pools, and the time spent on garbage collection.

In order to determine application memory requirements you can monitor JVM garbage collection and memory pool sizes using the following:

  • The JVM command line options:

    -verbose:gc
    -XX:+PrintGCDetails
    
    

    Look for "Full GC" to identify major collections.

  • jstat tool

  • visualgc tool

  • The Application Server Control Console JVM Metrics page shows JVM memory pool and garbage collector information. This page is available from the OC4J home page by clicking the Performance Secondary tab, and then, in the Related Links area, clicking JVM Metrics.

Set the -XX:+AggressiveHeap JVM option to tune internal VM parameters, and increase the total heap size, as described in "Ensure Sufficient Java Heap for OC4J" to reduce the overhead associated with Full CG garbage collections. The -XX:+AggressiveHeap option tunes internal VM parameters to be optimal for long-running, memory-intensive workloads. This option should follow, on the command line, the heap sizing options -Xms and -Xmx. See "Ensure Sufficient Java Heap for OC4J" for details on setting Java command line options in an Oracle Application Server managed environment (please disregard older versions of Sun documentation which advise against using -XX:+AggressiveHeap).


Note:

The JVM provides a variety of parameters to allow you to more finely tune heap management and garbage collection behavior. See the following links for a detailed description of these topics.

To determine if an application uses explicit garbage collection, which can have a negative impact on performance, set the -XX:+DisableExplicitGC option. This debugging option disables explicit garbage collection. Applications should avoid the use of system.gc() calls. If you suspect an application may be explicitly triggering garbage collection, set this parameter and observe the differences in your garbage collection behavior. If you determine that the application is making explicit system.gc() calls, discuss with the application developer why this was done and the impact of disabling the calls. Application developers sometimes use system.gc() calls to trigger finalizers. This is not a recommended practice and can yield indeterminate behavior.

Take the following steps to change the JVM command line options:

  1. Navigate to the Home page for the OC4J instance.

  2. Click Administration.

  3. If necessary, expand the Properties section of the table by clicking the Expand icon. Then, click the Go to Task icon in the Server Properties row.

  4. In the Command Line Options area, modify or change the appropriate command line options in the Options table.

  5. Click Apply.

  6. Navigate to the Cluster Topology page, select the OC4J instance that you modified, and click Restart. On the Confirmation page, click Yes.

3.1.4 Reuse Database Connections

To obtain better performance in your application, by lowering the overhead of creating and recreating database connections, specify the connection pool min-connections attribute to set the minimum number of connections that the connection pool maintains.

By default, the value of min-connections is 0. For best performance, you should specify a value for min-connections other than 0. If min-connections is set to a value other than zero, the specified number of connections is maintained; OC4J maintains the connections when they are not in use and they do not time out when the specified inactivity-timeout is reached. The min-connections attribute does not specify that OC4J pre-create connections at startup. Specify the initial-limit attribute to set the number of connections in the connection pool when the pool is initially created or reinitialized. Oracle recommends that you set the initial-limit attribute to the same value as the min-connections attribute.

If the specified value for min-connections is less than max-connections, then you should set the inactivity-timeout to make sure that connections only time out after an appropriately long period of inactivity. The connection pool inactivity-timeout specifies the time, in seconds, to cache unused connections before closing the connection.

To improve performance, you can set the inactivity-timeout to a value that allows the connection pool to avoid dropping and then reacquiring connections while your J2EE application is running. The default value for the inactivity-timeout is 60 seconds, which is typically too low for frequently accessed applications where there may be some inactivity between requests. For most applications, to improve performance, Oracle recommends that you increase the inactivity-timeout to 120 seconds.

To determine if the default inactivity-timeout is too low, monitor your system. If you see that the number of database connections grows and then shrinks during an idle period, and grows again soon after that, you have two options: you can increase the inactivity-timeout, or you can increase the min-connections.

Notes for reusing database connections:

  • Limiting the total number of open database connections to a number your database can handle is an important tuning consideration. You should check with your database administrator to make sure that the database is configured to support a number of connections that is greater than the following:

    At least as large a number of connections as the sum of the values specified for all the connection pool min-connections that could be concurrently active, and as large as the maximum desired concurrency across all the datasources for the database.

  • If the min-connections is set to a value other than zero, the specified number of connections is maintained; OC4J maintains the connections when they are not in use, and they do not time out when the specified inactivity-timeout is reached.

    Once the specified connections are opened, you need to either stop OC4J or use the refresh operation to close the connections. Application Server Control shows the refresh operation in Connection Pool area on the JDBC Resources page. Click the icon in the Refresh Connection Pool field to initiate a refresh operation.

3.1.5 Specify Sufficient Oracle HTTP Server Connections

The Oracle HTTP Server MaxClients directive limits the number of clients that can simultaneously connect to your web server, and thus the number of httpd processes.

For Windows, the analogous parameter is ThreadsPerChild. The Oc4jCacheSize directive specifies the maximum number of idle connections that mod_oc4j maintains per OC4J JVM, relevant only on Windows.

You can use the MaxClients, ThreadsPerChild, and Oc4jCacheSize directives to limit incoming connections to the OC4J instances from the Oracle HTTP Server. This section covers the following topics:


Note:

The discussion in this section only applies for the default Oracle HTTP Server supplied with Oracle Application Server (based on Apache 1.3). This discussion does not apply for the Apache 2.0 based standalone version of Oracle HTTP Server.

3.1.5.1 Configuring the MaxClients Directive (for UNIX)

You can configure the MaxClients directive in the httpd.conf file up to a maximum of 8K (the default value is 150). If your system is not resource-saturated and you have a user population of more than 150 concurrent HTTP connections, you can improve your performance by increasing MaxClients to increase server concurrency. Increase MaxClients until your system becomes fully utilized (85% is a good threshold).

When system resources are saturated, increasing MaxClients does not improve performance. In this case, the MaxClients value could be reduced as a throttle on the number of concurrent requests on the server.

If the server handles persistent connections, then it may require sufficient concurrent httpd server processes to handle both active and idle connections. When you specify MaxClients to act as a throttle for system concurrency, you need to consider that persistent idle httpd connections also consume httpd processes. Specifically, the number of connections includes the currently active persistent and non-persistent connections and the idle persistent connections. A persistent, KeepAlive, HTTP connection consumes an httpd child process, or thread, for the duration of the connection, even if no requests are currently being processed for the connection.

If you have sufficient capacity, KeepAlive should be enabled; using persistent connections improves performance and prevents wasting CPU resources reestablishing HTTP connections. Normally, you should not need to change KeepAlive parameters.


Note:

The default maximum requests for a persistent connection is 100, as specified with the MaxKeepAliveRequests directive in httpd.conf. By default, the server waits for 15 seconds between requests from a client before closing a connection, as specified with the KeepAliveTimeout directive in httpd.conf.

When there are no httpd processes available, connection requests are queued in the TCP/IP system until a process becomes available, and eventually clients terminate connections.

3.1.5.2 Configuring the ThreadsPerChild Directive (for Windows)

You can configure the ThreadsPerChild directive in the httpd.conf file up to a maximum of 8K (the default value is 50). The ThreadsPerChild parameter on Windows systems works like the MaxClients parameter on UNIX systems.

3.1.5.3 Configuring the Oc4jCacheSize Directive

The Oc4jCacheSize directive specifies the maximum number of idle connections that mod_oc4j maintains per OC4J JVM. On Windows only, it is sometimes useful to change the default value of this directive.

On UNIX systems where each Oracle HTTP Server process is single threaded, the only meaningful values are 1 which is the default value, and zero (0). A value of zero (0) specifies that Oracle HTTP Server should not maintain any connections and should open a new connection for every request. Since each process is single threaded, a process never needs more than one connection and hence a value of 1 or greater has the same effect on UNIX systems. For best performance, on UNIX systems, do not change the default value for Oc4jCacheSize.

On Windows systems, the default Oc4jCacheSize value is 75% of the value of ThreadsPerChild; the connection cache is shared among threads in the child process. If Oracle HTTP Server is serving a mixed load of static content along with OC4J requests, then the default should be adequate. If the user's load is all OC4J requests, that is, Oracle HTTP Server serves up little or no content and serves just as a front end for OC4J, then it is a good idea to set Oc4jCacheSize equal to ThreadsPerChild. This setting provides a dedicated connection per thread, if needed, and should give the best performance.

3.1.6 Enable Statement Caching for Data Sources

Enable statement caching to lower the overhead of repeated cursor creation and repeated statement parsing and creation by setting the num-cached-statements attribute to a value greater than 0 (the default value is 0, disabled). The number you set for num-cached-statements should be the number of SQL statements that you use in your application.

3.1.7 Verify Database Tuning

To achieve optimal performance in Oracle Application Server, for applications that use the database, the database tables you access need to be designed with performance in mind and you need to monitor and tune the database server to assure that the system is performant.

This section covers the following:


See Also:

Oracle Database Performance Tuning Guide

3.1.7.1 Tuning init.ora Database Parameters

Table 3-2 shows tuning information for several of the init.ora database initialization parameters.

Table 3-2 Important init.ora Database Tuning Parameters

nit.ora Parameter Description

DB_BLOCK_SIZE

The default block size of 8K is optimal for most systems. However, OLTP systems occasionally benefit from smaller block sizes, and DSS systems occasionally benefit from larger block sizes.

See Also: table 8-3, "Block Size Advantages and Disadvantages" in the Oracle Database Performance Tuning Guide.

PGA_AGGREGATE_TARGET

Specifies the target aggregate PGA memory available to all server processes attached to the instance.

See Also: "Memory Configuration and Use" in the Oracle Database Performance Tuning Guide for information on PGA memory management.

PROCESSES

Sets the maximum number of operating system processes that can be connected to Oracle concurrently. The value of this parameter must be 6 or greater (5 for the background processes plus 1 for each user process). For example, if you plan to have 50 concurrent users, set this parameter to at least 55. Many other initialization parameter values are deduced from this value.

SGA_MAX_SIZE

This parameter is the maximum size of the SGA for a running instance. Set this parameter to the amount of memory that you want dedicated for the SGA, which includes the following memory pools:

  • Database buffer cache

  • Shared pool

  • Large pool

  • Java pool

It is a good practice to regularly monitor the buffer cache hit ratio and size the SGA so that the buffer cache has an adequate number of frames for the workload. The buffer cache hit ratio may be calculated from data in the view V$SYSSTAT. Also the view V$DB_CACHE_ADVICE provides data that can be used to tune the buffer cache.

See Also: the chapter, "Memory Configuration and Use" in the Oracle Database Performance Tuning Guide for detailed information on how to set the SGA_MAX_SIZE parameter, including on how to use the V$SYSSTAT and V$DB_CACHE_ADVICE views to optimize the buffer cache hit ratio.

SGA_TARGET

Setting this parameter to a nonzero value enables Automatic Shared Memory Management. Oracle strongly recommends the use of automatic memory management, both to simplify configuration and to improve performance. Automatic Shared Memory Management was introduced with the Oracle Database 10g (10.1). For prior versions, you must manually configure individual SGA memory pools.

See Also: the section, "Automatic Shared Memory Management" in the Chapter, "Memory Configuration and Use" in the Oracle Database Performance Tuning Guide for details on chosing a value for the SGA_TARGET parameter.

UNDO_TABLESPACE

UNDO_MANAGEMENT

Oracle strongly recommends that you use automatic undo management (UNDO_MANAGEMENT = AUTO) and manage undo space using an UNDO_TABLESPACE. For backward compatibility reasons, the default value of UNDO_MANAGEMENT is MANUAL.

See Also: Oracle Database Performance Tuning Guide for additional information on undo space management.


3.1.7.2 Tuning Redo Logs Location and Sizing

Managing the database I/O load balancing is a non-trivial task. However, tuning the redo log options can provide performance improvement for applications running in an Oracle Application Server environment, and in some cases, you can significantly improve I/O throughput by moving the redo logs to a separate disk.

The size of the redo log files can also influence performance, because the behavior of the database writer and archiver processes depend on the redo log sizes. Generally, larger redo log files provide better performance by reducing checkpoint activity. It is not possible to provide a specific size recommendation for redo log files, but redo log files in the range of a hundred megabytes to a few gigabytes are considered reasonable. Size your online redo log files according to the amount of redo your system generates. A rough guide is to switch logs at most once every twenty minutes. Set the initialization parameter LOG_CHECKPOINTS_TO_ALERT = true to have checkpoint times written to the alert file.

The complete set of required redo log files can be created during database creation. After they are created, the size of a redo log size cannot be changed. However, new, larger files can be added later, and the original (smaller) ones can subsequently be dropped.


See Also:

The chapters, "Configuring a Database for Performance" and "I/O Configuration and Design" in the Oracle Database Performance Tuning Guide

3.1.7.3 Automatic Segment-Space Management (ASSM)

For permanent tablespaces, Oracle recommends using automatic segment-space management. Such tablespaces, often referred to as bitmap tablespaces, are locally managed tablespaces with bitmap segment space management.

For backward compatibility, the default local tablespace segment-space management mode is MANUAL.


See Also:

Oracle Database Concepts for a discussion of free space management, and Oracle Database Administrator's Guide for more information on creating and using automatic segment-space management for tablespaces.

3.1.8 Verify Logging Levels

You need to assure that application and server logging levels are set appropriately, and that debugging properties or other application level debugging flags are set to appropriate levels or disabled. Set Oracle Application Server OC4J logger log levels to log messages at the INFO level (do not set log levels to levels that produce more diagnostic message, including the FINE or TRACE levels).

To configure OC4J component loggers through Application Server Control Console, do the following from the OC4J home page:

  1. Click the Administration link

  2. In the table, under Properties, click the task for Logger Configuration.

  3. In the table, under Loggers, set the root Log Level to the desired value, or expand the tree to select individual Log Levels for specified loggers.

  4. Click Apply to apply your changes to the OC4J runtime.

3.1.9 Reuse EJB Instances

This section describes EJB tuning options for creating and reusing instances to improve EJB performance; these options are specific to OC4J and apply for all types of EJBs (except Stateful Session EJBs). You can configure these options by setting attributes in orion-ejb-jar.xml.

The min-instances attribute specifies the minimum number of bean implementation instances to be kept instantiated or pooled. The default value is 0. For best performance, you should specify a value for min-instances other than 0. If min-instances is > 0, OC4J maintains the min-instances number of instances in the pool when they are not in use. For instances above the min-instances, the instances are removed from the pool after the pool-cache-timeout specified timeout expires. The pool-cache-timeout cache expiration removes all bean instances that have not been accessed during the timeout window. For example, with the default value for pool-cache-timeout, all beans that have not been touched in 60 seconds would be removed from the pool and active or recently used beans are left in the pool.

The default value for the pool-cache-timeout is 60 seconds, which is typically too low for frequently accessed EJBs. If the pool-cache-timeout is 0 or negative, then the pool-cache-timeout is disabled and beans are not removed from the pool.

For performance tuning, try to reduce the frequency of the removal of beans from the pool by setting the pool-cache-timeout to a large value. You should set the pool-cache-timeout to a large enough value to allow OC4J to avoid destroying and then re-creating instances while your J2EE application is running.

3.2 Advanced Performance Areas

This section describes areas that can provide improved performance for some usage cases and environments.

This section covers the following topics:

3.2.1 Managing Concurrency and Limiting Connections

Oracle Application Server lets you limit concurrency at multiple layers of the system to match specific usage needs. In addition to controlling HTTP connections, you can control concurrency at additional levels of the product to meet specific usage requirements.

This section covers the following topics:

3.2.1.1 Using OC4J Thread Pools to Control Concurrency

By default, OC4J creates thread pools, as described in the following list. New threads are created and added to the pools on an as-needed basis.

  • http thread pool: A thread pool serving HTTP and AJP requests and possibly RMI requests (if rmi request thread pool is not configured) and RMI connections (if rmi connection thread pool is not configured). The default behavior provides for a maximum of 1024 threads, and threads for this pool are created on demand.

  • jca thread pool: contains the pool of threads that OC4J reserves specifically for use by deployed resource adapters. The default behavior provides for a maximum of 1024 threads, and threads for this pool are created on demand.

  • system thread pool: contains internal OC4J threads. The default behavior provides for a maximum of 1024 threads and threads for this pool are created on demand.

Thread pools create and store threads for use and reuse by an OC4J process. The default thread pool management configuration should be sufficient for most common usage scenarios. Reusing existing threads from a thread pool improves performance and reduces the burden on the JVM and on the underlying operating system.


Note:

When you use the thread pool management options, this is considered an expert-mode task. If you modify the default thread pool configuration, it is important to consider that the resulting concurrency is determined by the sum of the threads in all thread pools for the OC4J process.

There are cases where changing the default behavior, by specifying thread pool management options, can improve performance, including the following:

  • A hardware resource is nearing maximum capacity on Oracle Application Server 10g. For example, consider the case where a site has a user population of 1000 users with the site often processing 20 concurrent requests; at this rate, the site is nearing full resource utilization. Occasionally, the site needs to handle peaks of 75 to 100 concurrent requests. Specifying a thread pool with a maximum number of threads value in the range of 20 to 25 is likely to yield better overall results (setting the value in the range of 75 to 100 is unlikely to improve peak performance, since at this level there are no additional resources available to service the threads).

  • A hardware resource is nearing maximum capacity on the database or the database is a bottleneck due to other limitations, and you want to limit connections to the database using OC4J thread control (in addition, you can limit database connections using the datasource max-connections parameter).

Thread pool tuning does not improve performance in the following situations:

  • There are sufficient hardware resources available on the system when the system is under maximum load and there are no other known software issues such as database locking problems. This includes both Oracle Application Server 10g tier and database systems. For example, consider the case where a site has a user population of 1000 users but typically only sees a few concurrent requests. In this case, using application server thread pool tuning to limit threads is not advantageous, since the number of threads is determined by the concurrent request rate, which for this case is very low.

  • There already are sufficient concurrency limits specified elsewhere in Oracle Application Server 10g or in the database systems. For example when the Oracle HTTP Server MaxClients directive is set to control concurrency, at the HTTP server level, or when the data source max-connections attribute is set to control concurrency for connections to the database.


Note:

If the number of incoming requests is consistently higher than the request rate that the physical hardware can support, consider increasing the physical resources at the site. Likewise, if the response time at peak periods is unacceptable, then you may need to increase the hardware configuration to remedy this situation.

3.2.1.1.1 Controlling and Using Application Server Thread Pools

This section describes how to manage and use the thread pool configuration options.

You can manage application server thread pools as follows:

  • Adding the <thread-pool> element with appropriate attributes in server.xml. For example,

    <thread-pool name="http" min="120" max="120" queue="240" \>
    
    
  • Updating the attributes in the ThreadPool MBean, which is accessible through the system MBean Browser in Application Server Control Console.

  • Using the Thread Pool Configuration page in Application Server Control Console which is accessible from the OC4J Home page by selecting the Administration secondary tab and then clicking the Thread Pool Configuration task (see Figure 3-1).

Figure 3-1 Application Server Control Console Thread Pool Configuration

Application server control thread pool configuration screen
Description of "Figure 3-1 Application Server Control Console Thread Pool Configuration"

Notes for specifying thread pool options:

  • Oracle recommends setting the min value (Minimum Pool Size) equal to the max value (Maximum Pool Size). When min=max, you should set the queue to a value equal to the maximum concurrent number of requests you expect. For example, if you are using Oracle HTTP Server with one OC4J instance and no direct RMI connections, then the value of MaxClients would represent your maximum concurrency. Note: if you set MaxClients to a very large number, a setting of 300 or less for the queue size is probably sufficient. See "Validating and Monitoring Thread Pool Performance" for instructions on monitoring your thread pool and queues.

    Oracle recommends starting with a smaller number of threads, for example, using Oracle Application Server 10g Release 3 (10.1.3.1.0) with the Basic install option, set min and max to a value in the range of 80 to 100 and then monitor your resulting performance. In a standalone OC4J configuration, set min and max to a value in the range of 15 to 40, and then monitor your performance. If the site has sufficient CPU and memory resources available, and the concurrent request rate is higher than the current thread count setting, try increasing the number of threads. If you observe that the system is resource saturated, then try reducing the number of threads. Specific thread settings depend on the characteristics of your application.

  • A recommended starting value for setting thread pool options is to set the min value equal to the max value. However, if you set a larger max value to accommodate occasional peaks of traffic, OC4J will add a new thread on each request until the minimum number of threads are created. After the minimum number of threads are created, new threads will not be created until the queue is full. OC4J attempts to keep the number of threads at or near the minimum, unless the queue is full. If you set the min value less than the max value, it is generally advisable to keep the queue size small; the queue default value is 0, meaning requests are not queued and a new thread will be created if the number of threads is less than the specified max value. If you do not see your threads increasing beyond the minimum value, then you should decrease the queue size.


See Also:

Oracle Containers for J2EE Configuration and Administration Guide for more details on using the application server thread pool

3.2.1.1.2 Using RMI Connection and RMI Request Thread Pools

OC4J supports a thread pool for RMI connection threads. By default the RMI connection threads are allocated from the application server thread pool named http. Using the RMI connection thread pool provides a separate thread pool whose threads block-read on the RMI connections. Defining an RMI connection thread pool puts a separate limit on the RMI connections; they are not taken from the http thread pool. Specify the RMI connection thread pool, using the rmi connection name in the <thread-pool> element in server.xml, to separate potentially long-lived RMI connection threads from the threads used for application work. This configuration allows you to free other threads to do work, instead of being allocated to long lived RMI connections.

When an RMI connection thread pool is defined, the work from the RMI connections executes either from threads in the http thread pool, or if you also specify an RMI request thread pool, then the work from the RMI connections executes from threads in the RMI request thread pool (you specify the RMI request thread pool with the name rmi request specification in a <thread-pool> element in server.xml). Specify a RMI request thread pool if you want to control the work associated with the RMI connections, separately from other work (for example HTTP requests).


Note:

Specifying the rmi connection thread pool and the rmi request thread pool is only needed if you anticipate many concurrent RMI connections, where the number varies greatly and you are trying to bound the active worker threads.

3.2.1.1.3 Using the Work Manager (JCA) Thread Pool

Starting with 10g Release 3 (10.1.3), EJBs of type MDB use receiver threads with the JMS Resource Adapter. OC4J allocates these threads and threads for other JCA resource adapters from the work manager thread pool (jca thread pool). However, you need to consider receiver threads to control the overall concurrency on the system. Oracle recommends leaving the jca thread pool at the default setting (by default, JCA work manager threads are limited to a maximum value of 1024). When you want to control the concurrency for EJB MDBs, use the JMS ReceiverThreads maximum values. The overall concurrency limit on your system includes the max threads specified for the http thread pool, plus the sum of all the MDB receiverThreads configured for your applications deployed to OC4J, or the max jca threads if smaller (plus the RMI connection max threads and the RMI request max threads, if they are configured).

3.2.1.1.4 Validating and Monitoring Thread Pool Performance

You can use the Application Server Control Console Current Pool Size and Current Queue Size metrics to observe OC4J thread use on your system. You can access these metrics from OC4J home page by clicking the Administration secondary tab and then clicking the Thread Pool Configuration task (see Figure 3-1). The Current Pool Size shows the current number of threads in the pool. The Current Queue Size shows the current number of requests waiting in the queue for a thread to become available. Oracle recommends starting with the default settings and observing the behavior on your system. You should also observe these metrics after changing the thread pool or queue size limits for your instance.

3.2.1.2 Setting the Maximum Number of Connections for Data Sources

For applications that use a database, performance can improve when the connection pool associated with a data source limits the number of connections. You can use the max-connections attribute to limit the database requests from Oracle Application Server so that incoming requests do not saturate the database, or to limit the database requests so that the database access does not overload the Oracle Application Server-tier resource.

The connection pool max-connections attribute specifies the maximum number of connections that a connection pool allows. By default, the value of max-connections is set to -1 (unlimited). For best performance, you should specify a value for max-connections that matches the number appropriate to your database performance characteristics.

Limiting the total number of open database connections to a number your database can handle is an important tuning consideration. You should check to make sure that your database is configured to allow at least as large a number of open connections as the total of the values specified for all the data sources max-connections option, as specified in all the applications that access the database.

3.2.1.3 Controlling the Number of EJB Instances When Using EJBs

You may want to limit the number of EJB instances to reduce memory usage or to control concurrency to reduce contention on resources that the EJBs use (for example a data source).

The max-instances parameter specifies the number of bean instances allowed in memory – either instantiated or pooled.

  • For all types of EJBs except stateful session beans, when the max-instances value is reached, and a new EJB is requested, the container waits the number of milliseconds set in the call-timeout attribute to see if a bean instance becomes available in the pool. If no bean instance is available in the pool then a TimeoutExpiredException is thrown back to the client.

  • For stateful session beans, when the max-instances value is reached, the container attempts to passivate the oldest bean instance from memory. If unsuccessful, the container waits the number of milliseconds set in the call-timeout attribute to see if a bean instance is removed from memory, either through passivation, using the remove() method, or by bean expiration before a TimeoutExpiredException is thrown back to the client.

To allow an unlimited number of bean instances, use max-instances = 0 (the default value is 0).

Set max-instances < 0, for example to -1, to disable instance pooling. In this case OC4J creates a new bean instance or context when starting the EJB call, and releases the context and throws the instance away to Non-existence state at the end of the call.

The exception, com.evermind.server.ejb.TimeoutExpiredException: timeout expired waiting for an instance, occurs when there is no available EJB instance. To avoid this problem set the max-instances and call-timeout parameters appropriately.

3.2.1.4 Limiting Remote EJB Client Connections

To limit remote EJB client connections you can change the max value for the http thread pool to specify limits for all threads, or you can use the thread pool features that control the maximum number of threads that service incoming EJB clients. By default, the threads for remote EJB client connections come from the http thread pool. When you want use the RMI connection thread pool, configure the rmi connection name in the <thread-pool> element in server.xml. By setting the max value of the RMI connection thread pool, you can limit remote EJB client connections.


See Also:

Oracle Containers for J2EE Configuration and Administration Guide for more details on using the application server thread pool

3.2.2 Load Balancing

Oracle Application Server provides load-balancing features that spread the J2EE application load and incoming requests among multiple application server instances, which generally results in higher throughput and shorter response time. Using multiple application server instances with load-balancing allows you to improve performance by directing requests across the multiple application server instances. In addition, you can use multiple application server instances running on multiple hosts to handle high availability and failover needs.

This section covers the following topics:


Note:

The Oracle Application Server features that provide replication for failover with Web sessions and for stateful session EJBs have a performance overhead; only use these features if you need replication for failover.

3.2.2.1 Configuring Multiple Oracle Application Server Instances

This section covers the following:

3.2.2.1.1 Determining the Number of OC4J Processes

Determining the optimal ratio of OC4J processes to available CPUs is dependent on the characteristics of the applications you run, the OC4J configuration, the hardware configuration, and the type and number of expected incoming requests. In hardware configurations with a small number of CPUs, you may only need one OC4J instance.

Adding OC4J instances beyond the available resources of the system does not improve performance. For example, if one OC4J instance is sufficient to saturate the CPU resources of a system, adding additional OC4J instances is not likely to improve performance and may, in fact, degrade it. A good starting point is to initially configure one OC4J instance and measure the performance improvement from adding additional OC4J instances.


See Also:

  • Oracle Application Server High Availability Guide

  • Oracle Containers for J2EE Configuration and Administration Guide


3.2.2.1.2 Partitioning Applications into Different OC4J Instances

Partitioning different applications to be run under different OC4J instances, each of which has different requirements, may help improve the performance of your applications. In this case, you may want to configure different OC4J instances to service the different applications. After deploying the applications to different OC4J instances, you can monitor the performance to see if the overall throughput increases, or the response time decreases.

3.2.2.2 Web Application Load Balancing

In an Oracle Application Server environment, the Oracle HTTP Server uses mod_oc4j to load balance requests between the available OC4J instances. In this environment you can select mod_oc4j configuration options to choose the appropriate load balancing policies to improve performance. By default, the requests are routed using a roundrobin algorithm.

At many sites Oracle Application Server uses the Oracle HTTP Server module mod_oc4j to load balance incoming stateless HTTP requests. By selecting the appropriate load balancing policy for mod_oc4j you can improve performance on your site.The mod_oc4j module supports several configurable load balancing policies, including the following:

  • Round robin routing (this is the default mod_oc4j load balancing policy)

  • Random routing

  • Round robin or random with local affinity routing, using the local option

  • Round robin or random with host-level weighted routing, using the weighted option


Note:

For a session based request mod_oc4j always directs the request to the original OC4J which created the session, unless the original OC4J process is not available. In case of failure, mod_oc4j sends the request to another OC4J within the same group as the original request (either within same host if available, or on a different host).

Recommendations for Load Balancing with mod_oc4j:

  1. On a Single Host for both Oracle HTTP Server and OC4J, the default load balancing policy, round robin load balancing is recommended. Random load balancing typically gives comparable performance.

  2. With Oracle HTTP Server on a separate host from OC4J:

    • Using a Single OC4J host the default load balancing policy, round robin load balancing is recommended. Random load balancing typically gives comparable performance.

    • Using Multiple OC4J hosts, if all OC4J hosts provide comparable capacity, the default load balancing is recommended. Note that the number of OC4J processes started on a host will implicitly weight the number of requests sent to that host even with random or round robin load balancing. A host with 4 OC4J processes will receive 4 times as many requests as a host with 1 OC4J process.

    • With Multiple OC4J Hosts with varying hardware resources or capacity, you may wish to weight the number of requests sent to each host explicitly to match its capacity. In this case, use either the round robin with weighted option or the random with weighted option. If the hosts have comparable capacity, use simple random or round robin load balancing.

      For example, to configure the mod_oc4j module in Oracle HTTP Server to specify round robin with a routing weight of 3 for Host_A and a routing weight of 1 for Host_B, add the following directives to mod_oc4j.conf:

      Oc4jSelectMethod roundrobin:weighted
      Oc4jRoutingWeight   Host_A 3
      
      

      In this example, you do not need to specify a routing weight for Host_B, since the default routing weight is 1.

  3. With Multiple hosts with Oracle HTTP Server and OC4J on each host:

    • Multiple hosts with Oracle HTTP Server and OC4J on each host, and a hardware load balancer. Select the local affinity option to direct mod_oc4j to only select the local OC4J processes to service incoming requests. This will generally improve performance. When no local OC4J processes are available, mod_oc4j selects from the list of available remote OC4J processes.

      For example, to select the round robin policy with local affinity, specify the following directive in mod_oc4j.conf:

      Oc4jSelectMethod roundrobin:local
      
      

3.2.2.3 EJB Application Load Balancing

After an EJB application is deployed to multiple OC4J instances, an EJB client-side application can load balance its requests across the available OC4J instances. To use load balancing, the client-side application configures the JNDI properties to use load balancing. For good performance in some clients, you need to set oracle.j2ee.rmi.loadBalance=context to load balance for every initialcontext call, rather than only once for the entire client.


See Also:

  • "Configuring ORMI Request Load Balancing" in the Oracle Containers for J2EE Services Guide

  • "Understanding OC4J EJB Application Clustering Services" in the Oracle Containers for J2EE Enterprise JavaBeans Developer's Guide


3.2.3 Using the -XX:AppendRatio Option (on Standalone OC4J)

With the Sun 5.0 JVM, under some circumstances under heavy load, synchronization in an application can result in thread starvation. This may cause some requests for an application to appear hung or to timeout after a long time.

In 10g Release 3 (10.1.3.1.0) the parameter: -XX:AppendRatio=3 is specified by default for managed OC4J. For standalone OC4J, if you believe your installation has this problem, we recommend setting the JDK parameter: -XX:AppendRatio=3 to avoid this problem.


See Also:

See the SUN bug database for a description of this issue and the suggested workaround: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4985566