WebLogic Server Performance and Tuning
Java parameters must be specified whenever you start WebLogic Server. For simple invocations, this can be done from the command line with the
weblogic.Server command. However, because the arguments needed to start WebLogic Server from the command line can be lengthy and prone to error, BEA recommends that you incorporate the command into a script. To simply this process, you can modify the default values in the sample scripts that are provided with the WebLogic distribution to start WebLogic Server, as described in "Specifying Java Options for a WebLogic Server Instance".
If you used the Configuration Wizard to create your domain, the WebLogic startup scripts are located in the domain-name directory where you specified your domain. By default, this directory is BEA_HOME
\user_projects\domain\domain-name, where BEA_HOME is the directory that contains the product installation, and domain-name is the name of the domain directory defined by the selected configuration template. For more information about creating domains using the Configuration Wizard, see "Creating Domains Using the Configuration Wizard".
You need to modify some default Java values in these scripts to fit your environment and applications. The important performance tuning parameters in these files are the
JAVA_HOME parameter and the Java heap size parameters:
JAVA_HOMEto the location of your
JDK. For example:
See Specifying Heap Size Values for details about setting heap size options.
The WebLogic Server configuration file (
config.xml) contains a number of performance-related parameters that can be fine-tuned depending on your environment and applications. Tuning these parameters based on your system requirements (rather than running with default settings) can greatly improve both single-node performance and the scalability characteristics of an application.
Within a WebLogic Server domain, the configuration file is located on the machine that hosts the Administration Server, and provides persistent storage of WebLogic MBean attribute values. The Administration Server serves as a central point of contact for server instances and system administration tools. A domain may also include additional WebLogic Server instances called Managed Servers, which are used mainly for servicing applications.
When the Administration Server starts, it reads the domain configuration file and overrides the default attribute values of the administration MBeans with any attribute values found in the configuration file. Every time you change an attribute using the system administration tools (using either the command-line interface or the Administration Console), its value is stored in the appropriate administration MBean and written to the configuration file.
For more information about system administration infastructure, see "Overview of WebLogic Server System Administration" in the Administration Guide.
Table 4-1 lists the
config.xml file parameters that affect server performance.
You can indicate whether a domain is to be used in a development environment or a production environment. WebLogic Server uses different default values for various services depending on the type of environment you specify.
Table 4-2 lists the performance-related configuration parameters that differ when switching from development to production startup mode.
The tuning defaults discussed in throughout WebLogic Performance and Tuning Guide refer to the "development mode" defaults, which is the default startup mode when WebLogic Server is installed. For information on switching the startup mode from development to production, see Changing the Runtime Mode in the Administration Console Online Help.
For a complete listing of the differences between development and production startup modes, see the "Differences Between Configuration Startup Modes" section in Creating WebLogic Configurations Using the Configuration Wizard.
Benchmarks show major performance improvements when you use native performance packs on machines that host WebLogic Server instances. Performance packs use a platform-optimized, native socket multiplexor to improve server performance. For example, the native socket reader multiplexor threads have their own execute queue and do not borrow threads from the default execute queue, which frees up default execute threads to do application work.
However, if you must use the pure-Java socket reader implementation for host machines, you can still improve the performance of socket communication by configuring the proper number of socket reader threads for each server instance and client machine.
The use of native performance packs are enabled by default in the
config.xml shipped with your distribution. To verify this setting in your configuration file, check that the
NativeIOEnabled attribute of the
Server element is set to "true" (
The value of the
ThreadCount attribute of an
ExecuteQueue element in the
config.xml file equals the number of simultaneous operations that can be performed by applications that use the execute queue. As work enters an instance of WebLogic Server, it is placed in an execute queue. This work is then assigned to a thread that does the work on it. Threads consume resources, so handle this attribute with care—you can degrade performance by increasing the value unnecessarily.
By default, a new WebLogic Server instance is configured with a development mode execute queue,
weblogic.kernel.default, that contains 15 threads. In addition, WebLogic Server provides two other pre-configured queues:
weblogic.admin.HTTP—Available only on Administration Servers, this queue is reserved for communicating with the Administration Console; you cannot reconfigure it.
weblogic.admin.RMI—Both Administration Servers and Managed Servers have this queue; it is reserved for administrative traffic; you cannot reconfigure it.
Note: If native performance packs are not being used for your platform, you may need to tune the default number of execute queue threads and the percentage of threads that act as socket readers to achieve optimal performance. For more information, see Allocating Execute Threads to Act as Socket Readers.
Adding more threads to the default execute queue does not necessarily imply that you can process more work. Even if you add more threads, you are still limited by the power of your processor. Because threads consume memory, you can degrade performance by increasing the value of the
ThreadCount attribute unnecessarily. A high execute thread count causes more memory to be used and increases context switching, which can degrade performance.
The value of the
ThreadCount attribute depends very much on the type of work your application does. For example, if your client application is thin and does a lot of its work through remote invocation, that client application will spend more time connected — and thus will require a higher thread count — than a client application that does a lot of client-side processing.
If you do not need to use more than 15 threads (the development default) or 25 threads (the production default) for your work, do not change the value of this attribute. As a general rule, if your application makes database calls that take a long time to return, you will need more execute threads than an application that makes calls that are short and turn over very rapidly. For the latter case, using a smaller number of execute threads could improve performance.
To determine the ideal thread count for an execute queue, monitor the queue's throughput while all applications in the queue are operating at maximum load. Increase the number of threads in the queue and repeat the load test until you reach the optimal throughput for the queue. (At some point, increasing the number of threads will lead to enough context switching that the throughput for the queue begins to decrease.)
Note: The WebLogic Server Administration Console displays the cumulative throughput for all of a server's execute queues. To access this throughput value, follow steps 1-6 in Modifying the Default Thread Count.
Table 4-3 shows default scenarios for adjusting available threads in relation to the number of CPUs available in the WebLogic Server domain. These scenarios also assume that WebLogic Server is running under maximum load, and that all thread requests are satisfied by using the default execute queue. If you configure additional execute queues and assign applications to specific queues, monitor results on a pool-by-pool basis.
To determine the amount of stuck threads, see Tuning the Execute Thread Detection Behavior.
Although you can configure the default execute queue to supply the optimal number threads for all WebLogic Server applications, configuring multiple execute queues can provide additional control for key applications. By using multiple execute queues, you can guarantee that selected applications have access to a fixed number of execute threads, regardless of the load on WebLogic Server. See Using Execute Queues to Control Thread Usage for more information on assigning applications to configured execute queues.
For best socket performance, BEA recommends that you use the native socket reader implementation, rather than the pure-Java implementation, on machines that host WebLogic Server instances (see Using WebLogic Server "Native IO" Performance Packs). However, if you must use the pure-Java socket reader implementation for host machines, you can still improve the performance of socket communication by configuring the proper number of execute threads to act as socket reader threads for each server instance and client machine.
ThreadPoolPercentSocketReaders attribute sets the maximum percentage of execute threads that are set to read messages from a socket. The optimal value for this attribute is application-specific. The default value is 33, and the valid range is 1-99.
Allocating execute threads to act as socket reader threads increases the speed and the ability of the server to accept client requests. It is essential to balance the number of execute threads that are devoted to reading messages from a socket and those threads that perform the actual execution of tasks in the server.
On client machines, you can configure the number of available socket reader threads in the JVM that runs the client. Specify the socket readers by defining the following parameters in the
java command line for the client:
You can configure WebLogic Server to detect and optionally address potential overflow conditions in the default execute queue or any user-defined execute queue. WebLogic Server considers a queue to have a possible overflow condition when its current size reaches a user-defined percentage of its maximum size. When this threshold is reached, the server changes its health state to "warning" and can optionally allocate additional threads to perform the outstanding work in the queue, thereby reducing its size.
Queue Length: Specifies the maximum number of simultaneous requests that the server can hold in the queue. The default of 65536 requests represents a very large number of requests; outstanding requests in the queue should rarely, if ever reach this maximum value. Always leave the Queue Length at the default value of 65536 entries.
Queue Length Threshold Percent: The percentage (from 1-99) of the Queue Length size that can be reached before the server indicates an overflow condition for the queue. All actual queue length sizes below the threshold percentage are considered normal; sizes above the threshold percentage indicate an overflow. By default, the Queue Length Threshold Percent is set to 90 percent.
Thread Priority: The priority of the threads associated with the queue. By default, the Thread Priority is set to 5.
Threads Increase: The number of threads WebLogic Server should add to this execute queue when it detects an overflow condition. If you specify zero threads (the default), the server changes its health state to "warning" in response to an overflow condition in the execute queue, but it does not allocate additional threads to reduce the workload.
Threads Maximum: The maximum number of threads that this execute queue can have; this value prevents WebLogic Server from creating an overly high thread count in the queue in response to continual overflow conditions. By default, Threads Maximum is set to 400.
WebLogic Server automatically detects when a thread in an execute queue becomes "stuck." Because a stuck thread cannot complete its current work or accept new work, the server logs a message each time it diagnoses a stuck thread. If all threads in an execute queue become stuck, the server changes its health state to either "warning" or "critical" depending on the execute queue:
weblogic.admin.RMI, or a user-defined execute queue become stuck, the server changes its health state to "warning."
WebLogic Server diagnoses a thread as stuck if it is continually working (not idle) for a set period of time. You can tune a server's thread detection behavior by changing the length of time before a thread is diagnosed as stuck, and by changing the frequency with which the server checks for stuck threads.
Note: Although you can change the criteria WebLogic Server uses to determine whether a thread is stuck, you cannot change the default behavior of setting the "warning" and "critical" health states when all threads in a particular execute queue become stuck. For more information, see Overview of WebLogic Logging Services.
Stuck Thread Max Time: Enter the number of seconds, that a thread must be continually working before this server diagnoses the thread as being stuck. By default, WebLogic Server considers a thread to be "stuck" after 600 seconds of continuous use.
Stuck Thread Timer Interval: Enter the number of seconds, after which WebLogic Server periodically scans threads to see if they have been continually working for the length of time specified by
Stuck Thread Max Time. By default, WebLogic Server sets this interval to 600 seconds.
AcceptBacklog attribute of the
Server element in the
config.xml file to set the number of connection requests the WebLogic Server instance will accept before refusing additional requests. The
AcceptBacklog attribute specifies how many Transmission Control Protocol (TCP) connections can be buffered in a wait queue. This fixed-size queue is populated with requests for connections that the TCP stack has received, but the application has not accepted yet. The default value is 50 and the maximum value is operating system dependent.
Establishing a JDBC connection with a DBMS can be very slow. If your application requires database connections that are repeatedly opened and closed, this can become a significant performance issue. WebLogic connection pools offer an efficient solution to the problem.
When WebLogic Server starts, connections from the connection pools are opened and are available to all clients. When a client closes a connection from a connection pool, the connection is returned to the pool and becomes available for other clients; the connection itself is not closed. There is little cost to opening and closing pool connections.
How many connections should you create in the pool? A connection pool can grow and shrink according to configured parameters, between a minimum and a maximum number of connections. The best performance occurs when the connection pool has as many connections as there are concurrent client sessions.
In addition to the following subsections, see "Performance Tuning Your JDBC Application" in Programming WebLogic JDBC.
InitialCapacity attribute of the
JDBCConnectionPool element enables you to set the number of physical database connections to create when configuring the pool. If the server cannot create this number of connections, the creation of this connection pool will fail.
During development, it may be convenient to set the value of the
InitialCapacity attribute to a low number to help the server start up faster. In production systems, consider setting the
InitialCapacity value equal to the
MaxCapacity attribute's default production mode setting of 25. This way, all database connections are acquired during server start-up. And if you need to tune the
MaxCapacity value, make sure to set the
InitialCapacity so that it equals the
InitialCapacity is less than
MaxCapacity, the server needs to create additional database connections when its load is increased. When the server is under load, all resources should be working to complete requests as fast as possible, rather than creating new database connections.
MaxCapacity attribute of the
JDBCConnectionPool element allows you to set the maximum number of physical database connections that a connection pool can contain. Different JDBC drivers and database servers might limit the number of possible physical connections.
The default settings for development and production mode are equal to the default number of execute threads: 15 for development mode; 25 for production mode. However, in production, it is advisable that the number of connections in the pool equal the number of concurrent client sessions that require JDBC connections. The pool capacity is independent of the number of execute threads in the server. There may be many more ongoing user sessions than there are execute threads.
When you use a prepared statement or callable statement in an application or EJB, there is considerable processing overhead for the communication between the application server and the database server and on the database server itself. To minimize the processing costs, WebLogic Server can cache prepared and callable statements used in your applications. When an application or EJB calls any of the statements stored in the cache, WebLogic Server reuses the statement stored in the cache. Reusing prepared and callable statements reduces CPU usage on the database server, improving performance for the current statement and leaving CPU cycles for other tasks. For more details, see "Increasing Performance with the Statement Cache" in the Administration Console Online Help.
Using the statement cache can dramatically increase performance, but you must consider its limitations before you decide to use it. For more details, see "Usage Restrictions for the Statement Cache" in the Administration Console Online Help.
The standard Java compiler for compiling JSP servlets is
javac. You can improve performance significantly by setting your server's java compiler to
jikes instead of
javac. The following sections discuss this procedure and other compiler considerations.
compileCommandparameter to specify the Java compiler for compiling the generated JSP servlets.
precompileparameter to configure WebLogic Server to precompile your JSPs when WebLogic Server starts up.
For more information about setting your server's java compiler in the
weblogic.xml file, see the jsp-descriptor element.
weblogic.appc utility to compile EJB 2.0 and 1.1 container classes. If you compile
Jar files for deployment into the EJB container, you must use
weblogic.appc to generate the container classes. By default, ejbc uses the
javac compiler. For faster performance, specify a different compiler (such as Symantec
sj) using the
For more information, see "Implementing EJBs" in Programming WebLogic Enterpise JavaBeans.
-nativeflag to use native threads when starting the JVM.
A WebLogic Server cluster is a group of WebLogic Servers instances that together provide fail-over and replicated services to support scalable high-availability operations for clients within a domain. A cluster appears to its clients as a single server but is in fact a group of servers acting as one to provide increased scalability and reliability.
A domain can include multiple WebLogic Server clusters and non-clustered WebLogic Server instances. Clustered WebLogic Server instances within a domain behave similarly to non-clustered instances, except that they provide failover and load balancing. The Administration Server for the domain manages all the configuration parameters for the clustered and non-clustered instances.
For more information about clusters, see "Introduction to WebLogic Server Clustering".
Scalability is the ability of a system to grow in one or more dimensions as more resources are added to the system. Typically, these dimensions include (among other things), the number of concurrent users that can be supported and the number of transactions that can be processed in a given unit of time.
Given a well-designed application, it is entirely possible to increase performance by simply adding more resources. To increase the load handling capabilities of WebLogic Server, add another WebLogic Server instance to your cluster—without changing your application. Clusters provide two key benefits that are not provided by a single server: scalability and availability.
WebLogic Server clusters bring scalability and high-availability to J2EE applications in a way that is transparent to application developers. Scalability expands the capacity of the middle tier beyond that of a single WebLogic Server or a single computer. The only limitation on cluster membership is that all WebLogic Servers must be able to communicate by IP multicast. New WebLogic Servers can be added to a cluster dynamically to increase capacity.
A WebLogic Server cluster guarantees high-availability by using the redundancy of multiple servers to insulate clients from failures. The same service can be provided on multiple servers in a cluster. If one server fails, another can take over. The ability to have a functioning server take over from a failed server increases the availability of the application to clients.
Caution: Provided that you have resolved all application and environment bottleneck issues, adding additional servers to a cluster should provide linear scalability. When doing benchmark or initial configuration test runs, isolate issues in a single server environment before moving to a clustered environment.
In general, any operation that requires communication between the servers in a cluster is a potential scalability hindrance. The following sections provide information on issues that impact the ability to linearly scale clustered WebLogic servers:
In many cases where a cluster of WebLogic servers fails to scale, the database is the bottleneck. In such situations, the only solutions are to tune the database or reduce load on the database by exploring other options. See JDBC Application Tuning.
User session data can be stored in two standard ways in a J2EE application: stateful session EJBs or HTTP sessions. By themselves, they are rarely a impact cluster scalability. However, when coupled with a session replication mechanism required to provide high-availability, bottlenecks are introduced. If a J2EE application has Web and EJB components, you should store user session data in HTTP sessions rather than stateful session EJBs as HTTP session management provides more replication options than stateful session EJBs. See Managing Sessions.
Optimistic concurrency bean is updated, the EJB container sends a multicast message to other cluster members to invalidate their local copies of the bean. This is done to avoid optimistic concurrency exceptions being thrown by the other servers and hence the need to retry transactions. If updates to the EJBs are frequent, the work done by the servers to invalidate each other's local caches become a serious bottleneck.
ReadOnly with a read-write pattern—In this pattern, persistent data that would otherwise be represented by a single EJB are actually represented by two EJBs: one read-only and the other updateable. When the state of the updateable bean changes, the container automatically invalidates corresponding read-only EJB instance. If updates to the EJBs are frequent, the work done by the servers to invalidate the read-only EJBs becomes a serious bottleneck.
Similar to Invalidation of Entity EJBs, HTTP sessions can also be invalidated. This is not as expensive as entity EJB invalidation, since only the session data stored in the secondary server needs to be invalidated. BEA advises users to not invalidate sessions unless absolutely required.
In general, JNDI binds, unbinds and rebinds are expensive operations. However, these operations become a bigger bottleneck in clustered environments because JNDI tree changes have to be propagated to all members of a cluster. If such operations are performed too frequently, they can reduce cluster scalability significantly.
With multi-processor machines, additional consideration must be given to the ratio of the number of available CPUs to clustered WebLogic Server instances. Because WebLogic Server has no built-in limit to the number of server instances that reside in a cluster, large, multi-processor servers, such as Sun Microsystems' Sun Enterprise 10000, can potentially host very large clusters or multiple clusters.
In order to determine the optimal ratio of CPUs to WebLogic server instances, you must first ensure that an application is truly CPU-bound, rather than network or disk I/O-bound. Use the following steps to determine the optional ratio of CPUs to server instances:
If you discover that an application is primarily network I/O-bound, consider measures to increase network throughput before increasing the number of available CPUs. For truly network I/O-bound applications, installing a faster network interface card (NIC) may increase performance more than additional CPUs, because most CPUs would remain idle while waiting to read available sockets.
The tool for monitoring the health and performance of your WebLogic Server domain is the Administration Console. Using the Administration Console, you can view status and statistics for WebLogic Server resources such as servers, HTTP, the JTA subsystem, JNDI, security, CORBA connection pools, EJB, JDBC, and JMS.
For more details, see "Monitoring a WebLogic Server Domain in Configuring and Managing WebLogic Server.
For example, there is a Server
For more details, see "Server -> Monitoring -> Performance in the Administration Console Online Help.