2 Top Performance Areas

This chapter describes the top tuning areas for Oracle Fusion Middleware. It covers critical Oracle Fusion Middleware performance areas and provides a quick start for tuning Java applications.

This chapter contains the following sections:

Section 2.1, "Identifying Top Performance Areas"
Section 2.2, "Securing Sufficient Hardware Resources"
Section 2.3, "Tuning the Operating System"
Section 2.4, "Tuning Java Virtual Machines (JVMs)"
Section 2.5, "Tuning the WebLogic Server"
Section 2.6, "Tuning Database Parameters"
Section 2.7, "Reusing Database Connections"
Section 2.8, "Enabling Data Source Statement Caching"
Section 2.9, "Controlling Concurrency"
Section 2.10, "Setting Logging Levels"

2.1 Identifying Top Performance Areas

One of the most challenging aspects of performance tuning is knowing where to begin. This chapter serves as a 'quick start' guide to performance tuning your Oracle Fusion Middleware applications.

Table 2-1 provides a list of common performance considerations for Oracle Fusion Middleware. While the list is a useful tool in starting your performance tuning, it is not meant to be comprehensive list of areas to tune. You must monitor and track specific performance issues within your application to understand where tuning can improve performance. See Chapter 4, "Monitoring Oracle Fusion Middleware" for more information.

Table 2-1 Top Performance Areas for Oracle Fusion Middleware

Performance Area	Description and Reference
Hardware Resources	Ensure that your hardware resources meet or exceed the application's resource requirements to maximize performance. See Section 2.2, "Securing Sufficient Hardware Resources" for information on how to determine if your hardware resources are sufficient.
Operating System	Each operating system has native tools and utilities that can be useful for monitoring purposes. See Section 2.3, "Tuning the Operating System"
Java Virtual Machines (JVMs)	This section discusses best practices and provides practical tips to tune the JVM and improve the performance of a Java EE application. It also discusses heap size and JVM garbage collection options. See Section 2.4, "Tuning Java Virtual Machines (JVMs)".
Database	For applications that access a database, ensure that your database is properly configured to support your application's requirements. See Section 2.6, "Tuning Database Parameters" for more information on garbage collection.
WebLogic Server	If your Oracle Fusion Middleware applications are using the WebLogic Server, see Section 2.5, "Tuning the WebLogic Server".
Database Connections	Pooling the connections so they are reused is an important tuning consideration. See Section 2.7, "Reusing Database Connections"
Data Source Statement Caching	For applications that use a database, you can lower the performance impact of repeated statement parsing and creation by configuring statement caching properly. See Section 2.8, "Enabling Data Source Statement Caching"
Oracle HTTP Server	Tune the Oracle HTTP Server directives to set the level of concurrency by specifying the number of HTTP connections. See Section 2.9, "Controlling Concurrency".
Concurrency	This section discusses ways to control concurrency with Oracle Fusion Middleware components. See Section 2.9, "Controlling Concurrency"
Logging Levels	Logging levels are thresholds that a system administrator sets to control how much information is logged. Performance can be impacted by the amount of information that applications log therefore it is important to set the logging levels appropriately. See Section 2.10, "Setting Logging Levels".

2.2 Securing Sufficient Hardware Resources

A key component of managing the performance of Oracle Fusion Middleware applications is to ensure that there are sufficient CPU, memory, and network resources to support the user and application requirements for your installation.

No matter how well you tune your applications, if you do not have the appropriate hardware resources, your applications cannot reach optimal performance levels. Oracle Fusion Middleware has minimum hardware requirements for its applications and database tier. For details on Oracle Fusion Middleware supported configurations, see "Verifying Certification, System Requirements, and Interoperability" in Planning an Installation of Oracle Fusion Middleware.

Sufficient hardware resources should meet or exceed the acceptable response times and throughputs for applications without becoming saturated. To verify that you have sufficient hardware resources, you should monitor resource utilization over an extended period to determine if (or when) you have occasional peaks of usage or whether a resource is consistently saturated. For more information on monitoring, see Chapter 4, "Monitoring Oracle Fusion Middleware".

Tip:

Your target CPU usage should never reach 100% utilization. You should determine a target CPU utilization based on your application needs, including CPU cycles for peak usage.

If your CPU utilization is optimized at 100% during normal load hours, you have no capacity to handle a peak load. In applications that are latency sensitive and maintaining a fast response time is important, high CPU usage (approaching 100% utilization) can increase response times while throughput stays constant or even decreases. For such applications, a 70% - 80% CPU utilization is recommended. A good target for non-latency sensitive applications is about 90%.

If any of the hardware resources are saturated (consistently at or near 100% utilization), one or more of the following conditions may exist:

The hardware resources are insufficient to run the application.
The system is not properly configured.
The application or database must be tuned.

For a consistently saturated resource, the solutions are to reduce load or increase resources. For peak traffic periods when the increased response time is not acceptable, consider increasing resources or determine if there is traffic that can be rescheduled to reduce the peak load, such as scheduling batch or background operations during slower periods.

Oracle Fusion Middleware provides a variety of mechanisms to help you control resource concurrency; this can limit the impact of bursts of traffic. However, for a consistently saturated system, these mechanisms should be viewed as temporary solutions. For more information see Section 2.9, "Controlling Concurrency".

2.3 Tuning the Operating System

Each operating system has native tools and utilities that can be useful for monitoring and tuning purposes. Native operating system commands enable you to monitor CPU utilization, paging activity, swapping, and other system activity information.

For details on operating system commands, and guidelines for performance tuning of the network or operating system, refer to the documentation provided by the operating system vendor.

2.4 Tuning Java Virtual Machines (JVMs)

How you tune your Java virtual machine (JVM) greatly affects the performance of Oracle Fusion Middleware and your applications. For more information on tuning your JVM, see "Tuning Java Virtual Machines (JVM)" in Tuning Performance of Oracle WebLogic Server.

2.5 Tuning the WebLogic Server

If your Oracle Fusion Middleware applications are using the WebLogic Server, see "Tuning WebLogic Server" in Tuning Performance of Oracle WebLogic Server.

2.6 Tuning Database Parameters

To achieve optimal performance for applications that use the Oracle database, the database tables you access must be designed with performance in mind. Monitoring and tuning the database ensures that you get the best performance from your applications.

Note:

The information in this section is a subset of database tuning information for Fusion Middleware. Make sure that you have also reviewed the Database Performance Tuning Guide.

This section covers the following:

Tuning Database Parameters
Tuning Database Files
Tuning Automatic Segment-Space Management (ASSM)

Note:

Always review the tuning guidelines in your database-specific vendor documentation.

2.6.1 Tuning Database Parameters

The following tables provide common init.ora parameters and their descriptions. Consider following these guidelines to set the database parameters. Ultimately, however, the DBA should monitor the database health and tune parameters based on the need.

The database that is used for SOA is configured with the suggested values. Tuning the database involves adjusting the sizing parameters based on the available resource and load on the database.

The sga_target, pga_aggregate_target, and processes parameters from Table 2-2 are examples of such parameters that need to be tuned based on SGA and PGA advisories and looking into the number of open processes during peak load.

Table 2-2 Important Oracle 12c Database Tuning Parameters

Parameter	Description	Tuning Recommendation
`audit_trail` Default: DB	This parameter enables or disables database auditing.	Set this parameter to `NONE` if there is NO policy to audit database activity. Enabling auditing can impact performance.
`plsql_code_type` Default: INTERPRETED	`PLSQL_CODE_TYPE` specifies the compilation mode for PL/SQL library units. Possible modes are as follows: `INTERPRETED`: PL/SQL library units are compiled to PL/SQL bytecode format. Such modules are executed by the PL/SQL interpreter engine. `NATIVE`: PL/SQL library units are compiled to native (machine) code. Such modules are executed natively without incurring any interpreter impacts.	Set this parameter to `NATIVE`.
nls_sort Default: Derived from `NLS_LANGUAGE`	`NLS_SORT` specifies the collating sequence for ORDER BY queries. If the value is a named linguistic sort, the collating sequence is based on the order of the defined linguistic sort. Most languages supported by the `NLS_LANGUAGE` parameter also support a linguistic sort with the same name. If the value is set to `BINARY`, then the collating sequence is based on the numeric value of characters. This requires fewer system resources.	Set this parameter to `BINARY`.
open_cursors Default: 50	This parameter specifies the maximum number of open cursors that a session can have at once. Open cursors are handles to private SQL areas. The value of `OPEN_CURSORS` should be high enough to prevent your application from running out of open cursors.	Increase this parameter to `500`.
session_cached_cursors Default: 50	This parameter specifies the number of session cursors to cache. Repeated parse calls of the same SQL statement cause the session cursor for that statement to be moved into the session cursor cache. Subsequent parse calls find the cursor in the cache and do not reopen the cursor. Oracle uses a "least recently used" algorithm to remove entries in the session cursor cache to make room for new entries when needed. This parameter also constrains the size of the PL/SQL cursor cache which PL/SQL uses to avoid having to re-parse as statements are re-executed by a user.	Increase this parameter to `500`.
_b_tree_bitmap_plans Default: TRUE	This parameter enables or disables the use of bitmap access paths for b-tree indexes.	Set this parameter to `FALSE`.
processes Default: 100	This parameter sets the maximum number of operating system processes that can be connected to Oracle concurrently. The value of this parameter must account for Oracle background processes. The SESSIONS parameter is deduced from this value.	For most systems, increasing this parameter value to 1500 should suffice. For a large scale system, such as databases with large number of users, the recommended value is 5000.
Memory_target	MEMORY_TARGET specifies the Oracle system-wide usable memory. The database tunes memory to the MEMORY_TARGET value, reducing or enlarging the SGA and PGA as needed.	Consider setting the MEMORY_TARGET to `NONE`. Then set the SGA and PGA targets separately as setting MEMORY_TARGET does not allocate sufficient memory to SGA and PGA as needed.
sga_target Default: 0	Setting this parameter to a nonzero value enables Automatic Shared Memory Management. This can simplify configuration and improve performance.	For small systems, use a minimum of 2GB. For large systems, set this to 18GB.
pga_aggregate_target Default: 0	This parameter specifies the target aggregate PGA memory available to all server processes attached to the instance.	For small systems, use a minimum of 1GB. For large systems, set this to 8GB.
Disk_asynch_io Default: TRUE	DISK_ASYNCH_IO controls whether I/O to data files, control files, and log files is asynchronous. It decides what parallel server processes can overlap I/O requests with CPU processing during table scans.	If your platform supports asynchronous I/O, leave this parameter to its default value of `TRUE`. Otherwise, set it to `FALSE`.
Filesystemio_options Default: None	FILESYSTEMIO_OPTIONS specifies I/O operations for file system files.	Set this parameter to `SETALL`.
Secure_Files Default: PERMITTED	This parameter specifies how to store LOB objects from tables.	Set this parameter to `ALWAYS`.
parallel_max_servers Default: PARALLEL_THREADS_PER_CPUCPU_COUNTconcurrent_parallel_users*5	This parameter specifies the maximum number of parallel execution processes and parallel recovery processes for an instance. As demand increases, an Oracle Database increases the number of processes from the number created at instance startup up to this value.	Set this parameter to 12.
job_queue_processe be Default: 1000	Specifies the maximum number of job slaves per instance that can be created for the execution of DBMS_JOB jobs and Oracle Scheduler (DBMS_SCHEDULER) jobs.	Set this parameter to 12.
shared_servers Default: 0 (or) 1	Specifies the number of server processes that you want to create when an instance is started.	Set this parameter to 0.

In addition, you may consider setting a minimum value for SHARED_POOL_SIZE and DB_CACHE_SIZE to minimize frequent resizing.

2.6.2 Tuning Database Files

In addition to tuning the database parameters, the database administrator should properly configure the REDO logs, UNDO tablespace, and TEMP tablespaces to meet the demands of the expected or observed database workload. This is an empirical task. The recommendations in this section are intended to provide initial guidance in these areas.

The location of the database files should be optimized for I/O performance and growth. Segment Advisor should be leveraged to optimize the use of segment space and assure performance degradation does not occur. The advisor can provide historical growth trends of segments, which can be used to proactively plan for growth. See "Using the Segment Advisor" in Oracle Database Administrator's Guide for more information.

REDO Logs

Under demanding workloads, the size of the redo log files can influence performance. Generally, larger redo log files provide better performance. Undersized log files increase checkpoint activity and log file switches, which reduces performance.You can obtain sizing advice on the Redo Log Groups page of the Enterprise Manager.

Depending on your storage configuration and performance characteristics, you may need to redistribute redo logs to optimize I/O performance. Redo log files should be placed on a disk separate from data files to improve I/O performance.

See "Managing the Redo Log" in the Oracle Database Administrator's Guide for more information.

UNDO Tablespace

The suggested minimum size for the UNDO tablespace is 6 GB with auto-extend enabled. It is recommended that the default mode of automatic undo management be leveraged to maximize performance and efficiency.

The Oracle Enterprise Manager Automatic Undo Management Advisor should be leveraged to set configuration details for undo tablespace and retention settings. This advisor also provides access to the Undo Advisor that assesses the effect and provides advice of a new undo retention setting. For more information about using advisors, see "The Undo Advisor PL/SQL Interface" in the Oracle Database Administrator's Guide.

TEMP Tablespace

Oracle recommends the use of locally-managed temporary tablespaces with UNIFORM extents and the default size of 1 MB.

For tuning TEMP tablespaces for SOA, see Section 11.4.3.2.

2.6.3 Tuning Automatic Segment-Space Management (ASSM)

For permanent tablespaces, consider using automatic segment-space management. Such tablespaces, often referred to as bitmap tablespaces, are locally managed tablespaces with bitmap segment space management.

For backward compatibility, the default local tablespace segment-space management mode is MANUAL.

While configuring tablespaces, consider setting the extent allocation type to SYSTEM. If the allocation type is set to UNIFORM, it might impact performance.

For more information, see "Free Space Management" in Oracle Database Concepts, and "Specifying Segment Space Management in Locally Managed Tablespaces" in the Oracle Database Administrator's Guide.

2.7 Reusing Database Connections

Creating a database connection is a relatively resource intensive process in any environment. Typically, a connection pool starts with a small number of connections. As client demand for more connections grow, there may not be enough in the pool to satisfy the requests. WebLogic Server creates additional connections and adds them to the pool until the maximum pool size is reached.

One way to avoid connection creation delays is to initialize all connections at server startup, rather than on-demand as clients need them. This may be appropriate if your load is predictable and even. Set the initial number of connections equal to the maximum number of connections in the Connection Pool tab of your data source configuration. Determine the optimal value for the Maximum Capacity as part of your pre-production performance testing.

If your load is uneven, and has a much higher number of connections at peak load than at typical load, consider setting the initial number of connections equal to your typical load. In addition, consider setting the maximum number of connections based on your supported peak load. With these configurations, WebLogic server can free up some connections when they are not used for a period of time.

For more information, see "Tuning Data Source Connection Pool Options" in Administering JDBC Data Sources for Oracle WebLogic Server.

2.8 Enabling Data Source Statement Caching

When you use a prepared statement or callable statement in an application or EJB, there may be a performance impact associated with the processing of the communication between the application server and the database server and on the database server. To minimize the processing impact, enable the data source to cache prepared and callable statements used in your applications. When an application or EJB calls any of the statements stored in the cache, the server reuses the statement stored in the cache. Reusing prepared and callable statements reduces CPU usage on the database server, improving performance for the current statement and leaving CPU cycles for other tasks.

Consider the following data source configurations when performance is an issue:

When configuring the data source, ensure that the connection pool has enough free connections.
Statement caching can eliminate potential performance impacts caused by repeated cursor creation and repeated statement parsing and creation. Statement caching also reduces the performance impact of communication between the application server and the database server
Disable unnecessary connection testing and profiling.

Each connection in a data source has its own individual cache of prepared and callable statements used on the connection. However, you configure statement cache options per data source. That is, the statement cache for each connection in a data source uses the statement cache options specified for the data source, but each connection caches it's own statements. Statement cache configuration options include:

Statement Cache Type—The algorithm that determines which statements to store in the statement cache.
Statement Cache Size—The number of statements to store in the cache for each connection. The default value is 10. You should analyze your database's statement parse metrics to size the statement cache sufficiently for the number of statements you have in your application.

You can use the Administration Console to set statement cache options for a data source. See "Configure the statement cache for a JDBC data source" in the Oracle WebLogic Server Administration Console Online Help.

For more information on using statement caching, see "Increasing Performance with the Statement Cache" in the Administering JDBC Data Sources for Oracle WebLogic Server.

2.9 Controlling Concurrency

Limiting concurrency, at multiple layers of the system to match specific usage needs, can greatly improve performance. This section discusses a few of the areas within Oracle Fusion Middleware where concurrency can be controlled.

When system capacity is reached, and a web server or application server continues to accept requests, application performance and stability can deteriorate. There are several places within Oracle Fusion Middleware where you can throttle the requests to avoid overloading the mid-tier or database tier systems and tune for best performance.

Setting Server Connection Limits
Configuring Connection Pools
Tuning the WebLogic Sever Thread Pool

2.9.1 Setting Server Connection Limits

Oracle HTTP Server uses directives in httpd.conf. This configuration file specifies the maximum number of HTTP requests that can be processed simultaneously, logging details, and certain limits and time outs.

For more information on modifying the httpd.conf file, see "Configuring Oracle HTTP Server" in Administering Oracle HTTP Server.

You can use the MaxClients and ThreadsPerChild directives to limit incoming requests to WebLogic instances from the Oracle HTTP Server based on your expected client load and system resources. The following sections describe some Oracle HTTP Server tuning parameters related to connection limits that you typically need to tune based on your expected client load. See Chapter 6, "Tuning Oracle HTTP Server" for more information and a more complete list of tunable parameters.

2.9.1.1 MaxClients/ThreadsPerChild

Note:

The MaxClients parameter is applicable only to UNIX platforms and on Microsoft Windows (mpm_winnt), the same is achieved through the ThreadsPerChild and ThreadLimit parameters.

The MaxClients property specifies a limit on the total number of server threads running, that is, a limit on the number of clients who can simultaneously connect. If the number of client connections reaches this limit, then subsequent requests are queued in the TCP/IP system up to the limit specified (in the ListenBackLog directive).

You can configure the MaxClients directive in the httpd.conf file up to a maximum of 8K (the default value is 150). If your system is not resource-saturated and you have a user population of more than 150 concurrent HTTP connections, you can improve your performance by increasing MaxClients to increase server concurrency. Increase MaxClients until your system becomes fully utilized (85% is a good threshold).

When system resources are saturated, increasing MaxClients does not improve performance. In this case, the MaxClients value could be reduced as a throttle on the number of concurrent requests on the server.

If the server handles persistent connections, then it may require sufficient concurrent httpd server processes to handle both active and idle connections. When you specify MaxClients to act as a throttle for system concurrency, you need to consider that persistent idle httpd connections also consume httpd processes. Specifically, the number of connections includes the currently active persistent and non-persistent connections and the idle persistent connections. When there are no httpd server threads available, connection requests are queued in the TCP/IP system until a thread becomes available, and eventually clients terminate connections.

You can define a number of server processes and the threads per process (ThreadsPerChild) to handle the incoming connections to Oracle HTTP Server. The ThreadsPerChild property specifies the upper limit on the number of threads that can be created under a server (child) process.

Note:

ThreadsPerChild, StartServers, and ServerLimit properties are inter-related with the MaxClients setting. All of these properties must be set appropriately to achieve the number of connections as specified by MaxClients. See Table 6-1, "Oracle HTTP Server Configuration Properties" for a description of all the HTTP configuration properties.

2.9.1.2 KeepAlive

A persistent, KeepAlive, HTTP connection consumes an httpd child process, or thread, for the duration of the connection, even if no requests are currently being processed for the connection.

If you have sufficient capacity, KeepAlive should be enabled; using persistent connections improves performance and prevents wasting CPU resources re-establishing HTTP connections. Normally, you should not need to change KeepAlive parameters.

Note:

The default maximum requests for a persistent connection is 100, as specified with the MaxKeepAliveRequests directive in httpd.conf. By default, the server waits for 15 seconds between requests from a client before closing a connection, as specified with the KeepAliveTimeout directive in httpd.conf.

2.9.1.3 Tuning HTTP Server Modules

The Oracle HTTP Server (OHS) uses the mod_wl_ohs module to route requests to the underlying Weblogic server or the Weblogic Server cluster. The configuration details for mod_wl_ohs are available in the mod_wl_ohs.conf file in the config directory.

For more information on the tuning parameters for mod_wl_ohs see, "Understanding Oracle HTTP Server Modules" in Administering Oracle HTTP Server.

2.9.2 Configuring Connection Pools

Connection pooling is configured and maintained per Java runtime. Connections are not shared across different runtimes. To use connection pooling, no configuration is required. Configuration is necessary only if you want to customize how pooling is done, such as to control the size of the pools and which types of connections are pooled.

You configure connection pooling by using a number of system properties at program startup time. Note that these are system properties, not environment properties and that they affect all connection pooling requests.

For applications that use a database, performance can improve when the connection pool associated with a data source limits the number of connections. You can use the MaxCapacity attribute to limit the database requests from Oracle Application Server so that incoming requests do not saturate the database, or to limit the database requests so that the database access does not overload the Oracle Application Server-tier resource.

The connection pool MaxCapacity attribute specifies the maximum number of connections that a connection pool allows. By default, the value of MaxCapacity is set to 15. For best performance, you should specify a value for MaxCapacity that matches the number appropriate to your database performance characteristics.

Limiting the total number of open database connections to a number your database can handle is an important tuning consideration. You should check to make sure that your database is configured to allow at least as large a number of open connections as the total of the values specified for all the data sources MaxCapacity option, as specified in all the applications that access the database.

2.9.3 Tuning the WebLogic Sever Thread Pool

By default WebLogic Server uses a single thread pool, in which all types of work are executed. WebLogic Server uses Work Managers to prioritize work based on rules you can define, and run-time metrics, including the actual time it takes to execute a request and the rate at which requests are entering and leaving the pool. There is a default work manager that manages the common thread pool.

The common thread pool changes its size automatically to maximize throughput. WebLogic Server monitors throughput over time and based on history, determines whether to adjust the thread count. For example, if historical throughput statistics indicate that a higher thread count increased throughput, WebLogic increases the thread count. Similarly, if statistics indicate that fewer threads did not reduce throughput, WebLogic decreases the thread count.

Since the WebLogic Server thread pool by default is sized automatically, in most situations you do not need to tune this. However, for special requirements, an administrator can configure custom Work Managers to manage the thread pool at a more granular level for sets of requests that have similar performance, availability, or reliability requirements. With custom work managers, you can define priorities and guidelines for how to assign pending work (including specifying a min threads or max threads constraint, or a constraint on the total number of requests that can be queued or executing before WebLogic Server begins rejecting requests).

Use the following guidelines to help you determine when to use Work Managers to customize thread management:

The default fair share is not sufficient.

This usually occurs in situations where one application needs to be given a higher priority over another.
A response time goal is required.
A minimum thread constraint needs to be specified to avoid server deadlock.
You use MDBs in your application.

To ensure MDBs use a well-defined share of server thread resources, and to tune MDB concurrency, most MDBs should be modified to reference a custom work manager that has a max-threads-constraint. In general, a custom work manager is useful when you have multiple MDB deployments, or if you determine that a particular MDB needs more threads.

2.10 Setting Logging Levels

The amount of information that applications log depends on how the environment is configured and how the application code is instrumented. To maximize performance it is recommended that the logging level is not set higher than the default INFO level logging. If the logging setting does not match the default level, reset the logging level to the default for best performance.

Once the application and server logging levels are set appropriately, ensure that the debugging properties or other application level debugging flags are also set to appropriate levels or disabled. To avoid performance impacts, do not set log levels to levels that produce more diagnostic messages, including the FINE or TRACE levels.

Each component may have specific recommendations for logging levels. See the component chapters in this book for more information.