B Troubleshooting Oracle SOA Suite

This appendix describes how to troubleshoot issues you can encounter when using Oracle SOA Suite.

This appendix includes the following topics:

B.1 Best Practice for Starting and Stopping a Managed Server

As a best practice, it is always recommended that you start and stop a managed server through one, but not both, of the following methods. Do not mix these methods, such as starting the managed server from the command line and stopping it from Oracle Enterprise Manager Fusion Middleware Control Console, or vice versa.

  • Oracle Enterprise Manager Fusion Middleware Control Console

    With this method, the node manager must be up and running. The node manager tracks all managed server startups and shutdowns performed from Oracle Enterprise Manager Fusion Middleware Control Console. With this method, the server state is not an issue.

  • Command line

    With this method, the node manager does not track the server state. Therefore, if you start the server from the command line and shut it down from Oracle Enterprise Manager Fusion Middleware Control Console, the Oracle WebLogic Administration Server accesses the node manager to determine its status, which returns a state of Unknown.

Perform the following steps to stop and start the server from Oracle Enterprise Manager Fusion Middleware Control Console.

  1. Expand the WebLogic domain.

  2. Select the managed server (for example, named soa_server1).

  3. Select Control > Shut Down.

  4. Select Control > Start Up.

For information on starting and stopping managed servers from the command line, see Oracle Fusion Middleware Installation Guide for Oracle SOA Suite.

B.2 Optimizing the Loading of Pages with Instance and Faults Metrics

Since production systems can include numerous composite instances and faults, there is a possibility of timeouts in the SOA Oracle Enterprise Manager Fusion Middleware Control Console pages as information retrieval becomes relatively slow. To optimize the loading performance of pages, you can enable a property setting in the SOA Infrastructure Common Properties page that disables the loading of all metrics information upon page load. The instances and faults metrics can be obtained on demand from the server.

  1. In the navigator, click soa-infra.

  2. Note that values appear in the Running and Total fields in the Recent Composite Instances section and the Instances column of the Deployed Composites section. When these values are large, it can take time to load this page and other pages with similar information.

    Description of soaadmin_fetch1.gif follows
    Description of the illustration soaadmin_fetch1.gif

  3. From the SOA Infrastructure menu, select SOA Administration > Common Properties.

  4. In the Display Data Counts section, select the Disable fetching of instance and fault count metrics check box.

  5. Click Apply.

  6. Return to the Dashboard page of the SOA Infrastructure.

  7. Note that the values that previously displayed have been replaced with links.

  8. Click a link.

    Description of soaadmin_fetch2.gif follows
    Description of the illustration soaadmin_fetch2.gif

    The values are calculated for the link you selected. When the calculation is complete, a message displays the total values.

    Description of soaadmin_fetch3.gif follows
    Description of the illustration soaadmin_fetch3.gif

For more information about setting this property, see Section 3.1, "Configuring SOA Infrastructure Properties."

Notes:

  • If you click a link to retrieve instance and fault count metrics, and Oracle Enterprise Manager Fusion Middleware Control Console times out, increase the transaction timeout property. For more information, see Section B.5, "Resolving Connection Timeouts."

  • If you click Recalculate, and the recalculation occurs quickly, the progress indicator does not have a chance to render. However, any updates to the data are reflected on-screen.

B.3 Resolving Message Failure Caused By Too Many Open Files

You can receive the following error at run time or compilation time, depending on the number of JAR files being used, the use of file descriptors by JDK 6/JRE, or both.

Message send failed: Too many open files

To resolve this error, increase the number of file descriptors to at least 4096.

  1. Use the limit command (for the C shell) or the ulimit command (for the Bash shell) to identify the value for descriptors. A value of 1024 is typically too low, especially for JDK 6.

    % limit

cputime      unlimited
filesize     unlimited
datasize     unlimited
stacksize    10240 kbytes
coredumpsize unlimited
memoryuse    unlimited
vmemoryuse   unlimited
descriptors  1024
memorylocked 500000 kbytes
maxproc      46720

  2. Log in as the root user on your operating system.

  3. Edit the /etc/security/limits.conf file to increase the value for descriptors.

    For this example, the limits.conf file looks as follows after increasing the limit for all users to 4096:

    #<domain>      <type>  <item>         <value>
#

#*               soft    core            0
#*               hard    rss             10000
#@student        hard    nproc           20
#@faculty        soft    nproc           20
#@faculty        hard    nproc           50
#ftp             hard    nproc           0
#@student        -       maxlogins       4

# End of file
@svrgroup    soft    memlock         500000
@svrgroup    hard    memlock         500000
*           soft    nofile          4096
*           hard    nofile          4096

  4. Close your terminal and reopen for the change to take effect. A system restart is not required.

B.4 Extending Tablespaces to Avoid Problems at Run Time

If the database tablespace is not extended, run time processing can be impacted. Messages are not processed or persisted, and exception errors similar to the following can appear in the log files. This is because Oracle BPEL Process Manager relies on the database to store instance data. If the database is not available, run-time processing is impacted.

INFO: MediatorServiceEngine returning after processing the request for 
operation = processResponse 

[EL Warning]: 2009.01.14 11:46:16.783--UnitOfWork(32372128)--Exception 
[EclipseLink-4002] (Eclipse Persistence Services - 1.1 (Build 
SNAPSHOT-20081007)): org.eclipse.persistence.exceptions.DatabaseException 
Internal Exception: java.sql.BatchUpdateException: ORA-01691: unable to 
extend lob segment SH_SOAINFRA.SYS_LOB0000145067C00007$$ by 1024 in 
tablespace SH_SOAINFRA 

Error Code: 1691 
Query: InsertObjectQuery(com.collaxa.cube.persistence.dto.AuditTrail@199b33d) 
[EL Warning]: 2009.01.14 11:46:16.782--UnitOfWork(32372128)--Exception 
[EclipseLink-4002] (Eclipse Persistence Services - 1.1 (Build 
SNAPSHOT-20081007)): org.eclipse.persistence.exceptions.DatabaseException 
Internal Exception: java.sql.BatchUpdateException: ORA-01691: unable to 
extend lob segment SH_SOAINFRA.SYS_LOB0000145067C00007$$ by 1024 in 
tablespace SH_SOAINFRA 
. . .
. . .

Ensure that you set a tablespace to automatically extend itself by a specified amount when it reaches its size limit. If you do not enable autoextend, ensure that you respond when alerted that the tablespace is reaching its critical or warning threshold size. You can respond to size alerts by manually increasing the tablespace size.

B.5 Resolving Connection Timeouts

You can receive a connection timeout error under circumstances such as the following:

  • You run a SOA composite application with a large payload that takes more than 30 seconds to process.

  • You are invoking a stress test using a large payload from the Test Web Service page of Oracle Enterprise Manager Fusion Middleware Control Console.

  • You are passing a large number of message files (one million) into a composite with a file adapter service.

  • You are retrieving instance and fault count metrics in Oracle Enterprise Manager Fusion Middleware Control Console.

To avoid receiving timeout errors, increase the transaction timeout property as follows:

  1. Log into Oracle WebLogic Administration Console.

  2. Click JTA.

  3. Change the value of Timeout Seconds (the default is 30).

  4. Click Save.

  5. Restart Oracle WebLogic Server.

B.6 Increasing Database Connection Values

You can receive the following error message because of slow connections to the database.

Exception [TOPLINK-4002] (Oracle TopLink - 11g Release 1 (11.1.1.1.0) (Build 
090304)): oracle.toplink.exceptions.DatabaseException 
Internal Exception: java.sql.SQLException: Internal error: Cannot obtain 
XAConnection weblogic.common.resourcepool.ResourceDeadException: Pool 
SOADataSource has been disabled because of hanging connection tests, cannot 
allocate resources to applications.

If this occurs, perform the following steps:

  1. Open the DOMAIN_HOME\bin\setSOADomainEnv.cmd file.

  2. Uncomment the lines shown in bold.

    # 8331492: Value of weblogic.resourcepool.max_test_wait_secs is 10 
# seconds. It can be increased by uncommenting line below if your database 
# connections are slow. See SOA documentation for more details. 
 EXTRA_JAVA_PROPERTIES="${EXTRA_JAVA_PROPERTIES}
-Dweblogic.resourcepool.max_test_wait_secs=30" 
export EXTRA_JAVA_PROPERTIES

  3. Save your changes and restart the managed Oracle WebLogic Server.

B.7 Accessing Oracle Enterprise Manager Fusion Middleware Control Console on Dual Stack Hosts that Support IPv4 and IPv6

If you run Oracle SOA Suite on a dual stack host that supports both IPv4 and IPv6, you must update the etc/hosts file as shown in Table B-1 for IPv4 clients to access IPv6 URLs in Oracle Enterprise Manager Fusion Middleware Control Console.

Table B-1 IPv4 and IPv6 Settings in etc/hosts File

On The... Edit the etc/hosts File as Follows....

On the IPv4 client:

 
xx.xxx.xxx.xxx myhost10-ipv6

where xx.xxx.xxx.xxx is the IP address of IPv6.

On the IPv6 client

 
fdf5:74cc:db0a::0:1  myhost10-ipv6 myhost10-ipv6.us.oracle.com

Note: Replace fdf5:74cc:db0a::0:1 with a value appropriate to your host environment.