3 Troubleshooting Oracle Data Integrator

This chapter describes common problems that you might encounter when using Oracle Data Integrator (ODI) and explains how to solve them.

This chapter includes the following sections:

Section 3.1, "Introduction to Troubleshooting ODI"
Section 3.2, "Getting Started with Logging Basics for Oracle Data Integrator"
Section 3.3, "Resolving Common Problems"

In addition to this chapter, review the Oracle Fusion Middleware Error Messages Reference for information about the error messages you may encounter.

3.1 Introduction to Troubleshooting ODI

This section provides guidelines and a process for using the information in this chapter. Using the following guidelines and process will focus and minimize the time you spend resolving problems.

Guidelines

When using the information in this chapter, Oracle recommends:

After performing any of the solution procedures in this chapter, immediately retrying the failed task that led you to this troubleshooting information. If the task still fails when you retry it, perform a different solution procedure in this chapter and then try the failed task again. Repeat this process until you resolve the problem.
Making notes about the solution procedures you perform, symptoms you see, and data you collect while troubleshooting. If you cannot resolve the problem using the information in this chapter and you must log a service request, the notes you make will expedite the process of solving the problem.

Process

Follow the process outlined in Table 3-1 when using the information in this chapter. If the information in a particular section does not resolve your problem, proceed to the next step in this process.

Table 3-1 Process for Using the Information in this Chapter

Step	Section to Use	Purpose
1	Section 3.2	Get started troubleshooting ODI using log files.
2	Section 3.3	Perform problem-specific troubleshooting procedures for ODI. This section describes: Possible causes of the problems Solution procedures corresponding to each of the possible causes
3	Section 14.1	Use My Oracle Support to get additional troubleshooting information about Oracle Fusion Applications or ODI. My Oracle Support provides access to several useful troubleshooting resources, including Knowledge Base articles and Community Forums and Discussions.
4	Section 14.1	Log a service request if the information in this chapter and My Oracle Support does not resolve your problem. You can log a service request using My Oracle Support at `https://support.oracle.com`.

3.2 Getting Started with Logging Basics for Oracle Data Integrator

Use the following logging tools to troubleshoot ODI issues:

Section 3.2.1, "Viewing and Configuring ODI Agent Logs"
Section 3.2.2, "Viewing Session Logs for ODI"

3.2.1 Viewing and Configuring ODI Agent Logs

The ODI agents are deployed to a Managed Server. You can view the aggregate session events for the agent.

To view the logs for the ODI agent with Fusion Applications Control:

Log into Fusion Applications Control.
From the navigation pane, expand the farm, Application Deployments, oraclediagent (ODICluster).
Select oraclediagent (odi_server_name).

The Application Deployment page displays.
From the Application Deployment menu, choose Logs > View Log Messages.

The Log Messages page contains all of the latest ODI agent log messages. Click the messages to get more information and download the corresponding log file. For more information about viewing logs, see the "Viewing Log Files and Their Messages Using Fusion Middleware Control" chapter in the Oracle Fusion Middleware Administrator's Guide.

Note the session name and time or session ID from the log.

If the problem still cannot be solved, increase the log level of the system to debug the transactions. To simplify troubleshooting, it is recommended that you enable the following parent loggers at the TRACE32 (FINEST) level.

To change logger levels, perform the following steps in Fusion Applications Control:

From the navigation pane, expand the farm, Application Deployments, oraclediagent (ODICluster).
Select oraclediagent (odi_server_name).

The Application Deployment page displays.
Choose Logs > Log Configuration.
In the Logger Name column, expand the oracle and then oracle.odi to display the loggers.
In the Oracle Diagnostic Logging Level (Java Level) column, change the logging level to TRACE:32 for selected ODI loggers.

The change should take effect within a few minutes. Note that in a production system, setting the trace at a fine-grained level can result in a large amount of output that must be diagnosed. You can alternately use selective tracing that provides a way to get a detailed, on-disk trace selectively (for example, by user name, thereby eliminating trace output for other users).
To activate selective tracing, right-click the domain under WebLogic Domain and choose Logs > Selective Tracing.

Note that Selecting Tracing does not display as an option when you right-click an Administration Server or Managed Server and choose Logs.
From the Option Name list, choose the type of selective trace (for example, based on user name), and start the trace.
When the problem has been reproduced, disable the trace and view the output to narrow down the issue.

For more information on selective tracing, see the "Configuring and Using Selective Tracing" section of the Oracle Fusion Middleware Administrator's Guide.
Review the error logs (from Fusion Applications Control) for more information on the error.

Cross layer, server, and family functionality can be correlated through the execution context ID (ECID) (for example, you can look up the composite instance for a given expense report by correlating all the log entries with the ECID associated with that expense report transaction). For more information, see the "Correlating Messages Across Log Files and Components" section of the Oracle Fusion Middleware Administrator's Guide.

3.2.2 Viewing Session Logs for ODI

To view a specific session log file:

Log into Fusion Applications Control.
From the navigation pane, expand the farm, and then ODI.
Click MASTER REPOSITORY.

The Master Repository page displays.
From the Agents section, from the Running column (), click the number for the agent with the session to search the sessions, for example, FusionCrmOdiAgent

The Search Sessions page displays.
On the Search Sessions page, in the Sessions section, locate the session number corresponding to the job. Notice that while the session has a value in the Begin Date column, it does not have a value in the End Date column.
Click the session number corresponding to the job.

You are prompted to log in to the ODI Console.
From the Repository list, choose Work Repository.
Enter the credentials, and then click Sign In.
On the Session page, in the Execution Statistics section, note the values for the Session ID and Session Name in the Definition section, and the time of session in the Execution Statistics section.
Expand the Relationships and view the steps in the Session Steps tab.
For a step that did not successfully complete, click the link in the Step Name column to perform a deeper analysis.
On the Step page, expand the Relationships and view the tasks for the step in the Session Tasks tab.

Description of the illustration odisessiontask.jpg
For a step that did not successfully complete, click the link in the Step Name column to perform a deeper analysis.
On the Session Task Detail page, review the Message field in the Execution Statistics section and the Target Command command in the Target Details section for errors.

Description of the illustration odisessiondetail.jpg

3.3 Resolving Common Problems

This section describes common problems and solutions. For each problem and solution, use the example that fits best. It contains the following topics:

Section 3.3.1, "ODI Process Is Taking a Long Time to Complete Job Request"
Section 3.3.2, "A Coherence Node Connected to ODI Is Using UDP Multicasting for Cluster Discovery Instead of WKA"

3.3.1 ODI Process Is Taking a Long Time to Complete Job Request

Problem

An ODI job request was started and is running far past the point at which it should have completed. You can determine by this viewing the session log to determine if there is end date. See the Section 3.2.2, Steps 1 to 5 to determine if a session has an end date.

Cause

It is likely that the job is still running due to a job step stalling or failing to complete.

Solution

Check the ODI Agent log files and the ODI Console session log files for error messages. These messages provide information that could help you find the reason the job is not completing.

To resolve this issue:

View the ODI agent log file. See Section 3.2.1.
View the ODI session log. See Section 3.2.2.

If you find no error messages in the log files, the problem is likely due to environment issues outside of ODI.

3.3.2 A Coherence Node Connected to ODI Is Using UDP Multicasting for Cluster Discovery Instead of WKA

Problem

The Coherence node is operating in UDP Multicasting for cluster discovery instead of WKA, overriding the ODI server start settings indicated in the WebLogic Server Administration Console.

The Coherence log file contains diagnostic information that can confirm this issue. In some cases, there is a tangosol.coherence.log setting that is overriding the logging settings for odi_server1 in the WebLogic Server Administration console. You must disable the override to view the Coherence log. You can do this by editing the setDomainEnv.sh script (UNIX) or the setDomainEnv.cmd script (Windows).

To disable the logging override and view the output of the Coherence logs:

Navigate to the setDomainEnv script, located at this path:
```
(UNIX) DOMAIN_HOME/bin
(Windows) DOMAIN_HOME\bin
```

Open the script in a text editor. The EXTRA_JAVA_PROPERTIES section of the file will look similar to this:

EXTRA_JAVA_PROPERTIES="${EXTRA_JAVA_PROPERTIES} 
-Dsoa.archives.dir=${SOA_ORACLE_HOME}/soa -Dsoa.oracle.home=${SOA_ORACLE_HOME} 
-Dsoa.instance.home=${DOMAIN_HOME} 
-Dtangosol.coherence.log=jdk 
-Dtangosol.coherence.clusteraddress=10.0.0.0 
-Dtangosol.coherence.clusterport=9063 
-Djavax.xml.soap.MessageFactory=oracle.j2ee.ws.saaj.soap.MessageFactoryImpl -Dweblogic.transaction.blocking.commit=true 
-Dweblogic.transaction.blocking.rollback=true 
-Djavax.net.ssl.trustStore=${WL_HOME}/server/lib/fusion_trust.jks"

Remove -Dtangosol.coherence.log=jdk and save the script.
Delete the old coherence.log file.
Restart the domain. Coherence log messages will now properly be output to the coherence.log file.

For more information about restarting domains in Oracle Fusion Applications, see the "Starting and Stopping a Product Family Oracle WebLogic Server Domain" section in the Oracle Fusion Applications Administrator's Guide.

The Coherence log file displays the following output:

Oracle Coherence Version 3.6.0.4 Build 19111
Grid Edition: Development mode
Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.
 
2012-05-10 05:52:38.686/54.656 Oracle Coherence GE 3.6.0.4 <Info> (thread=[ACTIVE] 
ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)', 
member=n/a): Loaded cache configuration from 
"jar:file:/user/APPLTOP/fusionapps/soa/soa/modules/oracle.soa.fabric_
11.1.1/fabric-runtime.jar!/soa-coherence-cache-config.xml"
2012-05-10 05:52:39.557/55.527 Oracle Coherence GE 3.6.0.4 <D4> (thread=[ACTIVE] 
ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)', 
member=n/a): TCMP bound to /10.0.0.0:9066 using SystemSocketProvider
2012-05-10 05:52:43.062/59.032 Oracle Coherence GE 3.6.0.4 <Info> (thread=Cluster, 
member=n/a): Created a new cluster "cluster:0x000" with Member(Id=1, 
Timestamp=2012-05-10 05:52:39.592, Address=10.0.0.0:9066, MachineId=53980, 
Location=site:example.com,machine:machine_name,process:997, Role=WeblogicServer, 
Edition=Grid Edition, Mode=Development, CpuCount=24, SocketCount=24) 
UID=0x0AF129DC0000013736CEEC28D2DC236A
2012-05-10 05:52:43.107/59.077 Oracle Coherence GE 3.6.0.4 <Info> (thread=[ACTIVE] 
ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)', 
member=n/a): Started cluster Name=cluster:0x000
 
Group{Address=10.0.0.0, Port=9063, TTL=4}

The Group{Address=10.0.0.0, Port=9063, TTL=4} line indicates that UDP Multicasting is being used for cluster discovery instead of WKA. If WKA were being used, the log file would contain the following lines:

WellKnownAddressList(Size=1,
 WKA{Address=10.0.0.0, Port=9066}
 )

Cause

The SOA Coherence override file, tangosol-coherence-override.xml, takes precedence over the ODI Coherence override file and the server start settings in the WebLogic Server Administration Console.

Solution 1

If the ODI Server was started with Node Manager, you must add the WKA properties to the Server Start settings for the ODI server in the WebLogic Server Administration Console and restart the ODI server to enable WKA cluster discovery.

To configure WKA settings for ODI in the WebLogic Administration Console:

Follow the procedure outlined in the "Configure Coherence for the Cluster" section in the Oracle Fusion Middleware High Availability Guide.
Restart the ODI server.

For more information about restarting servers using the WebLogic Server Administration Console, see "Start Managed Servers from the Administration Console" and "Shutdown servers in a cluster" in the Oracle Fusion Middleware Oracle WebLogic Server Administration Console Online Help.

Solution 2

If the ODI server was started with the command prompt, you must add the WKA properties to the startManagedWebLogic.sh script (UNIX) or the startManagedWebLogic.cmd script (Windows) and restart the ODI server to enable WKA cluster discovery.

To configure WKA settings for ODI in the startManagedWebLogic script:

From the command prompt, navigate to the location of the startManagedWebLogic script:
```
(UNIX) DOMAIN_HOME/bin/
(Windows) DOMAIN_HOME\bin
```
Using a text editor, add the following lines to the script:
```
-Doracle.odi.coherence.wka1=machine1 -Doracle.odi.coherence.wka1.port=9088
-Doracle.odi.coherence.wka2=machine2 -Doracle.odi.coherence.wka2.port=9088
-Dtangosol.coherence.localport=9088
```
where machine1 and machine2 are the hostnames of the two machines in the cluster.

Note:

Use 9088 as the Coherence port if it is unused on the machine. Otherwise, choose another port to configure as the Coherence port.
Restart the ODI server using the fastartstop utility.

For more information about restarting servers using the fastartstop utility, see the "Stopping the Administration Servers and Managed Servers" and the "Starting the Administration Servers and Manages Servers" sections in the Oracle Fusion Applications Administrator's Guide.