19 Troubleshoot Oracle Data Integrator

1. From the navigation pane, expand the farm, Application Deployments, oraclediagent (ODICluster).
2. Select oraclediagent (odi_server_name).

   The Application Deployment page displays.

3. Choose Logs > Log Configuration.
4. In the Logger Name column, expand the oracle and then oracle.odi to display the loggers.
5. In the Oracle Diagnostic Logging Level (Java Level) column, change the logging level to TRACE:32 for selected ODI loggers.

   The change should take effect within a few minutes. Note that in a production system, setting the trace at a fine-grained level can result in a large amount of output that must be diagnosed. You can alternately use selective tracing that provides a way to get a detailed, on-disk trace selectively (for example, by user name, thereby eliminating trace output for other users).

6. To activate selective tracing, right-click the domain under WebLogic Domain and choose Logs, then Selective Tracing.

   Note that Selecting Tracing does not display as an option when you right-click an Administration Server or Managed Server and choose Logs.

7. From the Option Name list, choose the type of selective trace (for example, based on user name), and start the trace.
8. When the problem has been reproduced, disable the trace and view the output to narrow down the issue.

   For more information on selective tracing, see the Configuring and Using Selective Tracing section of the Administering Oracle Fusion Middleware.

9. Review the error logs (from Fusion Applications Control) for more information on the error.

   Cross layer, server, and family functionality can be correlated through the execution context ID (ECID) (for example, you can look up the composite instance for a given expense report by correlating all the log entries with the ECID associated with that expense report transaction). For more information, see the Correlating Messages Across Log Files and Components section of the Administering Oracle Fusion Middleware.

1. Log into Fusion Applications Control.
2. From the navigation pane, expand the farm, and then expand ODI.
3. Click MASTER REPOSITORY.

   The Master Repository page displays.

4. From the Agents section, from the Running column (), click the number for the agent with the session to search the sessions, for example, FusionCrmOdiAgent

   The Search Sessions page displays.

5. On the Search Sessions page, in the Sessions section, locate the session number corresponding to the job. Notice that while the session has a value in the Begin Date column, it does not have a value in the End Date column.
6. Click the session number corresponding to the job.

   You are prompted to log in to the ODI Console.

7. From the Repository list, choose Work Repository.
8. Enter the credentials, and then click Sign In.
9. On the Session page, in the Execution Statistics section, note the values for the Session ID and Session Name in the Definition section, and the time of session in the Execution Statistics section.
10. Expand the Relationships and view the steps in the Session Steps tab.
11. For a step that did not successfully complete, click the link in the Step Name column to perform a deeper analysis.
12. On the Step page, expand the Relationships and view the tasks for the step in the Session Tasks tab.

    Figure 19-1 Session Task Tab

    
    Description of "Figure 19-1 Session Task Tab"
13. For a step that did not successfully complete, click the link in the Step Name column to perform a deeper analysis.
14. On the Session Task Detail page, review the Message field in the Execution Statistics section and the Target Command command in the Target Details section for errors.

    Figure 19-2 Session Task Detail Page

    
    Description of "Figure 19-2 Session Task Detail Page"

1. View the ODI agent log file. For more information, see the View and Configure ODI Agent Logs section.
2. View the ODI session log. For more information, see the View Session Logs for ODI section.

Problem

The Coherence node is operating in UDP Multicasting for cluster discovery instead of Well-Known Address (WKA), overriding the ODI server start settings indicated in the WebLogic Server Administration Console.

The Coherence log file contains diagnostic information that can confirm this issue. In some cases, there is a tangosol.coherence.log setting that is overriding the logging settings for odi_server1 in the WebLogic Server Administration console. Disable the override to view the Coherence log by editing the setDomainEnv.sh script.

To disable the logging override and view the output of the Coherence logs, perform the following steps:

1. Navigate to the setDomainEnv script, located at this path:

   (UNIX) DOMAIN_HOME/bin
   

2. Open the script in a text editor. The EXTRA_JAVA_PROPERTIES section of the file will look similar to the following example:

   EXTRA_JAVA_PROPERTIES="${EXTRA_JAVA_PROPERTIES} 
   -Dsoa.archives.dir=${SOA_ORACLE_HOME}/soa -Dsoa.oracle.home=${SOA_ORACLE_HOME} 
   -Dsoa.instance.home=${DOMAIN_HOME} 
   -Dtangosol.coherence.log=jdk 
   -Dtangosol.coherence.clusteraddress=10.0.0.0 
   -Dtangosol.coherence.clusterport=9063 
   -Djavax.xml.soap.MessageFactory=oracle.j2ee.ws.saaj.soap.MessageFactoryImpl -Dweblogic.transaction.blocking.commit=true 
   -Dweblogic.transaction.blocking.rollback=true 
   -Djavax.net.ssl.trustStore=${WL_HOME}/server/lib/fusion_trust.jks"
   

3. Remove -Dtangosol.coherence.log=jdk and save the script.

4. Delete the old coherence.log file.

5. Restart the domain. Coherence log messages will now properly be output to the coherence.log file.

   For more information about restarting domains in Oracle Fusion Applications, see the Start an Oracle WebLogic Server Domain for a Product Family section.

The Coherence log file displays the following output:

Oracle Coherence Version 3.6.0.4 Build 19111
Grid Edition: Development mode
Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.
 
2012-05-10 05:52:38.686/54.656 Oracle Coherence GE 3.6.0.4 <Info> (thread=[ACTIVE] 
ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)', 
member=n/a): Loaded cache configuration from 
"jar:file:/user/APPLTOP/fusionapps/soa/soa/modules/oracle.soa.fabric_
11.1.1/fabric-runtime.jar!/soa-coherence-cache-config.xml"
2012-05-10 05:52:39.557/55.527 Oracle Coherence GE 3.6.0.4 <D4> (thread=[ACTIVE] 
ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)', 
member=n/a): TCMP bound to /10.0.0.0:9066 using SystemSocketProvider
2012-05-10 05:52:43.062/59.032 Oracle Coherence GE 3.6.0.4 <Info> (thread=Cluster, 
member=n/a): Created a new cluster "cluster:0x000" with Member(Id=1, 
Timestamp=2012-05-10 05:52:39.592, Address=10.0.0.0:9066, MachineId=53980, 
Location=site:example.com,machine:machine_name,process:997, Role=WeblogicServer, 
Edition=Grid Edition, Mode=Development, CpuCount=24, SocketCount=24) 
UID=0x0AF129DC0000013736CEEC28D2DC236A
2012-05-10 05:52:43.107/59.077 Oracle Coherence GE 3.6.0.4 <Info> (thread=[ACTIVE] 
ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)', 
member=n/a): Started cluster Name=cluster:0x000
 
Group{Address=10.0.0.0, Port=9063, TTL=4}

The Group{Address=10.0.0.0, Port=9063, TTL=4} line indicates that UDP Multicasting is being used for cluster discovery instead of WKA. If WKA were being used, the log file would contain the following lines:

WellKnownAddressList(Size=1,
 WKA{Address=10.0.0.0, Port=9066}
 )

Cause

The SOA Coherence override file, tangosol-coherence-override.xml, takes precedence over the ODI Coherence override file and the server start settings in the WebLogic Server Administration Console.

Solution 1

If the ODI Server was started with Node Manager, add the WKA properties to the Server Start settings for the ODI server in the WebLogic Server Administration Console and restart the ODI server to enable WKA cluster discovery.

To configure WKA settings for ODI in the WebLogic Administration Console, perform the following steps:

1. Configure Coherence for the Cluster by performing the following steps:

   1. In the Administration Console, select Environment, then select Server from the left hand tab.

   2. Select ess_server1.

   3. Click the Server Startup tab.

   4. In the Arguments box, enter the following (all on a single line):

      -Doracle.odi.coherence.wka1=machine1 -Doracle.odi.coherence.wka1.port=9088
      -Doracle.odi.coherence.wka2=machine2 -Doracle.odi.coherence.wka2.port=9088
      -Dtangosol.coherence.localport=9088
      

      Where machine1 and machine2 are the hostnames of the two machines in the cluster.

      Use 9088 as the Coherence port if it is unused on the machine. Otherwise, choose another port to configure as the Coherence port.

   5. Click Save.

   6. Repeat the steps above for ess_server2.

2. Restart the ODI server.

   For more information about restarting servers using the WebLogic Server Administration Console, see Start Managed Servers from the Administration Console and Shutdown Servers in a Cluster in the Oracle WebLogic Server Administration Console Online Help.

Solution 2

If the ODI server was started with the command prompt, you must add the WKA properties to the startManagedWebLogic.sh script and restart the ODI server to enable WKA cluster discovery.

To configure WKA settings for ODI in the startManagedWebLogic script:

1. From the command prompt, navigate to the location of the startManagedWebLogic script:

   (UNIX) DOMAIN_HOME/bin/
   

2. Using a text editor, add the following lines to the script:

   -Doracle.odi.coherence.wka1=machine1 -Doracle.odi.coherence.wka1.port=9088
   -Doracle.odi.coherence.wka2=machine2 -Doracle.odi.coherence.wka2.port=9088
   -Dtangosol.coherence.localport=9088
   

   where machine1 and machine2 are the hostnames of the two machines in the cluster.

   Use 9088 as the Coherence port if it is unused on the machine. Otherwise, choose another port to configure as the Coherence port.

3. Restart the ODI server using the fastartstop utility.

   For more information about restarting servers using the fastartstop utility, see the Start Administration Servers and Managed Servers and the Stop Administration Servers and Managed Servers sections.

Problem

ODI is unable to connect to the ODI repository.

Cause 1

Provisioning failed to populate the correct GUIDs.

Solution

Connect to the FUSION_ODI schema and use the following SQL commands:

select WUSER_NAME, PASS, GUID_EXTERNAL from SNP_USER;
select AGENT_NAME, HOST_NAME, HOST_PORT from SNP_AGENT;

If the GUIDs are missing: either the provisioning failed or the ODI repository was overwritten after provisioning. If the GUIDs are present, check that the repository was not overwritten after provisioning. Check the provisioning logs and resolve any errors.

For more information about provisioning logs and troubleshooting the provisioning process, see the Provisioning Log Files section in the Oracle Fusion Applications Installation Guide.

Cause 2

The repository was overwritten during provisioning.

Solution

Run the following SQL commands:

select count( * ) from SNP_PROJECT;
select count( * ) from SNP_MODEL;

If projects or models are missing, the ODI repository was not created correctly in the starter database or was overwritten after provisioning. Check the provisioning logs and correct any errors.

For more information about provisioning logs and troubleshooting the provisioning process, see the Provisioning Log Files section in the Oracle Fusion Applications Installation Guide.

Problem

The ODI Agent is unable to call back to Oracle Enterprise Scheduler to update the status of a job.

Solution

The ODI Agent may be prevented from communicating with Oracle Enterprise Scheduler. To resolve this issue, perform the following steps:

1. Check the ODI Agent logs for errors. For more information about checking the ODI Agent logs, see the View and Configure ODI Agent Logs section.

2. Resolve any errors listed in the logs.

If there are no errors in the ODI Agent logs, an error with Oracle Enterprise Scheduler might be causing the issue.