Oracle Data Integrator (ODI) provides a fully unified solution for building, deploying, and managing complex data warehouses or as part of data-centric architectures in an SOA or business intelligence environment. In addition, it combines all the elements of data integration - data movement, data synchronization, data quality, data management, and data services - to ensure that information is timely, accurate, and consistent across complex systems.
An ODI domain contains the following ODI components that can be managed using Enterprise Manager Cloud Control.
One Master and one or more Work repositories attached to it.
One or several Run-Time Agents attached to the Master Repositories. These agents must be declared in the Master Repositories to appear in the domain. These agents may be Standalone Agents, Colocated Standalone Agents, or Java EE Agents.
One or several Oracle Data Integrator Console applications. An Oracle Data Integrator Console application is used to browse Master and Work repositories.
Note:
Starting with Oracle Fusion Middleware Plug-in (12.1.0.6), you can monitor the repositories that are configured even with Microsoft SQL Server and IBM DB2. However, as a prerequisite, make sure you first deploy the Microsoft SQL Server Plug-in and IBM DB2 Plug-in, respectively, and then discover those database instances as targets in Enterprise Manager Cloud Control.This chapter describes how you can set up and manage ODI targets using Enterprise Manager Cloud Control:
Before you start managing ODI with Enterprise Manager, you must do the following:
Deploy the Oracle Management Agent
Oracle Management Agents must be installed on the database hosting the ODI repositories. Optionally, an Oracle Management Agent can also be installed on a machine hosting an ODI Agent.
See Installing the Oracle Management Agent in the Oracle Enterprise Manager Cloud Control Basic Installation Guide
Discover ODI Targets
ODI targets are discovered along with the WebLogic domain linked to them. Use the Fusion Middleware discovery to discover your WebLogic domain. This in turn discovers two types of ODI targets, mainly ODI Standalone Agent and ODI Java EE Agent.
For additional information about Fusion Middleware discovery, see Oracle Enterprise Manager Cloud Control Administrator's Guide.
Discover the Databases Hosting ODI Repositories
Each database instance needs to be discovered because more than one database could be hosting the ODI repositories.
All the operations are available out-of-box in Enterprise Manager.
This section describes the following:
To monitor ODI, follow these steps:
From the Targets menu, select Middleware.
On the Middleware page, from the Middleware Features menu, select ODI Home.
On the ODI Home page, click the Dashboard tab.
The Dashboard tab has the following regions:
This region reports the following:
Number of master repositories that are either up or down. Click the number for a list of the repositories.
Number of master repositories with incidents. Click the number to find out which repositories have incidents.
Note:
Starting with Oracle Fusion Middleware Plug-in (12.1.0.6), you can monitor the repositories that are configured even with Microsoft SQL Server and IBM DB2. However, as a prerequisite, make sure you first deploy the Microsoft SQL Server Plug-in and IBM DB2 Plug-in, respectively, and then discover those database instances as targets in Enterprise Manager Cloud Control.The database information that is stored in the ODI does not use local host or IP address to identify the database. It only uses the host name of the database. Ensure that the host name in the ODI is consistent with the host name stored in EMCC. Also, check the JDBC data sources defined in WLS for the Master and Work repositories. They should match the information stored in the ODI.
The supported JDBC patterns are:
jdbc:oracle:thin:@//adc2120612.us.example.com:19016/db8482.us.example.com
jdbc:oracle:thin:@adc2120612.us.example.com:19016:db8482
jdbc:weblogic:sqlserver://adc6140804.us.example.com:50457;databaseName=ODI_REPOSITORY
jdbc:weblogic:db2://slc02pfl.us.example.com:5031/orcl993
To resolve issues reported in this section:
If the ODI repositories are down, then act based on the statuses by either bringing up the databases, which are hosting the repositories, or troubleshooting why they are down and resolving the issues.
If there are any repositories that are undiscovered, then discover the databases, which are hosting the repositories, in Enterprise Manager Cloud Control.
If there are any repositories with alerts, then identify the root cause for those alerts and resolve the issues.
This region reports the following:
Number of Agents that are either up or down. Click the number for a list of the Agents.
Number of Agents that are not discovered as targets in Enterprise Manager. Click the number for a list of the Agents that have not been discovered.
Number of Agents with incidents. Click the number to find out which repositories have incidents.
To resolve issues reported in this section:
If the Agents are down, then act based on the statuses by either bringing up the Agents, which are down, or troubleshooting why they are down and resolving the issues.
If there are any Agents that are undiscovered, then either discover the Agents or refresh the Oracle WebLogic Domain that is linked to those Agents.
If there are any Agents with alerts, then identify the root cause for those alerts and resolve the issues.
This region reports the following:
Number of work repositories that are either up or down. Click the number for a list of the repositories.
Number of work repositories that have not been discovered in Enterprise Manager. Click the number of a list of the work repositories that have not been discovered.
Number of work Repositories with incidents. Click the number to find out which repositories have incidents.
To resolve issues reported in this section:
If the ODI repositories are down, then act based on the statuses by either bringing up the databases, which are hosting the repositories, or troubleshooting why they are down and resolving the issues.
If there are any repositories that are undiscovered, then discover the repositories in Enterprise Manager Cloud Control.
If there are any repositories with alerts, then identify the root cause for those alerts and resolve the issues.
This region reports the following:
Number of data servers that are either up or down. Click the number for a list of the servers.
Number of data servers that have not been discovered in Enterprise Manager. Click the number of a list of the data servers that have not been discovered.
Number of data servers with incidents. Click the number to find out which data servers have incidents.
To resolve issues reported in these sections:
If the data servers are down, then act based on the statuses by either bringing up the databases used by the data servers, or troubleshooting why they are down and resolving the issues.
If there are any data servers that are undiscovered, then discover the databases, which are used by the data servers, in Enterprise Manager Cloud Control.
If there are any data servers with alerts, then identify the root cause for those alerts and resolve the issues.
This region reports the following:
Number of sessions in error across all discovered ODI environments.
Number of sessions with error records across all discovered ODI environments.
Number of load plan executions in error across all discovered ODI environments.
Number of load plan executions with error records across all discovered ODI environments.
To monitor the ODI Agents, follow these steps:
From the Targets menu, select Middleware.
On the Middleware page, from the Middleware Features menu, select ODI Home.
On the ODI Home page, click the ODI Agents tab.
The ODI Agents tab has the following regions:
Use this region to search for agents for all Java EE and Standalone agents.
The latest specified search criteria are always retained. Specify a new criteria and click Search to see the updated results. Or, click Reset to reset the search form (you must still click Search to see the updated results). Note that the search criteria are reset each time you log out or navigate away from all the tabbed pages.
| Element | Description |
|---|---|
| Master Repository | Select the Master Repository. |
| Execution Agent | Select an Agent from the drop-down list. You can also select All to list all the Agents. |
| Agent Status | Select the status of the Agent: Up, Down, All. |
| Discovery Status | Select the status of the Agent: Discovered, Not Discovered, All. |
Use this region to view information about the ODI Agents declared in the Master Repository.
| Element | Description |
|---|---|
| Name | Displays the name of the Agent. Select an Agent to display the corresponding Agent Home page. |
| Status | Displays the current status of the Agent: Up, Down. |
| Discovery Status | A blue tick indicates that the Agent is discovered as a target in Enterprise Manager.
A clock indicates that the Agent is not discovered as a target in Enterprise Manager. |
| View Performance | Click the eye glass icon to view the performance data of the Agent. The metrics include:
|
| Active Sessions | Displays the number of active sessions. |
| Master Repository | A check mark indicates that the Master Repository is discovered.
A clock indicates that the Master Repository is not discovered. |
| Version | Displays the version and date of the Agent. |
| Response Time (ms) | Displays the repository database response time (in milliseconds). |
| User Defined Alerts | Displays the number of Critical and Warning alerts. Click the number to view the alerts in the Incident Manager page. |
To monitor the ODI repositories, follow these steps:
From the Targets menu, select Middleware.
On the Middleware page, from the Middleware Features menu, select ODI Home.
On the ODI Home page, click the Repositories tab.
Note:
The ODI database credentials have to be selected for this region to display. There are different credentials for different repositories. Choose the credentials based on your need.
Starting with Oracle Fusion Middleware Plug-in (12.1.0.6), you can monitor the repositories that are configured even with Microsoft SQL Server and IBM DB2. However, as a prerequisite, make sure you first deploy the Microsoft SQL Server Plug-in and IBM DB2 Plug-in, respectively, and then discover those database instances as targets in Enterprise Manager Cloud Control.
The Repositories tab has the following regions:
Use this region to search for repositories for all master and work repositories.
The latest specified search criteria are always retained. Specify a new criteria and click Search to see the updated results. Or, click Reset to reset the search form (you must still click Search to see the updated results). Note that the search criteria are reset each time you log out or navigate away from all the tabbed pages.
| Element | Description |
|---|---|
| Repository Type | Select the Repository type: Master Repository, Work Repository, All. |
| Repository Name | Enter the name or a part of the Repository name. |
| Repository Status | Select the status of the Repository: Up, Down, All. |
Use this region to view details of the work repositories.
| Element | Description |
|---|---|
| Name | Displays the name of the Master and Work Repository. A star icon against the name of the repository indicates that it is a non-Oracle Database repository.
|
| Status | Displays the status of the Work Repository database.
|
| Technology | Displays the technology used. |
| Host | Displays the name of the host on which the repository resides. |
| Port | Displays the port of the host on which the repository resides. |
| SID/Database Instance | Displays the system identifier of the repository or the database instance name. |
| Version | Displays the Repository version. |
| Response Time (ms) | Repository database response time in milliseconds. |
| External ID | Displays the ODI-specific unique identifier for the repository. |
| Incidents | Displays the number of incidents associated with this repository: Critical or Warning. |
| Schema Name | Displays the name of the schema associated with this repository. |
| LPE/Sessions Tablespace/File Group | Displays the total rows and segment size (in GB). |
| Purge | Click the icon to purge the ODI logs.
|
By looking at the database details, you have a clear picture of how your database is performing. For example, if the database tablespace is reaching near full, the Database Administrator can look at extending the table space.
In addition, by taking a look at the database performance chart, Throughput and Wait bottlenecks sections, the Database Administrator can recommend fine tuning the database.
Wait Bottlenecks
This section provides the following statistics: Average Instance (CPU%), Active Sessions Waiting I/O, and Active Sessions Waiting Others.
Throughput
This section provides the following statistics: Number of Transactions per second, Physical Writes per transaction, Physical Reads per transaction, and User Commits per transaction.
Performance
This section provides usage information for CPU, I/O Wait, and others for the active sessions.
Note: For this region to appear, you must select the credentials and the repository. The credentials must be of a DBA user and must be of the type Global. The credentials are required to depict the tablespace and schema-related charts.
Note:
For non-Oracle Database repositories, Enterprise Manager Cloud Control might not be able to display data for all the metricsThis section provides the growth rate for the tablespace by providing Space Used and Space Allocated statistics. Based on the information, you can decide whether to archive or purge the database data, or extend the tablespace.
Note:
For non-Oracle Database repositories, Enterprise Manager Cloud Control might not be able to display data for all the metricsTo monitor the load plan executions and sessions, follow these steps:
From the Targets menu, select Middleware.
On the Middleware page, from the Middleware Features menu, select ODI Home.
On the ODI Home page, click the Load Plan Executions/Sessions tab.
The Load Plan Executions/Sessions tab enables you to search and view information about the load plan executions and sessions executed by the Agent. This tab has the following regions:
Expand a session and review the Steps and Tasks information. For example if an ODI Interface was executed, you can review each task that this interface executed, view the generated code, and drill down to the database execution details.
Note:
Oracle Database Diagnostics and Tuning Packs are required to be able to use the Database Execution Details link and drill down into the Oracle Database monitoring pages.The Load Plan Executions/Sessions tab has the following regions.
Use this region to search for sessions and load plan executions for all master and work repositories.
The latest specified search criteria are always retained. Specify a new criteria and click Search to see the updated results. Or, click Reset to reset the search form (you must still click Search to see the updated results). Note that the search criteria are reset each time you log out or navigate away from all the tabbed pages of the Oracle Data Integrator Cloud Control application.
| Element | Description |
|---|---|
| Master Repository | Select the Master Repository containing the session information. |
| Work Repository | Select the Work Repository containing the session information. |
| Execution Agent | Select the Agent used to execute the session. |
| Context | Select the session's execution context |
| Execution Type | Select Sessions, Load Plan Executions, or All. |
| Begin Date | Use the calendar icon to select a date at which to start the search for sessions. Only session started after this date will be returned |
| End Date | Use the calendar icon to select a date at which to end the search for load plan executions and sessions. Only load plan executions and sessions ended before this date will be returned. |
| User Name | Name of the ODI user who started the execution. |
| Status | Select All or narrow the search to display specific statuses: Error, Running, Done, Warning, or Waiting. For example, you can select to view only Running and Warning statuses. |
| Message | Error message of the Load Plan Execution/Session run. |
| Keywords | Type keywords to narrow the search. When using multiple keywords, use a comma to separate each keyword, do not include spaces. For example use: lpe1,lpe2. |
| Execution Name | Type the name of the load plan execution. |
| Error Records | Select All or narrow the search to display load plan executions and sessions With Error Records or Without Error Records. |
| Execution ID | Specific Load Plan Execution or Session identifier. |
Use this region to view execution details of the Load Plan Executions and Sessions executed by the Agent.
To view more details such as hierarchy, status of each step, the start and end time of each step, and so on, for a particular Load Plan Execution or Session, select the row in the table and scroll down the page to see the Load Plan Executions/Session Detail table.
| Element | Description |
|---|---|
| Name | Displays the name of the Load Plan Execution or Session. |
| Execution ID | Load Plan Execution or Session identifier. Every time a Load Plan is executed, a new Load Plan Execution with a unique identifier is created. |
| Status | Displays an icon to indicate the status of the Load Plan Execution run or Session executed. Hover your mouse over the icon to understand the status and view more details if there is an error. The status can be one of the following:
|
| Started On | Start date and time of the Load Plan Execution/Session run. |
| Updated On | Displays the last updated date of the Load Plan Execution/Session. |
| Execution Time | Displays how long it took the Load Plan Execution/Session to run. |
| Error Records | Displays the number of error records. |
| Execution Type | Displays the Load Plan or Sessions type, for example, Scenario. |
| Work Repository Name | Displays the name of the Work Repository into which this Load Plan/Session run execution information is stored. |
| Agent Name | Displays the name of the agent on which the Load Plan Execution/Session ran. |
| ODI User | Displays the name of the ODI user who started the execution. |
Use this region to view more detailed information on the Load Plan Executions and Sessions executed by the Agent.
| Element | Description |
|---|---|
| Load Plan Executions/Session Hierarchy | Displays the hierarchy of the Load Plan Execution or Session. Click and expand the Load Plan Execution or Session name to view the complete hierarchy. |
| Status | Displays an icon to indicate the status of the Load Plan Execution or Session step. Hover your mouse over the icon to understand the status and view more details if there is an error. |
| Source Code | Displays the code executed on the source database. Click the icon to view details of the executed code.
If the source and target databases are Oracle Databases, which have been discovered in Enterprise Manager Cloud Control, then you will see a Database Execution Details hyperlink. Click the link to drill down to the ASH Analytics page and view information about the active sessions run for a particular time period. |
| Target Code | Displays the code executed in the target database. Click the icon to view details of the executed code.
If the source and target databases are Oracle Databases, which have been discovered in Enterprise Manager Cloud Control, then you will see a Database Execution Details hyperlink. Click the link to drill down to the ASH Analytics page and view information about the active sessions run for a particular time period. |
| Step Task Type | Displays the type of task performed by the step. The task type value is a hyperlink when the source and target systems are database systems. In that case, click the task type to view details of the source database and the target database that exchanged data. |
| Started On | Displays the date and time when the step started. |
| Ended On | Displays the date and time when the step ended. |
| Duration | Displays the time taken (in seconds) to execute the task. |
| Updates | Displays the number of updates or changes done to a row per task. |
| Inserts | Displays the number of data insertions done per task. |
| Error Records | Displays the number of error records reported per task. |
| Deletes | Displays the number of data deletions done per task. |
You can perform the following operations while administering Oracle Data Integrator:
Note:
Oracle Process Manager and Notification (OPMN) is used for release 11g Standalone Agents. WebLogic Management Framework is used for release 12c Colocated Standalone Agents only.
Only Start and Stop operations are supported for ODI Java EE Agents.
Start and Stop operations are supported for all ODI Standalone Agents managed by WebLogic Management Framework and OPMN instances. Restart operation is supported only for 11g Standalone Agents managed by OPMN instances, and not for 12c Colocated Standalone Agents managed by WebLogic Management Framework instances.
To start, stop, and restart Oracle Data Integrator Agents, follow these steps:
From the Targets menu, select Middleware.
On the Middleware page, from the Middleware Features menu, select ODI Home.
On the Oracle Data Integrator Home page, click the ODI Agents tab.
In the ODI Agents tab, search for the ODI agents. Then, in the ODI Agents table, click the name of an Agent.
On the ODI Agent Home page, from the ODI Agent menu, select Control, then select either Start Up, Shut Down, or Restart.
Note:
If you want to start or stop ODI Standalone Agents, that are not managed by OPMN or WebLogic Management Framework, you must use the Agent's startup and shutdown scripts. See "Managing Agents" in the Oracle Fusion Middleware Developer's Guide for Oracle Data Integrator for more information about how to start and shut down Agents.To manage the agent status and monitor its activities, follow these steps:
Click the target link corresponding to your JEE, Standalone, or Colocated Standalone Agent either in the target navigation pane or in the ODI Home Page. The Java EE Application Page for this agent appears.
From the Agent Page menu, select Monitoring then select Performance Summary.
Enterprise Manager Cloud Control displays the Performance Summary page, which enables you to view and customize the metrics and charts.
To sessions and load plan executions, follow these steps:
From the Targets menu on Enterprise Manager, select Middleware.
In the Middleware Features menu, select ODI Home.
Click the LPE/Sessions tab. For more information on the tab, click Help.
You can view log messages of Java EE agents in Enterprise Manager Cloud Control.
The steps for this process are:
From the Targets menu, select Middleware.
On the Middleware page, from the Middleware Features menu, select ODI Home.
On the Oracle Data Integrator Home page, click the ODI Agents tab.
In the ODI Agents tab, search for the ODI agents. Then, in the ODI Agents table, click the name of an Agent.
On the ODI Agent Home page, from the ODI Agent menu, select Logs, then select View Log Messages.
You can filter the displayed log messages, for example by date range and message type and search for a search term in the message.
To configure the log configuration settings, select Logs then select Log Configuration from the ODI Agent menu.
For detailed information on alerts and notifications, see Using Incident Management and Using Notifications chapters in the Oracle Enterprise Manager Cloud Control Administrator's Guide.
As an example, to create an alert for the Master Repository status, see the instructions below:
From the Targets menu, select Middleware.
On the Middleware page, from the Middleware Features menu, select ODI Home.
On the Oracle Data Integrator Home page, click the ODI Agents tab.
In the ODI Agents tab, search for the ODI agents. Then, in the ODI Agents table, click the name of an Agent.
On the ODI Agent Home page, from the ODI Agent menu, select Monitoring, then select Metric and Collection Settings.
In the Metric column, expand Master Repositories to see the Status row.
In the Critical Threshold text field, in the Status row, enter 0.
0 indicates that EM will generate an alert when the Master Repository is down, whereas 1 will generate an alert when the Master Repository is up.
Note:
Similarly, you can create warning or critical alerts for other rows mentioned in the Metric column.The Agents Home page enables you to monitor the Oracle Data Integrator run-time Agents. The Management Pack for ODI can monitor and manage the following ODI Agent types:
11g: Java EE Agents and Standalone Agents managed by OPMN.
12c: Java EE Agents and Collocated Standalone Agents managed by the WebLogic Management Framework.
To access the ODI Agent Home page, follow these steps:
From the Targets menu, select Middleware.
On the Middleware page, from the Middleware Features menu, select ODI Home.
On the ODI Home page, click the ODI Agents tab.
In the ODI Agents tab, search for ODI Agents, and in the search results table, click the name of the ODI Agent that interests you.
For further details on the agent home page, see Section 37.6, "Agent Home Page".
The Agent Home page is arranged in the following order:
The General Info region displays general information about this Agent.
| Element | Description |
|---|---|
| Response Time (ms) | Displays the repository database response time in milliseconds. |
| Agent Version | Displays the version of the Agent. |
| Host and Port | Displays the host (network name or IP address) of the machine where the Agent has been launched on and the port on which the Agent is listening. |
| Master Repository | Click to access the Database Performance page for the Master Repository. |
| Incidents | An event or a set of closely correlated events that represent an observed issue requiring resolution through (manual or automated) immediate action or root-cause problem resolution. |
The Load region displays the number of connections supported by the Agent over a period of time.
| Elements | Description |
|---|---|
| Maximum number of allowed sessions | Maximum number of sessions allowed on this Agent. |
| Maximum number of allowed threads | Maximum number of threads allowed on this Agent. |
| Count of active sessions | Number of active sessions on this Agent. |
| Count of active threads | Number of active threads on this Agent. |
The Target Incidents region displays notifications raised by the Agents attached to this Repository.
| Element | Description |
|---|---|
| Severity | Seriousness of the incident.
|
| ID | Incident ID. |
| Summary | Summary description of the incident. |
| Category | Classification of an incident, for example, Error. |
The Load Plan Executions/Sessions Execution Incidents region displays notifications raised by the Agents attached to this Repository.
| Element | Description |
|---|---|
| Severity | Seriousness of the incident.
|
| ID | Incident ID. |
| Summary | Summary description of the incident. |
| Category | Classification of an incident, for example, Error. |
The Load Balancing Agents region displays (if using ODI Load Balancing) the status and session metrics for the Agents declared as child Agents of the current Agent.
| Element | Description |
|---|---|
| Name | Displays the name of the agent. This is the name you specified when you created the Agent in Oracle Data Integrator. Select an Agent to display the corresponding Agent Home page. |
| Status | Displays the status of the Agent.
|
| Discovered | A blue tick indicates that the ODI Agent is discovered as a custom target in Enterprise Manager. Click the Agent name to access the ODI Console's Agent Detail Page.
A clock indicates that the ODI Agent is not discovered as a custom target in Enterprise Manager. Click the Agent name to access the Enterprise Manager Agent Target Page. |
| Originating LPEs/Sessions | Displays the status of the LPEs and Sessions.
|
| Avg Master Repo Response Time (ms) | Displays the master repository database response time in milliseconds. |
| Sessions | Maximum and active number of sessions allowed on this Agent. |
| Threads | Maximum and active number of threads allowed on this Agent. |
Oracle Data Integrator Console cannot be configured from Enterprise Manager Cloud Control. To make configuration changes you must use the Fusion Middleware Control Console. For information of how to configure Oracle Data Integrator, see Oracle Fusion Middleware Developer's Guide for Oracle Data Integrator.
However, you can configure Oracle Data Integrator Console from Enterprise Manager Cloud Control to define the linking between Enterprise Manager Cloud Control and Oracle Data Integrator Console.
By default, the fields on this page are populated with the Oracle Data Integrator Console host, the Oracle Data Integrator Console managed server port, and the default context root. If your Oracle Data Integrator Console must be accessed with a different configuration, you can change the configuration on this page.
The steps for this process are:
Navigate to the Agent home page.
From the Agent Page menu, select ODI Console Administration, then select Basic Configuration.
This page displays the current configuration for accessing the Oracle Data Integrator Console application. These values are automatically set when the application is discovered by Enterprise Manager and are used to access Oracle Data Integrator Console from Enterprise Manager, for example when clicking Browse.You can modify these values to access Oracle Data Integrator Console in a different way, for example to connect to Oracle Data Integrator Console by using a load balancer.
To modify this configuration, enter new values in the fields and click Apply. Click Revert to revert to the previous settings.
| Element | Description |
|---|---|
| Host | Displays the name of the server where your application is deployed. If using SSO, enter the Oracle HTTP Server (OHS). |
| Port | Displays the HTTP listener port number. If using SSO, enter the port of the machine where Oracle HTTP Server 10g or 11g Webgate is installed. |
| Context Root | Displays the Web application's context root. |
| Protocol | Displays the protocol of the connection |