1 Getting Started with Oracle Data Integrator Cloud
Learn about Oracle Data Integrator Cloud, what it is and how to use it.
What is Oracle Data Integrator Cloud
Data integration ensures that information is timely, accurate, and consistent across complex systems. This section provides an introduction to data integration and describes how Oracle Data Integrator Cloud provides support for data integration.
What is Data Integration?
Integrating data and applications throughout the enterprise, and presenting them in a unified view is a complex proposition. Not only are there broad disparities in technologies, data structures, and application functionality, but there are also fundamental differences in integration architectures. Some integration needs are Data Oriented, especially those involving large data volumes. Other integration projects lend themselves to an Event Driven Architecture (EDA) or a Service Oriented Architecture (SOA), for asynchronous or synchronous integration.
Data Integration ensures that information is timely, accurate, and consistent across complex systems. Although it is still frequently referred as Extract-Transform-Load (ETL), data integration was initially considered as the architecture used for loading Enterprise Data Warehouse systems. Data integration now includes data movement, data synchronization, data quality, data management, and data services.
About Oracle Data Integrator Cloud
Oracle Data Integrator Cloud provides a fully unified cloud-based solution for building, deploying, and managing complex data warehouses or as part of data-centric architectures in a SOA or business intelligence environment.
It combines all the elements of data integration — data movement, data synchronization, data quality, data management, and data services—to ensure that information is timely, accurate, and consistent across complex systems.
Oracle Data Integrator Cloud features an active integration platform that includes all styles of data integration: data-based, event-based and service-based. It unifies silos of integration by transforming large volumes of data efficiently, processing events in real time through its advanced Changed Data Capture (CDC) framework, and providing data services to the Oracle SOA Suite. It also provides robust data integrity control features, assuring the consistency and correctness of data. With powerful core differentiators - heterogeneous E-LT, Declarative Design and Knowledge Modules - Oracle Data Integrator Cloud meets the performance, flexibility, productivity, modularity and hot-pluggability requirements of an integration platform.
What is E-LT?
Traditional ETL tools operate by first Extracting the data from various sources, Transforming the data in a proprietary, middle-tier ETL engine that is used as the staging area, and then Loading the transformed data into the target data warehouse, integration server, or Hadoop cluster. Hence the term ETL represents both the names and the order of the operations performed, as shown in Figure 1-1.
Figure 1-1 Traditional ETL versus ODI E-LT
Description of "Figure 1-1 Traditional ETL versus ODI E-LT "
The data transformation step of the ETL process is by far the most compute-intensive, and is performed entirely by the proprietary ETL engine on a dedicated server. The ETL engine performs data transformations (and sometimes data quality checks) on a row-by-row basis, and hence, can easily become the bottleneck in the overall process. In addition, the data must be moved over the network twice – once between the sources and the ETL server, and again between the ETL server and the target data warehouse or Hadoop cluster. Moreover, if one wants to ensure referential integrity by comparing data flow references against values from the target data warehouse, the referenced data must be downloaded from the target to the engine, thus further increasing network traffic, download time, and leading to additional performance issues.
In response to the issues raised by ETL architectures, a new architecture has emerged, which in many ways incorporates the best aspects of manual coding and automated code-generation approaches. Known as E-LT, this new approach changes where and how data transformation takes place, and leverages existing developer skills, RDBMS and Big Data engines, and server hardware to the greatest extent possible. In essence, E-LT moves the data transformation step to the target RDBMS, changing the order of operations to: Extract the data from the source tables, Load the tables into the destination server, and then Transform the data on the target RDBMS using native SQL operators. Note, with E-LT there is no need for a middle-tier engine or server as shown in Figure 1-1.
Oracle Data Integrator supports both ETL- and E-LT-Style data integration. See the Designing E-LT and ETL-Style Mappings section in Developing Integration Projects with Oracle Data Integrator for more information.
Before you Begin with Oracle Data Integrator Cloud
There are a few prerequisites you need before you can use Oracle Data Integrator Cloud.
How to Begin with Oracle Data Integrator Cloud
Here’s how to get started with free Oracle Data Integrator Cloud promotions and subscriptions:
-
Sign up for a free credit promotion or purchase a subscription.
See Requesting and Managing Free Oracle Cloud Promotions or Buying an Oracle Cloud Subscription in Getting Started with Oracle Cloud
-
Access the Oracle Data Integrator Cloud service.
To grant access to others:
-
Learn about users.
See Users.
-
Create accounts for your users and assign them appropriate privileges and roles.
See Managing Users, User Accounts, and Roles in Getting Started with Oracle Cloud
Accessing Oracle Data Integrator Cloud
You can access Oracle Data Integrator Cloud through emails that you receive after subscribing or through the service web console.
To access Oracle Data Integrator Cloud:
-
Open your web browser and go to http://cloud.oracle.com.
-
Click Sign In.
-
From the Cloud Account menu, select Cloud Account with Identity Cloud Service.
-
Enter the name of your Cloud Account in the Cloud Account Name field.
-
Click My Services.
-
On the Sign In to Oracle Cloud page, enter your sign-in credentials.
If you don’t have your welcome mail to the My Services application, then contact your administrator.
Users
Administrators, Developers and Operators use the Oracle Data Integrator Studio to access the repositories. This Fusion Client Platform (FCP) based UI is used for administering the infrastructure (security and topology), reverse-engineering the metadata, developing projects, scheduling, operating and monitoring executions.
Business users (as well as developers, administrators and operators), can have read access to the repository, perform topology configuration and production operations through a web based UI called Oracle Data Integrator Console. This Web application can be deployed in a Java EE application server such as Oracle WebLogic.
ODI Studio provides four Navigators for managing the different aspects and steps of an ODI integration project:
Topology Navigator
Topology Navigator is used to manage the data describing the information system's physical and logical architecture. Through Topology Navigator you can manage the topology of your information system, the technologies and their datatypes, the data servers linked to these technologies and the schemas they contain, the contexts, the language and the agents, as well as the repositories. The site, machine, and data server descriptions will enable Oracle Data Integrator to execute the same mappings in different environments.
Designer Navigator
Designer Navigator is used to design data integrity checks and to build transformations such as for example:
-
Automatic reverse-engineering of existing applications or databases
-
Graphical development and maintenance of transformations and mappings
-
Visualization of data flows in the mappings
-
Automatic documentation generation
-
Customization of the generated code
The main objects you handle through Designer Navigator are Models and Projects.
Operator Navigator
Operator Navigator is the production management and monitoring tool. It is designed for IT production operators. Through Operator Navigator, you can manage your executions in the sessions, as well as the scenarios in production.
Security Navigator
Security Navigator is the tool for managing the security information in Oracle Data Integrator. Through Security Navigator you can create users and profiles and assign user rights for methods (edit, delete, etc) on generic objects (data server, datatypes, etc), and fine-tune these rights on the object instances (Server 1, Server 2, etc).