This chapter describes how to manage export and import operations in Oracle Data Integrator. An introduction to the import and export concepts is provided.
This chapter includes the following sections:
This section introduces you to the fundamental concepts of export and import operations in Oracle Data Integrator. All export and import operations require a clear understanding of the concepts introduced in this section.
Before performing export and import operations, it is essential to understand in detail the concept of internal identifiers (ID) in Oracle Data Integrator.
To ensure object uniqueness across several work repositories, ODI uses a specific mechanism to generate unique IDs for objects (such as technologies, data servers, Models, Projects, Integration Interfaces, KMs, etc.). Every object in Oracle Data Integrator is identified by an internal ID. The internal ID appears on the Version tab of each object.
ODI Master and Work Repositories are identified by their unique internal IDs. This RepositoryID of 3 digits must be unique across all work repositories of an ODI installation and is used to compute the internal ID of an object.
The internal ID of an object is calculated by appending the value of the RepositoryID to an automatically incremented number: <UniqueNumber><RepositoryID>
If the Repository ID is shorter than 3 digits, the missing digits are completed with "0". For example, if a repository has the ID 5, possible internal IDs of the objects in this repository could be: 1005, 2005, 3005, ..., 1234567005. Note that all objects created within the same repository have the same three last digits, in this example 005.
This internal ID is unique for the object type within the repository and also unique between repositories for the object type because it contains the repository unique ID. This mechanism allows to:
Avoid any ID conflicts when exporting and importing from one repository to another
Understand the provenance of every object, simply by looking at its Internal ID. The 3 last digits always refer to the repository where it was created.
Important Export/Import Rules and Guidelines
Due to the structure of the object IDs, these guidelines should be followed:
Work repositories must always have different internal IDs. Work repositories with the same ID are considered to contain same objects.
If an export/import operation is performed between two Master/Work repositories possessing identical internal IDs, there is a risk of overwriting objects when importing. Objects from both repositories that have same IDs are considered the same.
Oracle Data Integrator stores all objects in a relational database schema (the Repository) with dependencies between objects. Repository tables that store these objects maintain these dependencies as references using the IDs. When you drag and drop a target datastore into an integration interface, only the reference to the ID of this datastore is stored in the interface object. If you want to export this interface, and import it in Synonym mode into another work repository, a datastore with the same ID must already exist in this other work repository otherwise the import will create a missing reference. The missing references can be resolved either by fixing the imported object directly or by importing the missing object.
Solutions allow to export and import sets of dependent objects automatically. Using solutions and versioning is the recommended way of maintaining the dependencies automatically when doing export/import. See Chapter 18, "Working with Version Management".
Therefore, the Model or Sub-model holding this Datastore needs to be exported and imported in Synonym mode prior to importing the integration interface.
There are also dependencies between work repository objects and master repository objects. Dependencies within a work repository are ID-based. Dependencies between objects of the work and objects of the master are based on the Codes and not the IDs. This means that only the Code of the objects (for example ORACLE is the code of the Oracle technology in the master) of the master repository objects are referenced in the work repository.
It is important to import objects in the appropriate order. You can also use Solutions to preserve these dependencies. Table 19-1 lists the dependencies of an integration interface to other objects when importing the integration interface in synonym mode.
Table 19-1 Dependencies of an integration interface in the work and Master Repository
| Dependencies on other objects of Work Repository when importing in Synonym Mode | Dependencies on objects of the Master Repository |
|---|---|
|
|
Oracle Data Integrator can import objects, the topology or repositories using several modes.
Read carefully this section in order to determine the import mode you need.
This section provides tips for the import and export operations.
As a general rule, always use different internal IDs for your repositories in order to avoid any ID conflicts.
The import report is displayed after every import operation. It is advised to read it carefully in order to determine eventual errors of the import process.
The import report gives you details on the:
Import Mode
Imported Objects. For every imported object the object type, the original object name, the object name used for the import, the original ID, and the new, recalculated ID after the import is given.
Deleted Objects. For every deleted object the object type, the object name, and the original ID is given.
Created Missing References lists the missing references detected after the import.
Fixed Missing References lists the missing references fixed during the import.
You can save the import report as an.xml or .html file. Click Save... to save the import report.
In order to avoid missing references, use solutions to manage dependencies. See Section 18.4, "Working with Solutions" for more information.
Choose the import mode carefully. See Section 19.1.3, "Import Modes" for more information.
Exporting and importing Oracle Data Integrator objects means transferring objects between different repositories.
When exporting an Oracle Data Integrator object, an XML export file is created. ODI objects have dependencies, as described in Section 19.1.2, "Relationships between Objects". These dependencies will be exported in the XML export file.
The content of this XML file will depend on the export method you will use:
The choice will depend on your goal, if you need to do a partial export then the Export Without Child Components is the one to use.
The Export Multiple ODI Objects feature is useful when you need to regularly export the same set of Objects.
Once the export has been performed, it is very important to choose the import strategy to suite your requirements.
Exporting an Object with its Child Components
This option is the most common when you want to export an object. It allows you to export all subcomponents of the current object along with the object itself.
When an Object is exported with its child components, all container-dependent Objects – those which possess a direct parent/child relationship - are also exported. Referenced Objects are not exported.
For example, when you choose to export a Project with its child components, the export will contain the Project definition as well as all objects included in the Project, such as Folders, Interfaces, Procedures, Packages, Knowledge Modules, Variables, Sequences, Functions, etc. However, this export will not contain dependent objects referenced which are outside of the Project itself, such as Datastores and Columns, as defined previously in Section 19.1.2, "Relationships between Objects". Only the numeric ID references of these Objects will be exported.
Exporting an Object without its Child Components
This option can be useful in some particular situations where you would want to take control of the import process. It allows you to export only the top-level definition of an object without any of its subobjects.
For example, if you choose to export a Model without its children, it will only contain the Model definition but not the underlying Sub-models and Datastores when you import this model to a new repository.
If you have a very large project that contains thousands of interfaces and you only want to export a subset of these to another work repository, you can either export the entire Project and then import it, or choose to do a partial manual export/import as follows:
Export all Models referenced by the sub-items of your project and import them in “Synonym mode” in the new repository to preserve their IDs
Export the Project without its children and import it in “Synonym mode”. This will simply create the empty Project in the new repository (with the same IDs as in the source).
Export every first level Folder you want, without its children, and import them in “Synonym mode”. The empty Folders will be created in the new repository.
Export and Import all Markers, Knowledge Modules, Variables, Sequences, etc. that are referenced by every object you plan to export, and import them in “Synonym mode”. Refer to section “Import in “Synonym Mode” or Duplication and Impact on Object IDs” for special caution regarding import of Knowledge Modules in Synonym Mode.
Finally, export the Interfaces you are interested in and import them in “Synonym mode” in the new repository.
Exporting one Oracle Data Integrator Object means export one single ODI object in order to transfer it from one repository to another.
To export an object from Oracle Data Integrator:
Select the object to be exported in the appropriate Oracle Data Integrator Navigator.
Right-click the object, and select Export...
If this menu item does not appear, then this type of object does not have the export feature.
In the Export dialog, set the Export parameters as indicated inTable 19-2.
Table 19-2 Object Export Parameters
| Properties | Description |
|---|---|
|
Export Directory |
Directory in which the export file will be created. |
|
Export Name |
Name given to the export |
|
Child Components Export |
If this option is checked, the objects linked to the object to be exported will be also exported. These objects are those visible under the exported object in the tree. It is recommended to leave this option checked. Refer to Exporting an Object with its Child Components for more details. Note that when you are exporting a Load Plan, scenarios will not be exported even if you check this option. |
|
Replace exiting files without warning |
If this option is checked, the existing file will be replaced by the ones of the export. If a file with the same name as the export file already exists, it will be overwritten by the export file. |
|
Advanced options |
This set of options allow to parameterize the XML output file format. It is recommended that you leave the default values. |
|
XML Version |
XML Version specified in the export file. Parameter .xml version in the XML file header.
|
|
Character Set |
Encoding specified in the export file. Parameter encoding in the XML file header.
|
|
Java Character Set |
Java character set used to generate the file |
You must at least specify the Export Name.
Click OK.
The object is exported as an XML file in the specified location.
You can export one or more objects at once, using the Export Multiple... menu item. This lets you export ODI objects to a zip file or a directory, and lets you re-use an existing list of objects to export.
A more powerful mechanism for doing this is using Solutions. See Section 18.4, "Working with Solutions" for more information.
How to export multiple objects at once:
Select Export Multiple... from the Designer, Topology, Security or Operator Navigator toolbar menu.
In the Export Multiple Objects dialog, specify the export parameters as indicated in Table 19-2.
The objects are either exported as .xml files directly into the directory, or as a zip file containing .xml files. If you want to generate a zip file, you need to select Export as zip file and enter the name of the zip file in the Zip file name field.
Specify the list of objects to export:
Drag and drop the objects from the Oracle Data Integrator Navigators into the Export list. Note that you can export objects from different Navigators at once.
Click Load a list of objects... to load a previously saved list of objects. This is useful if you regularly export the same list of objects.
To save the current list of objects, click SaveExport List and specify a file name. If the file already exists, it will be overwritten without any warning.
To import multiple objects at once, you must use a Solution. See Section 18.4, "Working with Solutions" for more information.
Importing and exporting allows you to transfer objects (Interfaces, Knowledge Modules, Models,...) from one repository to another.
To import an object in Oracle Data Integrator:
In the Navigator, select the object or object node under which you want to import the object.
Right-click the object, and select Import, then the type of the object you wish to import.
In the Import dialog:
Select the Import Mode. Refer to Section 19.1.3, "Import Modes" for more information.
Enter the File Import Directory.
Select the file(s) to import from the list.
Click OK.
The XML-formatted files are imported into the work repository, and the imported objects appear in the Oracle Data Integrator Navigators.
Note that the parent or node under which objects are imported is dependent on the import mode used. When using DUPLICATION mode, the objects will be imported into where the Import option was selected. For Synonym imports, the objects will be imported under the parent specified by the objects parent id in the import file.
To import a Knowledge Module into Oracle Data Integrator:
In Designer Navigator, select the project into which you want to import the KM.
Right-click the project, and select Import > Import Knowledge Modules....
In the Import dialog:
The Import Mode is set to Duplication. Refer to Section 19.1.3, "Import Modes" for more information.
Enter the File Import Directory.
Select the Knowledge Module file(s) to import from the list.
Click OK.
The Knowledge Modules are imported into the work repository and appear in your project under the Knowledge Modules node.
Importing a KM in Replace Mode
Knowledge modules are usually imported into new projects in Duplication mode.
When you want to replace a KM in a project by another one and have all interfaces automatically use the new KM, you must use the Import Replace mode.To import a Knowledge Module in Replace mode:
Select the Knowledge Module you wish to replace.
Right-click the Knowledge Module and select Import Replace...
In the Replace Object dialog, specify the Knowledge Module export file.
Click OK.
The Knowledge Module is now replaced by the new one.
WARNING:
Replacing a Knowledge module by another one in Oracle Data Integrator sets the options in the new module using the option name similarities with the old module's options. New options are set to the default value.
It is advised to check the values of these options in the interfaces as well as the interfaces' design and execution with the new KM.
Refer to the Import Replace mode description in the Section 19.1.3, "Import Modes" for more information.
At repository level you can export and import the master repository and the work repositories.
The master repository export/import procedure allows you to transfer the whole repository (Topology and Security domains included) from one repository to another.
It can be performed in Topology Navigator, to import the exported objects in an existing repository, or while creating a new master repository.
Exporting the Master Repository in Topology Navigator
The objects that are exported when exporting the master repository are objects, methods, profiles, users, languages, versions (if option selected), solutions (if option selected), open tools, password policies, entities, links, fields, lookups, technologies, datatypes, datatypes conversions, logical agents, contexts and the child objects.
To export a master repository:
From the Topology Navigator toolbar menu select Export > Master Repository...
In the Export dialog, set the Export parameters as indicated inTable 19-2.
The master repository and its topology and security settings are either exported as .xml files directly into the directory, or as a zip file containing .xml files. If you want to generate a zip file, you need to select Export to zip file and enter the name of the zip file in the Zip File Name field.
Select Export versions, if you want to export all stored versions of objects that are stored in the repository. You may wish to unselect this option in order to reduce the size of the exported repository, and to avoid transferring irrelevant project work.
Select Export solutions, if you want to export all stored solutions that are stored in the repository. You may wish to unselect this option for similar reasons.
Click OK.
The export files are created in the specified export directory.
Importing the Master Repository
To import the exported master repository objects into an existing master repository:
From the Topology Navigator toolbar menu select Import > Master Repository...
In the Import dialog:
Select the Import Mode. Refer to Section 19.1.3, "Import Modes" for more information.
Select whether you want to import the files From a Folder or From a ZIP file.
Enter the file import folder or zip file.
Click OK.
The master repository contains now the objects you have imported.
Creating a new Master Repository using a previous Master export
To create a new master repository using an export of another master repository:
Open the New Gallery by choosing File > New.
In the New Gallery, in the Categories tree, select ODI.
Select from the Items list the Master Repository Import Wizard.
Click OK.
The Master Repository Import Wizard appears.
Specify the Database Connection parameters as follows:
Login: User ID/login of the owner of the tables you have created for the master repository
JDBC Driver: The driver used to access the technology, which will host the repository.
JDBC URL: The complete path for the data server to host the repository.
Note that the parameters JDBC Driver and URL are synchronized and the default values are technology dependant.
User: The user id/login of the owner of the tables.
Password: This user's password.
DBA User: The database administrator's username
DBA Password: This user's password
Specify the Repository Configuration parameters as follows:
ID: A specific ID for the new master repository, rather than the default 0. This will affect imports and exports between repositories.
WARNING:
All master repositories should have distinct identifiers. Check that the identifier you are choosing is not the identifier of an existing repository
Use a Zip File: If using a compressed export file, check the Use a Zip File box and select in the Export Zip File field the file containing your master repository export.
Export Path: If using an uncompressed export, select the directory containing the export in the Export Path field.
Technology: From the list, select the technology your repository will be based on.
Click Test Connection to test the connection to your master repository.
The Information dialog opens and informs you whether the connection has been established.
Click Next.
Specify the password storage details:
Select Use Password Storage Configuration specified in Export if you want to use the configuration defined in the export.
Select Use New Password Storage Configuration if you do not want to use the configuration defined in the export and select
Internal Password Storage if you want to store passwords in the Oracle Data Integrator repository
External Password Storage if you want use JPS Credential Store Framework (CSF) to store the data server and context passwords. Indicate the MBean Server Parameters to access the credential store as described in Table 23-2.
Refer to the Section 23.3.1, "Setting Up External Password Storage" for more information on password storage details.
In the Master Repository Import Wizard click Finish to validate your entries.
A new repository is created and the exported components are imported in this master repository.
Exporting then importing the topology or security allows you to transfer a domain from one master repository to another.
Exporting the Topology and Security Settings
The domains that can be exported are given below:
Topology: the full topology (logical and physical architectures including the local repository, data servers, hosts, agents, generic actions, technologies, datatypes, logical schemas, and contexts).
Logical Topology: technologies (connection, datatype or language information), logical agents, logical schemas, actions and action groups.
Security: objects, methods, users, profiles, privileges, password policies and hosts.
Execution Environment: technologies, data servers, contexts, generic actions, load balanced agents, physical schemas and agents.
To export the topology/security:
From the Topology or Security Navigator toolbar menu select Export and then one of the following options:
Topology...
Logical Topology...
Security Settings...
Execution Environment...
In the Export dialog, specify the export parameters as indicated in Table 19-2.
The topology and security settings are either exported as .xml files directly into the directory, or as a zip file containing .xml files. If you want to generate a zip file, you need to select Export to zip file and enter the name of the zip file in the Zip File Name field.
Click OK.
The export files are created in the specified export directory.
To import a topology export:
From the Topology Navigator toolbar menu select Import and then one of the following options:
Topology...
Logical Topology...
Execution environment...
In the Import dialog:
Select the Import Mode. Refer to Section 19.1.3, "Import Modes" for more information.
Select whether to import the topology export from a Folder or a Zip File.
Enter the file import directory.
Click OK.
The specified files are imported into the master repository.
Importing the Security Settings
To import a Security export:
From the Security Navigator toolbar menu select Import > Security Settings...
In the Import dialog:
Select the Import Mode. Refer to Section 19.1.3, "Import Modes" for more information.
Select whether to import the security export from a Folder or a Zip File.
Enter the file import directory.
Click OK.
The specified files are imported into the master repository.
Importing or exporting a work repository allows you to transfer all work repository objects from one repository to another.
To export a work repository:
From the Designer Navigator toolbar menu select Export > Work Repository...
In the Export dialog, set the Export parameters as indicated inTable 19-2.
The work repository with its models and projects are either exported as .xml files directly into the directory, or as a zip file containing .xml files. If you want to generate a zip file, you need to select Export to zip file and enter the name of the zip file in the Zip File Name field
Click OK.
The export files are created in the specified export directory.
To import a work repository:
From the Designer Navigator toolbar menu select Import > Work Repository...
In the Import dialog:
Select the Import mode. Refer to Section 19.1.3, "Import Modes" for more information.
Select whether to import the work repository from a Folder or a Zip File.
Enter the file import directory.
Click OK.
The specified files are imported into the work repository.
This feature produces a comma separated (.csv) file in the directory of your choice, containing the details of the technical environment. This information is useful for support purposes.
You can customize the format of this file.
To produce the technical environment file:
From the Topology Navigator toolbar menu select Export >Technical Environment...
In the Technical environment dialog, specify the export parameters as indicated in Table 19-3:
Table 19-3 Technical Environment Export Parameters
| Properties | Description |
|---|---|
|
Export Directory |
Directory in which the export file will be created. |
|
File Name |
Name of the |
|
Advanced options |
This set of options allow to parameterize the XML output file format. It is recommended that you leave the default values. |
|
Character Set |
Encoding specified in the export file. Parameter encoding in the XML file header.
|
|
Field codes |
The first field of each record produced contains a code identifying the kind of information present on the row. You can customize these codes as necessary.
|
|
Record Separator and Field Separator |
These separators define the characters used to separate records (lines) in the file, and fields within one record. |
Click OK.