Skip Headers
Oracle® Fusion Middleware Upgrade Guide for Oracle Enterprise Repository
11g Release 1 (11.1.1.6.3)

Part Number E15746-12
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Master Index
Master Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

3 Data Migration Tool

This chapter describes how to use the data migration utility to migrate assets from earlier versions of Oracle Enterprise Repository.

This chapter contains the following sections:

3.1 Overview

The Data Migration Tool migrates the source asset and artifact file information to new OER 11g Asset Models (Harvested Models), when the file information artifacts are remote and links them with a special relationship (harvester-migrator), for tracking. Here is a sample view of a pre and post migration asset model.

Figure 3-1 Ad-hoc Created Asset View–(Pre-Migration View)

Surrounding text describes Figure 3-1 .

Figure 3-2 OER 11g Asset View – (Post-Migration View)

Surrounding text describes Figure 3-2 .

Key Features of the Data Migration Tool

The Data Migration Tool assists you in these tasks:

3.1.1 Best Practices

In the Oracle Enterprise Repository 11g R1 release, the harvester was extended to support Oracle SOA Suite 11g and Oracle Service Bus 10g. The Exchange Utility was enhanced to support Oracle Service Bus. The Enterprise Manager Integration utility was added to integrate with metrics in Enterprise Manager 10g. To support these products, the Oracle Enterprise Repository asset model was extended, which is known as the 11g R1 Asset Model.

This section describes the key points about Oracle Enterprise Repository content migration in an upgrade:

  • The custom types are left intact when upgrading to 11g. Also, the extensions, for example, the addition of metadata attributes, but not necessarily deletions, made in the type manager to standard out-of the box asset types also remain intact. The instances of these types with their extended metadata are ordinarily preserved in an upgrade.

  • Some or all instances of service assets/artifacts that you previously harvested or captured at design time must be harvested or reharvested with deployment/endpoint information. This is true, for example, if you and your customer require Oracle Enterprise Repository/Oracle Service Registry use cases such as consumption in Oracle SOA Suite/JDeveloper, registry/repository exchange between Oracle Enterprise Repository and Oracle Service Registry, and other more advanced lifecycle use cases that are beyond simple visibility in Oracle Enterprise Repository).

  • To behave properly in Oracle Enterprise Repository 11g, artifact files must either be hosted remotely on an HTTP/HTTPS/FTP site or harvested directly from deployment instances. In addition, endpoints must be present from concrete WSDL in order for Oracle Enterprise Repository to publish to Oracle Service Registry.

    Typically, this re-harvesting is recommended for any services/artifacts that have locally cached artifact files in file info.

  • If you have manually entered service assets/artifacts (non-harvested) of the same types that are typically used by the harvester, then you can manually plug-in a remote reference to the concrete WSDL hosted on an HTTP/HTTPS/FTP site before an upgrade to Oracle Enterprise Repository 11g and the harvester Migration Tool (11.1.1.x.x-OER-Migrate.zip) creates the new Harvester model.

3.1.2 Prerequisites

Before using the migration tool, you must perform the following prerequisites:

  • The Oracle Enterprise Repository server instance must first be upgraded to 11g R1, and must be started.

  • The Oracle Enterprise Repository database must be backed up, so it can be restored if there are any problems with the migration.

  • The migration tool requires the Java JRE or JDK version 6 or higher.

  • The migration tool connects to Oracle Enterprise Repository using its WebService API. So it can be run on any computer that can access the Oracle Enterprise Repository server. It runs much faster if run on the same computer as the Oracle Enterprise Repository server.

  • The Oracle Enterprise Repository System Setting, cmee.extframework.enabled, must be set to true. This can be set from the Web UI in the Admin, System Settings page.

  • Only assets with a status of Active will be migrated.

3.1.3 High Level Use Cases

You can use the migration tool to:

  • Migrate assets that were imported into Oracle Enterprise Repository through the ALER 3.x WSDL importer in the Web console.

  • Migrate assets that were imported into Oracle Enterprise Repository through the ALER 3.x UDDI importer in the Web console.

  • Migrate assets that were imported into Oracle Enterprise Repository through the ALER 3.x BPEL importer in the Web console.

  • Migrate assets that were imported into Oracle Enterprise Repository through the ALER 3.x ALRR-XU (version 3.x).

  • Migrate assets that were imported into Oracle Enterprise Repository through the ALER 3.x ad-hoc "Submit an Asset" functionality in the Web console.

  • Migrate assets that were imported into Oracle Enterprise Repository through the 10g R3 Harvester.

  • Migrate assets that were imported into Oracle Enterprise Repository through the 10g R3 Exchange Utility.

  • Migrate WSDL assets that were imported into Oracle Enterprise Repository through the Open API (REX) WSDL import functionality.

  • Preview the results of a migration, without committing any changes to Oracle Enterprise Repository.

3.1.4 Migration Tool Functionality

This section describes the migration tool functionality for ALER 3.x assets and Oracle Enterprise Repository 11g assets.

3.1.4.1 Migration Tool Functionality - ALER 3.x Assets

In the past, Oracle Enterprise Repository had a variety of tools for importing metadata. In version 3.x of ALER, these included Web and Open API (REX) utilities for importing from WSDL, BPEL, and UDDI and the command-line RRXU tool. These import service metadata into an asset model called the ALER 3.x Asset Model, as shown.

Figure 3-3 ALER 3.x Asset Model

Surrounding text describes Figure 3-3 .

Service Assets

The contents of the WSDL that is pointed to by the Service FileInfo is introspected from its download URI. An interface asset is created, and related to the service. An Artifact: WSDL asset is introspected from its download URI. An Artifact: WSDL asset is created and related to the endpoint. A relationship, Harvester-Migrator, is created between the original Service asset and the new Service asset that is created by the introspection.

Business Process Assets

The contents of the BPEL that is pointed to by a business process asset is introspected from its download URI. An Artifact: BPEL asset is created and related to the business process. The business process asset has its asset type changed to Business Process: BPEL. A relationship, Harvester-Migrator, is created between the original BPEL asset and the new Business Process: BPEL asset that is created by the introspection.

Imports

Any WSDLs or XSDs imported by the WSDL results in new artifact assets and relationships, as in the Harvester (or points to existing assets if their fingerprints match).

PartnerLinks

Any WSDLs used by a BPEL in partnerlinks results in new artifact assets and relationships, as in the Harvester.

FileInfos

The FileInfo is removed from the service and endpoint assets. The new Artifact: WSDL asset contains the updated FileInfo.

Fingerprints

The SFID (fingerprint) is removed from the service and endpoint assets. Any Artifact: WSDL assets and Artifact: XSD assets contain a new SFID, using the Harvester's SFID algorithm.

Internal Names

Any non-artifact asset that is migrated gets a new internal name, which is used for duplicate checking.

Names

Pre-existing service, endpoint, and business process assets keep their original name. New assets are named according to the Harvester's naming rules.

WSDL Summary

WSDL Summary metadata entries are created on the migrated assets, as in Harvester.

Manifest Metadata

Manifest metadata entries are created on the migrated assets, as in Harvester, to support the new download functionality in Oracle Enterprise Repository.

Ad-hoc Assets (through "Submit an Asset"): Limited Support

Assets that were imported into Oracle Enterprise Repository in an ad-hoc manner are migrated as described in Section 3.3.2.1, "Asset Types to Migrate", only if they conform to the ALER 3.x Asset model. For example, if an ad-hoc service asset contains a FileInfo that points to a WSDL, it is migrated as described above. But an asset of another type, for example, "TestCase", are not migrated.

3.1.4.2 Migration Tool Functionality - Oracle Enterprise Repository 10g R3 Assets

Fingerprints

Recalculates and store the SFID for artifact assets from the original artifact files, using the 11gR1 fingerprinting code.

Manifests

Updates the internal.introspector.manifest.store CMF entry and FileInfo with the correct artifact location according to the following 11g R1 harvester rules:

  • Artifacts that were harvested from remote URLs have the FileInfo updated to point to the remote location (to match the Manifest).

  • Artifacts that were harvested from local files have the Manifest updated to point to the repository download URL (to match the FileInfo).

  • Deletes obsolete internal.artifact.store CMF entries for artifacts that were harvested from remote URLs.

Exchange Stores

Updates the internal.alrr.exchange.store CMF entry to conform to the latest Exchange Utility rules.

  • Sets <tmodel> entries in the internal.alrr.exchange.store CMF entry on Service and Endpoint assets, with the UDDI keys of porttype and binding TModels. (Connects to the UDDI registry to get these).

  • Sets the <uddiRegistries> custom data table on Endpoint assets, including the registry-name, registry-url, and service-key elements. The registry-name and registry-url elements are set from the original internal.alrr.exchange.store CMF entry. The service-key is read from the UDDI registry during migration.

  • Sets the <uddi> custom data elements on Service assets, including the business-key and service-key elements. These are read from the UDDI registry during migration.

  • Sets the <uddi> custom data elements on Business Entity assets, specifically the business-key element. This is read from the UDDI registry during migration.

Harvester Properties

Converts the internal.introspector.store CMF entry to the new format used by 11gR1 harvester.

  • Sets the intname harvester property for abstract assets, according to the 11gR1 harvester rules.

  • Calculates and sets the following harvester properties, which are automatically set by 11gR1 harvester, on the appropriate assets: Namespace, Short Name, SCA Name, Endpoint URI, Deployment URL, Transport Type, Interface Type, Service Type.

  • Updates the wsdlLocation element in the WSDLSummary, based on the download URL calculated in "Manifests".

Asset Type

Updates the asset types according to the 11g R1 model:

  • Changes Endpoint: WebService assets to Endpoint

  • Changes Interface: WebService assets to Interface

3.2 Using the Data Migration Tool

This section contains the following topics:

3.2.1 Running from Command Line

The migration tool is available in the 11.1.1.x.x-OER-10gMigrate.zip file. Unzip this file to a directory on your computer. The migration tool can be run in the command line using the migrate.bat utility (for Windows) or migrate.sh (for Linux and Unix).

Before running migrate.bat or migrate.sh, ensure that the environment variables mentioned in Table 3-1 are set. In Windows, from a command window, you can type set X to see the value of the variable X, and set X=abc to set the value of FOO to abc.

Table 3-1 Command Line Script

Environment Variable Description

JAVA_HOME

Ensure that the JAVA_HOME environment variable points to an installed java runtime (JRE) or SDK. This must be Java version 5 or higher.

JAVA_OPTS

Ensure that you set your JAVA_OPTS parameter as follows:

set JAVA_OPTS=-Dhttp.proxyPort=80 -Dhttp.proxyHost=www-proxy.us.oracle.com -Dhttp.nonProxyHosts= "*.oracle.com|localhost"

JAVA_OPTS refers to the extra options to the java executable. In normal cases, there is no need to set this variable. However, a common exception when you need to set this variable is when your computer is inside a firewall, and you need to use an HTTP proxy to access external servers.

See Also: http://java.sun.com/javase/6/docs/technotes/guides/net/proxies.html


You can preview the results of a migration in the Data Migration tool, without committing any changes to Oracle Enterprise Repository, by using the command-line options. Table 3-2 describes the command-line options for the Data Migration tool.

Table 3-2 Command Line Options for the Data Migration Tool

Environment Variable Description

-preview

If true, the data migration tool runs in the preview mode. A detailed information about successes and failures are logged. No changes are committed to Oracle Enterprise Repository.If false, the data migration tool runs in the production mode. A detailed information about successes and failures are logged, and the changes are committed to Oracle Enterprise Repository.


3.3 Configuring the Data Migration Tool

This section contains the following topics:

3.3.1 Setting the Repository Connection Information for the Command-line Utility

Open the HarvesterSettings.xml file located at <Data Migration Home> and modify the following XML chunk to point to the Oracle Enterprise Repository instance with the right credentials.

<repository>
  <uri>http://localhost:8080/oer/services/FlashlineRegistry</uri>
  <credentials>
    <user>admin</user>
    <password>*****</password>
    <enableTransaction>false</enableTransaction>
    <triggerEvent>false</triggerEvent>
  </credentials>
  <timeout>120000</timeout>
</repository>

Note:

You should run the Migration Tool as a user with the Basic Access Settings for Assets - View, Edit, Accept, and Register.

The passwords in this file must be encrypted before running the migrate script. To encrypt the passwords, use the encrypt.bat/encrypt.sh script, which is located in <Oracle_home>/tools/solutions/11.1.1.x.x-OER-PasswordTools.zip.

3.3.2 Advanced Configuration

The section describes the following advanced configuration options for the migration tool:

3.3.2.1 Asset Types to Migrate

By default, the migration tool is configured to migrate the assets of the types that are created by default by the ALER 3.x and 10g R3 tools. However, these legacy importers can be configured to use different asset types, through the types.properties file.

If the assets were imported using nondefault asset types, you must configure the migration tool to use the same nondefault asset types. These can be configured by editing the following XML chunk in the HarvesterSettings.xml file:

<query>
    <repositoryQuery>
      <typesToMigrate>
        <assetType.service>Service</assetType.service>
        <assetType.endpoint>Endpoint: Web Service</assetType.endpoint>
        <assetType.endpoint>Endpoint</assetType.endpoint>
        <assetType.interface>Interface: Web Service</assetType.interface>
        <assetType.interface>Interface</assetType.interface>
        <assetType.process>Business Process: BPEL</assetType.process>
        <assetType.process>Business Process</assetType.process>
        <assetType.processDeployment>Deployment:
        BPEL</assetType.processDeployment>
        <assetType.xsd>Artifact: XSD</assetType.xsd>
        <assetType.wsdl>Artifact: WSDL</assetType.wsdl>
        <assetType.bpel>Artifact: BPEL</assetType.bpel>
        <assetType.xslt>Artifact: XSLT</assetType.xslt>
      </typesToMigrate>
    </repositoryQuery>

Note:

You should migrate all Assets during a single run of the migration tool. Individually migrating specific Asset types can cause problems such as having the same internal names.

A new attribute has been added that determines what the "source" asset's active status will be after the migration has completed.

For example, when migrating "CustomerServiceType" to "Service", you can say you want all assets of type "CustomerServiceType" to be set to "Retired".

<assetType.service sourceActiveStatus="retired" name="CustomerServiceType" />

3.3.2.2 Custom Data Migration

Configuration of custom data is done using the provided customDataMigration.xml file. Configuration must occur for each asset type that will be migrated.

Determine what the "source" asset type is, and what the "destination" (11gR1) asset type is, for example, "MyCustomServiceType" is the source, and it will be migrated to "Service", the destination asset type.

In the customDataMigration.xml, this is defined as:

<migratingAssetTypes>
     <migrationAssetType>
          <sourceTypeName>MyCustomServiceType</sourceTypeName>
          <destinationTypeName>Service</destinationTypeName>

Once the asset types are established, determine which custom data elements to migrate. To do this, define the XML Mappings for each element for both the source and the destination.

The XML Mapping names can be obtained doing the following:

  1. Launch the Asset Editor.

  2. Launch the Type Manager.

  3. Navigate to File->Export Schema and select the desired Asset Type.

The schema will display all of the XML Mappings for the asset type, specifically within the element, <xs:element name="custom-data">. For example: <xs:element name="development-hours" >, where the name attribute is the XMLMapping.

Configuration requires that you determine the XML Mapping for the custom data element in the source and ALSO the XML Mapping that you want to migrate the source element to.

The example below migrates custom data from the MyCustomServiceType asset type to the Service asset type. For every asset of that type, the value stored in the "development-hours" field will be migrated to the corresponding "development-hours" field in the Service asset type, as well as "estimated-time-to-use". In the customDataMigration.xml, this is defined as:

<migratingAssetTypes>
  <migrationAssetType>
   <sourceTypeName>MyCustomServiceType</sourceTypeName>
   <destinationTypeName>Service</destinationTypeName>
    <customDataFields>
     <customDataField>
      <sourceElement>
       <xmlMapping name="development-hours" />
      </sourceElement>
      <destinationElement>
       <xmlMapping name="development-hours" />        
      </destinationElement>
     </customDataField>
     <customDataField>
      <sourceElement>
       <xmlMapping name="estimated-time-to-use" />
      </sourceElement>
      <destinationElement>
       <xmlMapping name="estimated-time-to-use" />    
      </destinationElement>
     </customDataField>                              
    </customDataFields>

In this sample, the source and destination XML Mappings are the same, however that is not always the case. The configuration allows the ability to migrate data into any defined XML Mapping field.

Note:

If the defined XML Mapping field in the destination asset does not exist, it must be added in the Type Manager before running the migration tool. Failure to define it results in a failure while running the migration tool.

Migrating Custom Data Fields that are Tables

There are certain custom data fields that are tables, which means they have custom data fields within custom data fields. These types can be migrated, but the setup is more complicated. A table, called "SourceTable1", has two fields within it, "SourceText1" and "SourceDate1". When defining these XML Mapping fields in the configuration file, the highest level XML Mapping must be defined first, in this case, "SourceTable1". The children of that field do not need to be in any order, however, they must correspond correctly when defining the destination XML Mappings.

<migratingAssetTypes>
  <migrationAssetType>
   <sourceTypeName>MyCustomServiceType</sourceTypeName>
   <destinationTypeName>Service</destinationTypeName>
    <customDataFields>
     <customDataField>
      <sourceElement>
       <xmlMapping name="SourceTable1" />
       <xmlMapping name="SourceText1" />
       <xmlMapping name="SourceDate1" />
      </sourceElement>
      <destinationElement>
       <xmlMapping name="DestinationTable1" />
       <xmlMapping name="DestinationText1" />
       <xmlMapping name="DestinationDate1" />
      </destinationElement>
     </customDataField>
    </customDataFields>

This example provides an alternative to the example above:

<sourceElement>
      <xmlMapping name="SourceTable1" />
      <xmlMapping name="SourceDate1" />
      <xmlMapping name="SourceText1" />
     </sourceElement>
     <destinationElement>
      <xmlMapping name="DestinationTable1" />
      <xmlMapping name="DestinationDate1" />
      <xmlMapping name="DestinationText1" />
     </destinationElement>
 

This example illustrates an error because the SourceText1 field will be migrated to DestinationDate1.

<sourceElement>
      <xmlMapping name="SourceTable1" />
      <xmlMapping name="SourceText1" />
      <xmlMapping name="SourceDate1" />
     </sourceElement>
     <destinationElement>
      <xmlMapping name="DestinationTable1" />
      <xmlMapping name="DestinationDate1" />
      <xmlMapping name="DestinationText1" />
     </destinationElement>

Migrating Custom Data Fields with Acceptable Value Lists

Custom Data fields with acceptable value lists (AVLs) can also be migrated, however there is one extra step involved. The custom data field has its XML Mapping, but it also references the XML Mapping of the actual AVL.

This can also be found within the exported asset type schema.

<xs:element name="avl-dropdown">
     <xs:complexType>
       <xs:simpleContent>
         <xs:extension base="AcceptableValueList_50018">
           <xs:attribute name="id" type="xs:long" use="optional"/>
         </xs:extension>
       </xs:simpleContent>
     </xs:complexType>
   </xs:element>

This is translated in the custom data configuration like this:

<sourceElement>
      <xmlMapping name="avl-dropdown"
       multipleValueListItemMapping="AcceptableValueList_50018"/>
     </sourceElement>
     <destinationElement>
      <xmlMapping name="destination-avl-dropdown"
       multipleValueListItemMapping="Destination-AcceptableValueList_50020"/>
     </destinationElement>

3.3.2.3 Logging

The migration tool uses log4j for logging the detailed tasks performed and the log file is placed in the < Migration Tool Home>. The logging options can be changed by updating the log4fl.properties file located in the <Migration Tool Home>.

3.3.3 Known Issues

This section describes the following known issues in the Data Migration tool:

  • The Data Migration tool does not support assets that are created using the ALSync framework, from ALSB or ALDSP. ALSB assets must be re-introspected using Harvester. Harvesting from ALDSP is currently not supported.

  • The Data Migration tool, when migrating exceptionally large data sets, may take hours to run. Some application servers may time out the sessions and cause the migration to fail. To prevent this timeout, you must add the following entry to the web.xml file, within the <webapps> element, where the number you supply is in minutes:

    <session-config>
    <session-timeout>120</session-timeout>
    </session-config>
    
  • The Data Migration tool ignores the assets with local referenced artifacts, such as Fileinfo local with rep://.

  • Migration of following metadata element Types are NOT supported.

    • Name

    • Version

    • Service Type

    • Uploaded Submission Files

    • Consuming Projects

    • Asset Usage

    • Review

    • Metadata Files

    • Assembly Properties Viewer

  • Migration of SFID is NOT supported.

  • Migration of UNIQUE element is not supported between the same type by logging warning message.