This chapter contains the following sections:
The Data Migration Tool migrates the source asset and artifact file information to new OER 11g Asset Models (Harvested Models), when the file information artifacts are remote and links them with a special relationship (harvester-migrator), for tracking. Here is a sample view of a pre and post migration asset model.
The Data Migration Tool assists you in these tasks:
Migrating ad-hoc created assets to OER11g Asset model, given the source asset has valid file info with WSDL/XSD artifacts (all harvester supported artifacts <provide link>).
Migrating all the ALER 3.x & OER 10.x Harvested Assets to confirm to OER11g Harvester model.
Migrating most of the key mandatory-data from source asset to migrated root service asset (ref the <known issue> section for unsupported data migration elements)
Migrating any Custom Type Asset Create by Customer to 11g Harvester model to take the all benefits provided in OER 11g SOA governance feature.
Migrating custom-data defined on out-of-the-box or custom asset types to destination assets (*)
Setting the source asset to Retire /Delete at the end of post migration.
In the Oracle Enterprise Repository 11g R1 release, the harvester was extended to support Oracle SOA Suite 11g and Oracle Service Bus 10g. The Exchange Utility was enhanced to support Oracle Service Bus. The Enterprise Manager Integration utility was added to integrate with metrics in Enterprise Manager 10g. To support these products, the Oracle Enterprise Repository asset model was extended, which is known as the 11g R1 Asset Model.
This section describes the key points about Oracle Enterprise Repository content migration in an upgrade:
The custom types are left intact when upgrading to 11g. Also, the extensions, for example, the addition of metadata attributes, but not necessarily deletions, made in the type manager to standard out-of the box asset types also remain intact. The instances of these types with their extended metadata are ordinarily preserved in an upgrade.
Some or all instances of service assets/artifacts that you previously harvested or captured at design time must be harvested or reharvested with deployment/endpoint information. This is true, for example, if you and your customer require Oracle Enterprise Repository/Oracle Service Registry use cases such as consumption in Oracle SOA Suite/JDeveloper, registry/repository exchange between Oracle Enterprise Repository and Oracle Service Registry, and other more advanced lifecycle use cases that are beyond simple visibility in Oracle Enterprise Repository).
To behave properly in Oracle Enterprise Repository 11g, artifact files must either be hosted remotely on an HTTP/HTTPS/FTP site or harvested directly from deployment instances. In addition, endpoints must be present from concrete WSDL in order for Oracle Enterprise Repository to publish to Oracle Service Registry.
Typically, this re-harvesting is recommended for any services/artifacts that have locally cached artifact files in file info.
If you have manually entered service assets/artifacts (non-harvested) of the same types that are typically used by the harvester, then you can manually plug-in a remote reference to the concrete WSDL hosted on an HTTP/HTTPS/FTP site before an upgrade to Oracle Enterprise Repository 11g and the harvester Migration Tool (
11.1.1.x.x-OER-Migrate.zip) creates the new Harvester model.
Before using the migration tool, you must perform the following prerequisites:
The Oracle Enterprise Repository server instance must first be upgraded to 11g R1, and must be started.
The Oracle Enterprise Repository database must be backed up, so it can be restored if there are any problems with the migration.
The migration tool requires the Java JRE or JDK version 6 or higher.
The migration tool connects to Oracle Enterprise Repository using its WebService API. So it can be run on any computer that can access the Oracle Enterprise Repository server. It runs much faster if run on the same computer as the Oracle Enterprise Repository server.
The Oracle Enterprise Repository System Setting,
cmee.extframework.enabled, must be set to
true. This can be set from the Web UI in the Admin, System Settings page.
Only assets with a status of Active will be migrated.
You can use the migration tool to:
Migrate assets that were imported into Oracle Enterprise Repository through the ALER 3.x WSDL importer in the Web console.
Migrate assets that were imported into Oracle Enterprise Repository through the ALER 3.x UDDI importer in the Web console.
Migrate assets that were imported into Oracle Enterprise Repository through the ALER 3.x BPEL importer in the Web console.
Migrate assets that were imported into Oracle Enterprise Repository through the ALER 3.x ALRR-XU (version 3.x).
Migrate assets that were imported into Oracle Enterprise Repository through the ALER 3.x ad-hoc "Submit an Asset" functionality in the Web console.
Migrate assets that were imported into Oracle Enterprise Repository through the 10g R3 Harvester.
Migrate assets that were imported into Oracle Enterprise Repository through the 10g R3 Exchange Utility.
Migrate WSDL assets that were imported into Oracle Enterprise Repository through the Open API (REX) WSDL import functionality.
Preview the results of a migration, without committing any changes to Oracle Enterprise Repository.
This section describes the migration tool functionality for ALER 3.x assets and Oracle Enterprise Repository 11g assets.
In the past, Oracle Enterprise Repository had a variety of tools for importing metadata. In version 3.x of ALER, these included Web and Open API (REX) utilities for importing from WSDL, BPEL, and UDDI and the command-line RRXU tool. These import service metadata into an asset model called the ALER 3.x Asset Model, as shown.
The contents of the WSDL that is pointed to by the Service FileInfo is introspected from its download URI. An interface asset is created, and related to the service. An Artifact: WSDL asset is introspected from its download URI. An Artifact: WSDL asset is created and related to the endpoint. A relationship, Harvester-Migrator, is created between the original Service asset and the new Service asset that is created by the introspection.
The contents of the BPEL that is pointed to by a business process asset is introspected from its download URI. An Artifact: BPEL asset is created and related to the business process. The business process asset has its asset type changed to Business Process: BPEL. A relationship, Harvester-Migrator, is created between the original BPEL asset and the new Business Process: BPEL asset that is created by the introspection.
Any WSDLs or XSDs imported by the WSDL results in new artifact assets and relationships, as in the Harvester (or points to existing assets if their fingerprints match).
Any WSDLs used by a BPEL in partnerlinks results in new artifact assets and relationships, as in the Harvester.
The FileInfo is removed from the service and endpoint assets. The new Artifact: WSDL asset contains the updated FileInfo.
The SFID (fingerprint) is removed from the service and endpoint assets. Any Artifact: WSDL assets and Artifact: XSD assets contain a new SFID, using the Harvester's SFID algorithm.
Any non-artifact asset that is migrated gets a new internal name, which is used for duplicate checking.
Pre-existing service, endpoint, and business process assets keep their original name. New assets are named according to the Harvester's naming rules.
WSDL Summary metadata entries are created on the migrated assets, as in Harvester.
Manifest metadata entries are created on the migrated assets, as in Harvester, to support the new download functionality in Oracle Enterprise Repository.
Assets that were imported into Oracle Enterprise Repository in an ad-hoc manner are migrated as described in Section 22.214.171.124, "Asset Types to Migrate", only if they conform to the ALER 3.x Asset model. For example, if an ad-hoc service asset contains a FileInfo that points to a WSDL, it is migrated as described above. But an asset of another type, for example, "TestCase", are not migrated.
Recalculates and store the SFID for artifact assets from the original artifact files, using the 11gR1 fingerprinting code.
internal.introspector.manifest.store CMF entry and FileInfo with the correct artifact location according to the following 11g R1 harvester rules:
Artifacts that were harvested from remote URLs have the FileInfo updated to point to the remote location (to match the Manifest).
Artifacts that were harvested from local files have the Manifest updated to point to the repository download URL (to match the FileInfo).
internal.artifact.store CMF entries for artifacts that were harvested from remote URLs.
internal.alrr.exchange.store CMF entry to conform to the latest Exchange Utility rules.
<tmodel> entries in the
internal.alrr.exchange.store CMF entry on Service and Endpoint assets, with the UDDI keys of porttype and binding TModels. (Connects to the UDDI registry to get these).
<uddiRegistries> custom data table on Endpoint assets, including the
service-key elements. The
registry-url elements are set from the original
internal.alrr.exchange.store CMF entry. The service-key is read from the UDDI registry during migration.
Sets the <uddi> custom data elements on Service assets, including the business-key and service-key elements. These are read from the UDDI registry during migration.
Sets the <uddi> custom data elements on Business Entity assets, specifically the business-key element. This is read from the UDDI registry during migration.
internal.introspector.store CMF entry to the new format used by 11gR1 harvester.
intname harvester property for abstract assets, according to the 11gR1 harvester rules.
Calculates and sets the following harvester properties, which are automatically set by 11gR1 harvester, on the appropriate assets:
Deployment URL, Transport Type, Interface Type, Service Type.
wsdlLocation element in the WSDLSummary, based on the download URL calculated in "Manifests".
Updates the asset types according to the 11g R1 model:
Changes Endpoint: WebService assets to Endpoint
Changes Interface: WebService assets to Interface
This section contains the following topics:
The migration tool is available in the
11.1.1.x.x-OER-10gMigrate.zip file. Unzip this file to a directory on your computer. The migration tool can be run in the command line using the migrate.bat utility (for Windows) or migrate.sh (for Linux and Unix).
Before running migrate.bat or migrate.sh, ensure that the environment variables mentioned in Table 3-1 are set. In Windows, from a command window, you can type
set X to see the value of the variable X, and
set X=abc to set the value of
Ensure that the
Ensure that you set your
See Also: http://java.sun.com/javase/6/docs/technotes/guides/net/proxies.html
You can preview the results of a migration in the Data Migration tool, without committing any changes to Oracle Enterprise Repository, by using the command-line options. Table 3-2 describes the command-line options for the Data Migration tool.
If true, the data migration tool runs in the preview mode. A detailed information about successes and failures are logged. No changes are committed to Oracle Enterprise Repository.If false, the data migration tool runs in the production mode. A detailed information about successes and failures are logged, and the changes are committed to Oracle Enterprise Repository.
This section contains the following topics:
HarvesterSettings.xml file located at
<Data Migration Home> and modify the following XML chunk to point to the Oracle Enterprise Repository instance with the right credentials.
<repository> <uri>http://localhost:8080/oer/services/FlashlineRegistry</uri> <credentials> <user>admin</user> <password>*****</password> <enableTransaction>false</enableTransaction> <triggerEvent>false</triggerEvent> </credentials> <timeout>120000</timeout> </repository>
You should run the Migration Tool as a user with the Basic Access Settings for Assets - View, Edit, Accept, and Register.
The passwords in this file must be encrypted before running the migrate script. To encrypt the passwords, use the
encrypt.sh script, which is located in <Oracle_home>/tools/solutions/11.1.1.x.x-OER-PasswordTools.zip.
The section describes the following advanced configuration options for the migration tool:
By default, the migration tool is configured to migrate the assets of the types that are created by default by the ALER 3.x and 10g R3 tools. However, these legacy importers can be configured to use different asset types, through the
If the assets were imported using nondefault asset types, you must configure the migration tool to use the same nondefault asset types. These can be configured by editing the following XML chunk in the
<query> <repositoryQuery> <typesToMigrate> <assetType.service>Service</assetType.service> <assetType.endpoint>Endpoint: Web Service</assetType.endpoint> <assetType.endpoint>Endpoint</assetType.endpoint> <assetType.interface>Interface: Web Service</assetType.interface> <assetType.interface>Interface</assetType.interface> <assetType.process>Business Process: BPEL</assetType.process> <assetType.process>Business Process</assetType.process> <assetType.processDeployment>Deployment: BPEL</assetType.processDeployment> <assetType.xsd>Artifact: XSD</assetType.xsd> <assetType.wsdl>Artifact: WSDL</assetType.wsdl> <assetType.bpel>Artifact: BPEL</assetType.bpel> <assetType.xslt>Artifact: XSLT</assetType.xslt> </typesToMigrate> </repositoryQuery>
You should migrate all Assets during a single run of the migration tool. Individually migrating specific Asset types can cause problems such as having the same internal names.
A new attribute has been added that determines what the "source" asset's active status will be after the migration has completed.
For example, when migrating "CustomerServiceType" to "Service", you can say you want all assets of type "CustomerServiceType" to be set to "Retired".
<assetType.service sourceActiveStatus="retired" name="CustomerServiceType" />
Configuration of custom data is done using the provided customDataMigration.xml file. Each migrated asset type needs to be configured.
Determine what the source asset type is, and what the destination (11gR1) asset type is, for example, MyCustomServiceType is the source, and it will be migrated to Service, the destination asset type.
In the customDataMigration.xml, this is defined as:
<migratingAssetTypes> <migrationAssetType> <sourceTypeName>MyCustomServiceType</sourceTypeName> <destinationTypeName>Service</destinationTypeName>
Once the asset types are established, determine which custom data elements to migrate. To do this, define the XML Mappings for each element for both the source and the destination.
Click Generate Migration XML from the OER Web Tool - Admin tab - Types section.
Select the Source type, the available custom data elements and the Destination type with its available custom data elements.
Click Generate XML to display the XML.
Copy and paste this XML into the customDataMigration.xml file within the element <migratingAssetTypes>.
Do this for each Asset Type you want to migrate.
The XML Mapping names can be obtained doing the following:
Launch the Asset Editor.
Launch the Type Manager.
Navigate to File->Export Schema and select the desired Asset Type.
The schema will display all of the XML Mappings for the asset type, specifically within the element,
<xs:element name="custom-data">. For example:
<xs:element name="development-hours" >, where the name attribute is the XMLMapping.
Configuration requires that you determine the XML Mapping for the custom data element in the source and ALSO the XML Mapping that you want to migrate the source element to.
The example below migrates custom data from the MyCustomServiceType asset type to the Service asset type. For every asset of that type, the value stored in the "development-hours" field will be migrated to the corresponding "development-hours" field in the Service asset type, as well as "estimated-time-to-use". In the customDataMigration.xml, this is defined as:
<migratingAssetTypes> <migrationAssetType> <sourceTypeName>MyCustomServiceType</sourceTypeName> <destinationTypeName>Service</destinationTypeName> <customDataFields> <customDataField> <sourceElement> <xmlMapping name="development-hours" /> </sourceElement> <destinationElement> <xmlMapping name="development-hours" /> </destinationElement> </customDataField> <customDataField> <sourceElement> <xmlMapping name="estimated-time-to-use" /> </sourceElement> <destinationElement> <xmlMapping name="estimated-time-to-use" /> </destinationElement> </customDataField> </customDataFields>
In this sample, the source and destination XML Mappings are the same, however that is not always the case. The configuration allows the ability to migrate data into any defined XML Mapping field.
If the defined XML Mapping field in the destination asset does not exist, it must be added in the Type Manager before running the migration tool. Failure to define it results in a failure while running the migration tool.
There are certain custom data fields that are tables, which means they have custom data fields within custom data fields. These types can be migrated, but the setup is more complicated. A table, called "SourceTable1", has two fields within it, "SourceText1" and "SourceDate1". When defining these XML Mapping fields in the configuration file, the highest level XML Mapping must be defined first, in this case, "SourceTable1". The children of that field do not need to be in any order, however, they must correspond correctly when defining the destination XML Mappings.
<migratingAssetTypes> <migrationAssetType> <sourceTypeName>MyCustomServiceType</sourceTypeName> <destinationTypeName>Service</destinationTypeName> <customDataFields> <customDataField> <sourceElement> <xmlMapping name="SourceTable1" /> <xmlMapping name="SourceText1" /> <xmlMapping name="SourceDate1" /> </sourceElement> <destinationElement> <xmlMapping name="DestinationTable1" /> <xmlMapping name="DestinationText1" /> <xmlMapping name="DestinationDate1" /> </destinationElement> </customDataField> </customDataFields>
This example provides an alternative to the example above:
<sourceElement> <xmlMapping name="SourceTable1" /> <xmlMapping name="SourceDate1" /> <xmlMapping name="SourceText1" /> </sourceElement> <destinationElement> <xmlMapping name="DestinationTable1" /> <xmlMapping name="DestinationDate1" /> <xmlMapping name="DestinationText1" /> </destinationElement>
This example illustrates an error because the SourceText1 field will be migrated to DestinationDate1.
<sourceElement> <xmlMapping name="SourceTable1" /> <xmlMapping name="SourceText1" /> <xmlMapping name="SourceDate1" /> </sourceElement> <destinationElement> <xmlMapping name="DestinationTable1" /> <xmlMapping name="DestinationDate1" /> <xmlMapping name="DestinationText1" /> </destinationElement>
Custom Data fields with acceptable value lists (AVLs) can also be migrated, however there is one extra step involved. The custom data field has its XML Mapping, but it also references the XML Mapping of the actual AVL.
This can also be found within the exported asset type schema.
<xs:element name="avl-dropdown"> <xs:complexType> <xs:simpleContent> <xs:extension base="AcceptableValueList_50018"> <xs:attribute name="id" type="xs:long" use="optional"/> </xs:extension> </xs:simpleContent> </xs:complexType> </xs:element>
This is translated in the custom data configuration like this:
<sourceElement> <xmlMapping name="avl-dropdown" multipleValueListItemMapping="AcceptableValueList_50018"/> </sourceElement> <destinationElement> <xmlMapping name="destination-avl-dropdown" multipleValueListItemMapping="Destination-AcceptableValueList_50020"/> </destinationElement>
This section describes the following known issues in the Data Migration tool:
The Data Migration tool does not support assets that are created using the ALSync framework, from ALSB or ALDSP. ALSB assets must be re-introspected using Harvester. Harvesting from ALDSP is currently not supported.
The Data Migration tool, when migrating exceptionally large data sets, may take hours to run. Some application servers may time out the sessions and cause the migration to fail. To prevent this timeout, you must add the following entry to the
web.xml file, within the
<webapps> element, where the number you supply is in minutes:
<session-config> <session-timeout>120</session-timeout> </session-config>
The Data Migration tool ignores the assets with local referenced artifacts, such as
Fileinfo local with rep://.
Migration of following metadata element Types are NOT supported.
Uploaded Submission Files
Assembly Properties Viewer
Migration of SFID is NOT supported.
Migration of UNIQUE element is not supported between the same type by logging warning message.