Overview

Oracle NoSQL Data Migrator lets you move Oracle NoSQL tables from one data source to another, such as Oracle NoSQL Database on-premise or cloud or even a simple JSON file.

There can be many situations that require you to migrate NoSQL tables from or to an Oracle NoSQL Database. For instance, a team of developers enhancing a NoSQL Database application may want to test their updated code in the local Oracle NoSQL Database Cloud Service (NDCS) instance using cloudsim. To verify all the possible test cases, they must set up the test data similar to the actual data. To do this, they must copy the NoSQL tables from the production environment to their local NDCS instance, the cloudsim environment. In another situation, NoSQL developers may need to move their application data from on-premise to the cloud and vice-versa, either for development or testing.

In all such cases and many more, you can use Oracle NoSQL Data Migrator to move your NoSQL tables from one data source to another, such as Oracle NoSQL Database on-premise or cloud or even a simple JSON file. You can also copy NoSQL tables from a MongoDB-formatted JSON input file into your NoSQL Database on-premise or cloud.

Oracle NoSQL Data Migrator is created to replace and enhance the existing on-premise-only import/export utility. To know how the NoSQL Data Migrator is different from the existing import/export utility, see Oracle NoSQL Data Migrator Vs. Import/Export Utility.

As depicted in the following figure, the NoSQL Data Migrator utility acts as a connector or pipe between the data source and the target (referred to as the sink). In essence, this utility exports data from the selected source and imports that data into the sink. This tool is table-oriented, that is, you can move the data only at the table level. A single migration task operates on a single table and supports the following options:

JSON file to Oracle NoSQL Database on-premise and vice versa
JSON file to Oracle NoSQL Database Cloud Service and vice versa
Oracle NoSQL Database on-premise to Oracle NoSQL Database Cloud Service and vice versa
MongoDB-formatted JSON file to an Oracle NoSQL Database table
MongoDB-formatted JSON file to an Oracle NoSQL Database Cloud Service table
One Oracle NoSQL Database on-premise to another
One Oracle NoSQL Database Cloud Service to another

Oracle NoSQL Data Migrator is designed such that it can support additional sources and sinks in the future. For a list of sources and sinks supported by Oracle NoSQL Data Migrator as of the current release, see Supported Sources and Sinks.

Description of migrator_overview.eps follows

Description of the illustration migrator_overview.eps

Terminology used with NoSQL Data Migrator

Learn about the different terms used in the above diagram, in detail.

Source: An entity from where the NoSQL tables are exported for migration. Some examples of sources are Oracle NoSQL Database on-premise or cloud, JSON file, and MongoDB-formatted JSON file.
Sink: An entity that imports the NoSQL tables from NoSQL Data Migrator. Some examples for sinks are Oracle NoSQL Database on-premise or cloud and JSON file.
Migration Pipe: The data from a source will be transferred to the sink by NoSQL Data Migrator. This can be visualized as a Migration Pipe.
Transformations: You can add rules to modify the NoSQL table data in the migration pipe. These rules are called Transformations. Oracle NoSQL Data Migrator allows data transformations at the top-level fields or columns only. It does not let you transform the data in the nested fields. Some examples of permitted transformations are:
- Drop or ignore one or more columns,
- Rename one or more columns, or
- Aggregate several columns into a single field, typically a JSON field.

Configuration File : A configuration file is a JSON file where you define all the parameters required for the migration activity. Later, you pass this JSON file as a single parameter to the runMigrator command from the CLI. A typical configuration file format looks like as shown below.

{
 "source": {
   "type" : <source type>,
   //source-configuration for type. See Source Configuration Templates.
 },
 "sink": {
   "type" : <sink type>,
   //sink-configuration for type. See Sink Configuration Templates.
 },
 "transforms" : {
   //transforms configuration. See Transformation Configuration Templates.
 },
 "migratorVersion" : "1.0.0",
 "abortOnError" : <true|false>
}

Group	Parameters	Mandatory (Y/N)	Purpose	Supported Values
`source`	`type`	Y	Represents the source from which to migrate the data. The source provides data and metadata (if any) for migration.	To know the `type` value for each source, see Supported Sources and Sinks.
	source-configuration for type	Y	Defines the configuration for the source. These configuration parameters are specific to the type of source selected above.	See Source Configuration Templates for the complete list of configuration parameters for each source type.
`sink`	`type`	Y	Represents the sink to which to migrate the data. The sink is the target or destination for the migration.	To know the `type` value for each source, see Supported Sources and Sinks.
	sink-configuration for type	Y	Defines the configuration for the sink. These configuration parameters are specific to the type of sink selected above.	See Sink Configuration Templates for the complete list of configuration parameters for each sink type.
`transforms`	transforms configuration	N	Defines the transformations to be applied to the data in the migration pipe.	See Transformation Configuration Templates for the complete list of transformations supported by the NoSQL Data Migrator.
-	`migratorVersion`	N	Version of the NoSQL Data Migrator	-
-	`abortOnError`	N	Specifies whether to stop the migration activity in case of any error or not. The default value is true indicating that the migration stops whenever it encounters a migration error. If you set this value to false, the migration continues even in case of failed records or other migration errors. The failed records and migration errors will be logged as WARNINGs on the CLI terminal.	true, false

Note:

As JSON is case-sensitive, all the parameters defined in the configuration file are case-sensitive unless specified otherwise.