Learn about Oracle NoSQL Database Migrator and how to use it for data migration.

Oracle NoSQL Database Migrator is a tool that enables you to migrate Oracle NoSQL tables from one data source to another. This tool can operate on tables in Oracle NoSQL Database Cloud Service, Oracle NoSQL Database on-premises and AWS S3. The Migrator tool supports several different data formats and physical media types. Supported data formats are JSON, Parquet, MongoDB-formatted JSON, DynamoDB-formatted JSON, and CSV files. Supported physical media types are files, OCI Object Storage, Oracle NoSQL Database on-premises, Oracle NoSQL Database Cloud Service and AWS S3.

This article has the following topics:

Oracle NoSQL Database Migrator lets you move Oracle NoSQL tables from one data source to another, such as Oracle NoSQL Database on-premises or cloud or even a simple JSON file.

There can be many situations that require you to migrate NoSQL tables from or to an Oracle NoSQL Database. For instance, a team of developers enhancing a NoSQL Database application may want to test their updated code in the local Oracle NoSQL Database Cloud Service (NDCS) instance using cloudsim. To verify all the possible test cases, they must set up the test data similar to the actual data. To do this, they must copy the NoSQL tables from the production environment to their local NDCS instance, the cloudsim environment. In another situation, NoSQL developers may need to move their application data from on-premise to the cloud and vice-versa, either for development or testing.

In all such cases and many more, you can use Oracle NoSQL Database Migrator to move your NoSQL tables from one data source to another, such as Oracle NoSQL Database on-premise or cloud or even a simple JSON file. You can also copy NoSQL tables from a MongoDB-formatted JSON input file, DynamoDB-formatted JSON input file (either stored in AWS S3 source or from files), or a CSV file into your NoSQL Database on-premises or cloud.

As depicted in the following figure, the NoSQL Database Migrator utility acts as a connector or pipe between the data source and the target (referred to as the sink). In essence, this utility exports data from the selected source and imports that data into the sink. This tool is table-oriented, that is, you can move the data only at the table level. A single migration task operates on a single table and supports migration of table data from source to sink in various data formats.

Oracle NoSQL Database Migrator is designed such that it can support additional sources and sinks in the future. For a list of sources and sinks supported by Oracle NoSQL Database Migrator as of the current release, see Supported Sources and Sinks.

Description of the illustration migrator_overview.eps

Learn about the different terms used in the above diagram, in detail.

• Source: An entity from where the NoSQL tables are exported for migration. Some examples of sources are Oracle NoSQL Database on-premise or cloud, JSON file, MongoDB-formatted JSON file, DynamoDB-formatted JSON file, and CSV files.
• Sink: An entity that imports the NoSQL tables from NoSQL Database Migrator. Some examples for sinks are Oracle NoSQL Database on-premise or cloud and JSON file.

The NoSQL Database Migrator tool supports different types of sources and sinks (that is physical media or repositories of data) and data formats (that is how the data is represented in the source or sink). Supported data formats are JSON, Parquet, MongoDB-formatted JSON, DynamoDB-formatted JSON, and CSV files. Supported source and sink types are files, OCI Object Storage, Oracle NoSQL Database on-premise, and Oracle NoSQL Database Cloud Service.

• Migration Pipe: The data from a source will be transferred to the sink by NoSQL Database Migrator. This can be visualized as a Migration Pipe.
• Transformations: You can add rules to modify the NoSQL table data in the migration pipe. These rules are called Transformations. Oracle NoSQL Database Migrator allows data transformations at the top-level fields or columns only. It does not let you transform the data in the nested fields. Some examples of permitted transformations are:
  • Drop or ignore one or more columns,
  • Rename one or more columns, or
  • Aggregate several columns into a single field, typically a JSON field.
• Configuration File : A configuration file is where you define all the parameters required for the migration activity in a JSON format. Later, you pass this configuration file as a single parameter to the runMigrator command from the CLI. A typical configuration file format looks like as shown below.

  {
   "source": {
     "type" : <source type>,
     //source-configuration for type. See Source Configuration Templates  .
   },
   "sink": {
     "type" : <sink type>,
     //sink-configuration for type. See Sink Configuration Templates  .
   },
   "transforms" : {
     //transforms configuration. See Transformation Configuration Templates  .
   },
   "migratorVersion" : "<migrator version>",
   "abortOnError" : <true|false>
  }

  Group	Parameters	Mandatory (Y/N)	Purpose	Supported Values
  source	type	Y	Represents the source from which to migrate the data. The source provides data and metadata (if any) for migration.	To know the type value for each source, see Supported Sources and Sinks.
  source	source-configuration for type	Y	Defines the configuration for the source. These configuration parameters are specific to the type of source selected above.	See Source Configuration Templates . for the complete list of configuration parameters for each source type.
  sink	type	Y	Represents the sink to which to migrate the data. The sink is the target or destination for the migration.	To know the type value for each source, see Supported Sources and Sinks.
  sink	sink-configuration for type	Y	Defines the configuration for the sink. These configuration parameters are specific to the type of sink selected above.	See Sink Configuration Templates for the complete list of configuration parameters for each sink type.
  transforms	transforms configuration	N	Defines the transformations to be applied to the data in the migration pipe.	See Transformation Configuration Templates for the complete list of transformations supported by the NoSQL Data Migrator.
  -	migratorVersion	N	Version of the NoSQL Data Migrator	-
  -	abortOnError	N	Specifies whether to stop the migration activity in case of any error or not. The default value is true indicating that the migration stops whenever it encounters a migration error. If you set this value to false, the migration continues even in case of failed records or other migration errors. The failed records and migration errors will be logged as WARNINGs on the CLI terminal.	true, false

  Note:

  As JSON file is case-sensitive, all the parameters defined in the configuration file are case-sensitive unless specified otherwise.

This topic provides the list of the sources and sinks supported by the Oracle NoSQL Database Migrator.

You can use any combination of a valid source and sink from this table for the migration activity. However, you must ensure that at least one of the ends, that is, source or sink must be an Oracle NoSQL product. You can not use the NoSQL Database Migrator to move the NoSQL table data from one file to another.

Some of the source and sink types have optional or mandatory security information for authentication purposes.

All sources and sinks that use services in the Oracle Cloud Infrastructure (OCI) can use certain parameters for providing optional security information. This information can be provided using an OCI configuration file or Instance Principal.

Oracle NoSQL Database sources and sinks require mandatory security information if the installation is secure and uses an Oracle Wallet-based authentication. This information can be provided by adding a jar file to the <MIGRATOR_HOME>/lib directory.

If an Oracle NoSQL Database installation uses Oracle Wallet-based authentication, you must include additional jar files that are a part of the EE installation. For more information, see Oracle Wallet.

Without the jar files, you will get the following error message:

Could not find kvstore-ee.jar and kvstore-ee-<version>.jar in lib directory. Copy kvstore-ee.jar and kvstore-ee-<version>.jar to lib directory

Instance principals is an IAM service feature that enables instances to be authorized actors (or principals) that can perform actions on service resources. Each compute instance has its own identity, and it authenticates using the certificates added to it.

Oracle NoSQL Database Migrator provides an option to connect to a NoSQL cloud and OCI Object Storage sources and sinks using instance principal authentication. It is only supported when the NoSQL Database Migrator tool is used within an OCI compute instance, for example, the NoSQL Database Migrator tool running in a VM hosted on OCI. To enable this feature use the useInstancePrincipal attribute of the NoSQL cloud source and sink configuration file. For more information on configuration parameters for different types of sources and sinks, see Source Configuration Templates and Sink Configuration Templates .

For more information on instance principals, see Calling Services from an Instance.

Learn about the various steps involved in using the Oracle NoSQL Database Migrator utility for migrating your NoSQL data.

The high level flow of tasks involved in using NoSQL Database Migrator is depicted in the below figure.

Description of the illustration migrator_flow.eps

//Row 1
{
    "id" : 1,
    "name" : "xyz",
    "age" : 45,
    "_metadata" : {
        "expiration" : 1629709200000   //Row Expiration time in milliseconds
    }
}

//Row 2
{
    "id" : 2,
    "name" : "abc",
    "age" : 52,
    "_metadata" : {
        "expiration" : 1629709400000   //Row Expiration time in milliseconds
    }
}

//Row 3 No Metadata for below row as it will not expire
{
    "id" : 3,
    "name" : "def",
    "age" : 15
}

Extended time = expiration time - reference time

CREATE TABLE IF NOT EXISTS migrateID(ID INTEGER GENERATED ALWAYS AS IDENTITY, name STRING, course STRING, PRIMARY KEY
      (ID))

{"name":"John", "course":"Computer Science"}
{"name":"Jane", "course":"BioTechnology"}
{"name":"Tony", "course":"Electronics"}

{"ID":1001,"name":"Jane","course":"BioTechnology"}
{"ID":1003,"name":"John","course":"Computer Science"}
{"ID":1002,"name":"Tony","course":"Electronics"}

{"ID":1, "name":"John", "course":"Computer Science"}
{"ID":2, "name":"Jane", "course":"BioTechnology"}
{"ID":3, "name":"Tony", "course":"Electronics"}

{"ID":2003,"name":"John","course":"Computer Science"}
{"ID":2002,"name":"Tony","course":"Electronics"}
{"ID":2001,"name":"Jane","course":"BioTechnology"}

CREATE TABLE IF NOT EXISTS migrateID(ID INTEGER GENERATED BY DEFAULT AS IDENTITY, name STRING, course STRING, PRIMARY KEY
      (ID))

{"name":"John", "course":"Computer Science"}
{"name":"Jane", "course":"BioTechnology"}
{"name":"Tony", "course":"Electronics"}

{"ID":1,"name":"John","course":"Computer Science"}
{"ID":2,"name":"Jane","course":"BioTechnology"}
{"ID":3,"name":"Tony","course":"Electronics"}

{"ID":1, "name":"John", "course":"Computer Science"}
{"ID":2, "name":"Jane", "course":"BioTechnology"}
{"ID":3, "name":"Tony", "course":"Electronics"}

{"ID":1002,"name":"John","course":"Computer Science"}
{"ID":1001,"name":"Jane","course":"BioTechnology"}
{"ID":1003,"name":"Tony","course":"Electronics"}

{"ID":1,"name":"John","course":"Computer Science"}
{"ID":2,"name":"Jane","course":"BioTechnology"}
{"ID":3,"name":"Tony","course":"Electronics"}

SELECT max(ID) FROM migrateID

{"Column_1":3}

ALTER Table migrateID (MODIFY ID GENERATED BY DEFAULT AS IDENTITY (START WITH 4))

Learn how to perform data migration using the Oracle NoSQL Database Migrator for specific use cases. You can find detailed systematic instructions with code examples to perform migration in each of the use cases.

This article has the following topics:

This example shows how to use the Oracle NoSQL Database Migrator to copy data and the schema definition of a NoSQL table from Oracle NoSQL Database Cloud Service (NDCS) to a JSON file.

1. Open the command prompt and navigate to the directory where you extracted the NoSQL Database Migrator utility.
2. To generate the configuration file using the NoSQL Database Migrator, run the runMigrator command without any runtime parameters.

   [~/nosqlMigrator]$./runMigrator

3. As you did not provide the configuration file as a runtime parameter, the utility prompts if you want to generate the configuration now. Type y.

   Configuration file is not provided. Do you want to generate
   configuration? (y/n) [n]: y
   
   Generating a configuration file interactively.
   
   

4. Based on the prompts from the utility, choose your options for the Source configuration.

   Enter a location for your config [./migrator-config.json]: /home/<user>/nosqlMigrator/NDCS2JSON
   Select the source: 
   1) nosqldb
   2) nosqldb_cloud
   3) file
   4) object_storage_oci
   5) aws_s3
   #? 2
   
   Configuration for source type=nosqldb_cloud
   Enter endpoint URL or region ID of the Oracle NoSQL Database Cloud: us-phoenix-1
   Select the authentication type: 
   1) credentials_file
   2) instance_principal
   3) delegation_token
   #? 1
   Enter path to the file containing OCI credentials [/home/<user>/.oci/config]:
   Enter the profile name in OCI credentials file [DEFAULT]: 
   Enter the compartment name or id of the table []: developers
   Enter table name: myTable
   Include TTL data? If you select 'yes' TTL of rows will also 
   be included in the exported data.(y/n) [n]: 
   Enter percentage of table read units to be used for migration operation. (1-100) [90]:
   Enter store operation timeout in milliseconds. (1-30000) [5000]:

5. Based on the prompts from the utility, choose your options for the Sink configuration.

   Select the sink:
   1) nosqldb
   2) nosqldb_cloud
   3) file
   #? 3
   Configuration for sink type=file
   Enter path to a directory to store JSON data: /home/<user>/nosqlMigrator
   would you like to export data to multiple files for each source?(y/n) [y]: n
   Would you like to store JSON in pretty format? (y/n) [n]: y
   Would you like to migrate the table schema also? (y/n) [y]: y
   Enter path to a file to store table schema: /home/<user>/nosqlMigrator/myTableSchema

6. Based on the prompts from the utility, choose your options for the source data transformations. The default value is n.

   Would you like to add transformations to source data? (y/n) [n]:

7. Enter your choice to determine whether to proceed with the migration in case any record fails to migrate.

   Would you like to continue migration in case of any record/row is failed to migrate?: (y/n) [n]:
   

8. The utility displays the generated configuration on the screen.

   generated configuration is:
   {
     "source": {
       "type": "nosqldb_cloud",
       "endpoint": "us-phoenix-1",
       "table": "myTable",
       "compartment": "developers",
       "credentials": "/home/<user>/.oci/config",
       "credentialsProfile": "DEFAULT",
       "readUnitsPercent": 90,
       "requestTimeoutMs": 5000
     },
     "sink": {
       "type": "file",
       "format": "json",
       "useMultiFiles" : false,
       "schemaPath": "/home/<user>/nosqlMigrator/myTableSchema",
       "pretty": true,
       "dataPath": "/home/<user>/nosqlMigrator"
     },
     "abortOnError": true,
     "migratorVersion": "1.6.5"
   }

9. Finally, the utility prompts for your choice to decide whether to proceed with the migration with the generated configuration file or not. The default option is y.

   Note:

   If you select n, you can use the generated configuration file to run the migration using the ./runMigrator -c or the ./runMigrator --config option.

   would you like to run the migration with above configuration?
   If you select no, you can use the generated configuration file to run the migration using
   ./runMigrator --config /home/<user>/nosqlMigrator/NDCS2JSON
   (y/n) [y]:

10. The NoSQL Database Migrator migrates your data and schema from NDCS to the JSON file.

    Records provided by source=10,Records written to sink=10,Records failed=0,Records skipped=0.
    Elapsed time: 0min 1sec 277ms
    Migration completed.

-- Exported myTable Data. JSON files are created in the supplied data path
 
[~/nosqlMigrator]$cat myTable_1_5.json
{
  "id" : 10,
  "document" : {
    "course" : "Computer Science",
    "name" : "Neena",
    "studentid" : 105
  }
}
{
  "id" : 3,
  "document" : {
  "course" : "Computer Science",
    "name" : "John",
    "studentid" : 107
  }
}
{
  "id" : 4,
  "document" : {
    "course" : "Computer Science",
    "name" : "Ruby",
    "studentid" : 100
  }
}
{
  "id" : 6,
  "document" : {
    "course" : "Bio-Technology",
    "name" : "Rekha",
    "studentid" : 104
  }
}
{
  "id" : 7,
  "document" : {
    "course" : "Computer Science",
    "name" : "Ruby",
    "studentid" : 100
  }
}
{
  "id" : 5,
  "document" : {
    "course" : "Journalism",
    "name" : "Rani",
    "studentid" : 106
  }
}
{
  "id" : 8,
  "document" : {
    "course" : "Computer Science",
    "name" : "Tom",
    "studentid" : 103
  }
}
{
  "id" : 9,
  "document" : {
    "course" : "Computer Science",
    "name" : "Peter",
    "studentid" : 109
  }
}
{
  "id" : 1,
  "document" : {
    "course" : "Journalism",
    "name" : "Tracy",
    "studentid" : 110
  }
}
{
  "id" : 2,
  "document" : {
    "course" : "Bio-Technology",
    "name" : "Raja",
    "studentid" : 108
  }
}

-- Exported myTable Schema
 
[~/nosqlMigrator]$cat myTableSchema
CREATE TABLE IF NOT EXISTS myTable (id INTEGER, document JSON, PRIMARY KEY(SHARD(id)))

This example shows how to use the Oracle NoSQL Database Migrator to copy data and the schema definition of a NoSQL table from Oracle NoSQL Database to Oracle NoSQL Database Cloud Service (NDCS).

1. Prepare the configuration file (in JSON format) with the identified Source and Sink details. See Source Configuration Templates and Sink Configuration Templates .

   {
     "source" : {
       "type" : "nosqldb",
       "storeName" : "kvstore",
       "helperHosts" : ["<hostname>:5000"],
       "table" : "myTable",
       "requestTimeoutMs" : 5000
     },
     "sink" : {
       "type" : "nosqldb_cloud",
       "endpoint" : "us-phoenix-1",
       "table" : "myTable",
       "compartment" : "developers",
       "schemaInfo" : {
         "schemaPath" : "<complete/path/to/the/JSON/file/with/DDL/commands/for/the/schema/definition>",
         "readUnits" : 100,
         "writeUnits" : 100,
         "storageSize" : 1
       },
       "credentials" : "<complete/path/to/oci/config/file>",
       "credentialsProfile" : "DEFAULT",
       "writeUnitsPercent" : 90,
       "requestTimeoutMs" : 5000
     },
     "abortOnError" : true,
     "migratorVersion" : "1.0.0"
   }

2. Open the command prompt and navigate to the directory where you extracted the NoSQL Database Migrator utility.
3. Run the runMigrator command by passing the configuration file using the --config or -c option.

   [~/nosqlMigrator/nosql-migrator-1.0.0]$./runMigrator --config <complete/path/to/the/JSON/config/file>
   

4. The utility proceeds with the data migration, as shown below.

   Records provided by source=10, Records written to sink=10, Records failed=0.
   Elapsed time: 0min 10sec 426ms
   Migration completed.

This example shows the usage of Oracle NoSQL Database Migrator to copy data from a JSON file source to Oracle NoSQL Database Cloud Service.

1. Prepare the configuration file (in JSON format) with the identified source and sink details. See Source Configuration Templates and Sink Configuration Templates .

   {
     "source" : {
       "type" : "file",
       "format" : "json",
       "schemaInfo" : {
         "schemaPath" : "[~/nosql-migrator-1.5.0]/schema_json.ddl"
       },
       "dataPath" : "[~/nosql-migrator-1.5.0]/SampleData.json"
     },
     "sink" : {
       "type" : "nosqldb_cloud",
       "endpoint" : "us-ashburn-1",
       "table" : "Migrate_JSON",
       "compartment" : "Training-NoSQL",
       "includeTTL" : false,
       "schemaInfo" : {
         "readUnits" : 100,
         "writeUnits" : 60,
         "storageSize" : 1,
         "useSourceSchema" : true
       },
       "credentials" : "/home/user/.oci/config",
       "credentialsProfile" : "DEFAULT",
       "writeUnitsPercent" : 90,
       "overwrite" : true,
       "requestTimeoutMs" : 5000
     },
     "abortOnError" : true,
     "migratorVersion" : "1.5.0"
   }

2. Open the command prompt and navigate to the directory where you extracted the Oracle NoSQL Database Migrator utility.
3. Run the runMigrator command by passing the configuration file using the --config or -c option.

   [~/nosql-migrator-1.5.0]$./runMigrator --config <complete/path/to/the/config/file>

4. The utility proceeds with the data migration, as shown below. The Migrate_JSON table is created at the sink with the schema provided in the schemaPath.

   creating source from given configuration:
   source creation completed
   creating sink from given configuration:
   sink creation completed
   creating migrator pipeline
   migration started
   [cloud sink] : start loading DDLs
   [cloud sink] : executing DDL: create table Migrate_JSON (id INTEGER, val_json JSON, PRIMARY KEY(id)),limits: [100, 60, 1]
   [cloud sink] : completed loading DDLs
   [cloud sink] : start loading records
   [json file source] : start parsing JSON records from file: SampleData.json
   [INFO] migration completed.
   Records provided by source=4, Records written to sink=4, Records failed=0, Records skipped=0.
   Elapsed time: 0min 5sec 778ms
   Migration completed.

Validation

To validate the migration, you can log in to your Oracle NoSQL Database Cloud Service console and verify that the Migrate_JSON table is created with the source data. For the procedure to access the console, see Accessing the Service from the Infrastructure Console article in the Oracle NoSQL Database Cloud Service document.

Figure - Oracle NoSQL Database Cloud Service Console Tables

Description of "Figure - Oracle NoSQL Database Cloud Service Console Tables"

Figure - Oracle NoSQL Database Cloud Service Console Table Data

Description of "Figure - Oracle NoSQL Database Cloud Service Console Table Data"

This example shows how to use the Oracle NoSQL Database Migrator to copy Mongo-DB Formatted Data to the Oracle NoSQL Database Cloud Service (NDCS).

{"_id":0,"name":"Aimee Zank","scores":[{"score":1.463179736705023,"type":"exam"},{"score":11.78273309957772,"type":"quiz"},{"score":35.8740349954354,"type":"homework"}]}
{"_id":1,"name":"Aurelia Menendez","scores":[{"score":60.06045071030959,"type":"exam"},{"score":52.79790691903873,"type":"quiz"},{"score":71.76133439165544,"type":"homework"}]}
{"_id":2,"name":"Corliss Zuk","scores":[{"score":67.03077096065002,"type":"exam"},{"score":6.301851677835235,"type":"quiz"},{"score":66.28344683278382,"type":"homework"}]}
{"_id":3,"name":"Bao Ziglar","scores":[{"score":71.64343899778332,"type":"exam"},{"score":24.80221293650313,"type":"quiz"},{"score":42.26147058804812,"type":"homework"}]}
{"_id":4,"name":"Zachary Langlais","scores":[{"score":78.68385091304332,"type":"exam"},{"score":90.2963101368042,"type":"quiz"},{"score":34.41620148042529,"type":"homework"}]}

1. Prepare the configuration file (in JSON format) with the identified Source and Sink details. See Source Configuration Templates and Sink Configuration Templates .

   {
     "source" : {
       "type" : "file",
       "format" : "mongodb_json",
       "dataPath" : "<complete/path/to/the/MongoDB/Formatted/JSON/file>"
     },
     "sink" : {
       "type" : "nosqldb_cloud",
       "endpoint" : "us-phoenix-1",
       "table" : "mongoImport",
       "compartment" : "developers",
       "schemaInfo" : {
         "defaultSchema" : true,
         "readUnits" : 100,
         "writeUnits" : 60,
         "storageSize" : 1
       },
       "credentials" : "<complete/path/to/the/oci/config/file>",
       "credentialsProfile" : "DEFAULT",
       "writeUnitsPercent" : 90,
       "requestTimeoutMs" : 5000
     },
     "abortOnError" : true,
     "migratorVersion" : "1.0.0"
   }

2. Open the command prompt and navigate to the directory where you extracted the NoSQL Database Migrator utility.
3. Run the runMigrator command by passing the configuration file using the --config or -c option.

   [~/nosqlMigrator/nosql-migrator-1.0.0]$./runMigrator --config <complete/path/to/the/JSON/config/file>
   

4. The utility proceeds with the data migration, as shown below.

   Records provided by source=29,353, Records written to sink=29,353, Records failed=0.
   Elapsed time: 9min 9sec 630ms
   Migration completed.

This example shows how to use Oracle NoSQL Database Migrator to copy DynamoDB JSON file to NoSQL Database.

Use Case:

After evaluating multiple options, an organization finalizes Oracle NoSQL Database over DynamoDB database. The organization wants to migrate their tables and data from DynamoDB to Oracle NoSQL Database (on-premises).

See Mapping of DynamoDB table to Oracle NoSQL table for more details.

You can migrate a file or directory containing the DynamoDB exported JSON data from a file system by specifying the path in the source configuration template.

{"Item":{"Id":{"N":"101"},"Phones":{"L":[{"L":[{"S":"555-222"},{"S":"123-567"}]}]},"PremierCustomer":{"BOOL":false},"Address":{"M":{"Zip":{"N":"570004"},"Street":{"S":"21 main"},"DoorNum":{"N":"201"},"City":{"S":"London"}}},"FirstName":{"S":"Fred"},"FavNumbers":{"NS":["10"]},"LastName":{"S":"Smith"},"FavColors":{"SS":["Red","Green"]},"Age":{"N":"22"},"ttl": {"N": "1734616800"}}}
{"Item":{"Id":{"N":"102"},"Phones":{"L":[{"L":[{"S":"222-222"}]}]},"PremierCustomer":{"BOOL":false},"Address":{"M":{"Zip":{"N":"560014"},"Street":{"S":"32 main"},"DoorNum":{"N":"1024"},"City":{"S":"Wales"}}},"FirstName":{"S":"John"},"FavNumbers":{"NS":["10"]},"LastName":{"S":"White"},"FavColors":{"SS":["Blue"]},"Age":{"N":"48"},"ttl": {"N": "1734616800"}}}

You copy the exported DynamoDB table data from AWS S3 storage to a local mounted file system.

 java -jar lib/sql.jar -helper-hosts localhost:5000 -store kvstore

SELECT * FROM sampledynDBImp

{"Id":102,"document":{"Address":{"City":"Wales","DoorNum":1024,"Street":"32 main","Zip":560014},"Age":48,"FavColors":["Blue"],"FavNumbers":[10],"FirstName":"John","LastName":"White","Phones":[["222-222"]],"PremierCustomer":false,"_metadata":{"expiration":1734616196000}}}
{"Id":101,"document":{"Address":{"City":"London","DoorNum":201,"Street":"21 main","Zip":570004},"Age":22,"FavColors":["Red","Green"],"FavNumbers":[10],"FirstName":"Fred","LastName":"Smith","Phones":[["555-222","123-567"]],"PremierCustomer":false,"_metadata":{"expiration":1734616196000}}}

This example shows how to use the Oracle NoSQL Database Migrator to copy DynamoDB JSON file stored in an AWS S3 store to the Oracle NoSQL Database Cloud Service (NDCS).

Use Case:

After evaluating multiple options, an organization finalizes Oracle NoSQL Database Cloud Service over DynamoDB database. The organization wants to migrate their tables and data from DynamoDB to Oracle NoSQL Database Cloud Service.

See Mapping of DynamoDB table to Oracle NoSQL table for more details.

You can migrate a file containing the DynamoDB exported JSON data from the AWS S3 storage by specifying the path in the source configuration template.

{"Item":{"Id":{"N":"101"},"Phones":{"L":[{"L":[{"S":"555-222"},{"S":"123-567"}]}]},"PremierCustomer":{"BOOL":false},"Address":{"M":{"Zip":{"N":"570004"},"Street":{"S":"21 main"},"DoorNum":{"N":"201"},"City":{"S":"London"}}},"FirstName":{"S":"Fred"},"FavNumbers":{"NS":["10"]},"LastName":{"S":"Smith"},"FavColors":{"SS":["Red","Green"]},"Age":{"N":"22"}}}
{"Item":{"Id":{"N":"102"},"Phones":{"L":[{"L":[{"S":"222-222"}]}]},"PremierCustomer":{"BOOL":false},"Address":{"M":{"Zip":{"N":"560014"},"Street":{"S":"32 main"},"DoorNum":{"N":"1024"},"City":{"S":"Wales"}}},"FirstName":{"S":"John"},"FavNumbers":{"NS":["10"]},"LastName":{"S":"White"},"FavColors":{"SS":["Blue"]},"Age":{"N":"48"}}}

You export the DynamoDB table to AWS S3 storage as specified in Exporting DynamoDB table data to Amazon S3.

This example shows the usage of Oracle NoSQL Database Migrator to perform cross-region migration.

[DEFAULT]
user=ocid1.user.oc1....
fingerprint=fd:96:....
tenancy=ocid1.tenancy.oc1....
region=us-ashburn-1
key_file=</fully/qualified/path/to/the/private/key/>
pass_phrase=abcd


[DEFAULT2]
user=ocid1.user.oc1....
fingerprint=1b:68:....
tenancy=ocid1.tenancy.oc1....
region=us-phoenix-1
key_file=</fully/qualified/path/to/the/private/key/>
pass_phrase=23456

1. Prepare the configuration file (in JSON format) with the identified source and sink details. See Source Configuration Templates and Sink Configuration Templates .

   {
     "source" : {
       "type" : "nosqldb_cloud",
       "endpoint" : "us-ashburn-1",
       "table" : "user_data",
       "compartment" : "ocid1.compartment.oc1..aaaaaaaahcrgrgptoaq4cgpoymd32ti2ql4sdpu5puroausdf4og55z4tnya",
       "credentials" : "/home/<user>/.oci/config",
       "credentialsProfile" : "DEFAULT",
       "readUnitsPercent" : 100,
       "includeTTL" : false,
       "requestTimeoutMs" : 5000
     },
     "sink" : {
       "type" : "nosqldb_cloud",
       "endpoint" : "us-phoenix-1",
       "table" : "user_data",
       "compartment" : "ocid1.compartment.oc1..aaaaaaaaleiwplazhwmicoogv3tf4lum4m4nzbcv5wfjmoxuz3doreagvdma",
       "includeTTL" : false,
       "schemaInfo" : {
         "readUnits" : 100,
         "writeUnits" : 60,
         "storageSize" : 1,
         "useSourceSchema" : true
       },
       "credentials" : "/home/<user>/.oci/config",
       "credentialsProfile" : "DEFAULT2",
       "writeUnitsPercent" : 90,
       "overwrite" : true,
       "requestTimeoutMs" : 5000
     },
     "abortOnError" : true,
     "migratorVersion" : "1.5.0"
   }

2. On your machine, navigate to the directory where you extracted the NoSQL Database Migrator utility. You can access the runMigrator command from the command line interface.
3. Run the runMigrator command by passing the configuration file using the --config or -c option.

   [~/nosql-migrator-1.5.0]$./runMigrator --config <complete/path/to/the/config/file>

4. The utility proceeds with data migration as shown below. The user_data table is created at the sink with the same schema as the source table as you have configured the useSourceSchema parameter as true in the sink configuration template.

   creating source from given configuration:
   source creation completed
   creating sink from given configuration:
   sink creation completed
   creating migrator pipeline
   migration started
   [cloud sink] : start loading DDLs
   [cloud sink] : executing DDL: CREATE TABLE IF NOT EXISTS user_data (id INTEGER, firstName STRING, lastName STRING, otherNames ARRAY(RECORD(first STRING, last STRING)), age INTEGER, income INTEGER, address JSON, connections ARRAY(INTEGER), email STRING, communityService STRING, PRIMARY KEY(SHARD(id))),limits: [100, 60, 1]
   [cloud sink] : completed loading DDLs
   [cloud sink] : start loading records
   migration completed.
   Records provided by source=5, Records written to sink=5, Records failed=0, Records skipped=0.
   Elapsed time: 0min 5sec 603ms
   Migration completed.

   Note:

   • If the table already exists at the sink with the same schema as the source table, the rows with the same primary keys are overwritten as you have provided the overwrite parameter as true in the configuration template.
   • If the table already exists at the sink with a different schema from the source table, the migration will fail.

This example shows the usage of Oracle NoSQL Database Migrator from a Cloud Shell.

1. Prepare the configuration file (in JSON format) with the identified source and sink details. See Source Configuration Templates and Sink Configuration Templates . Ensure that you set the useDelegationToken parameter to true in source and sink configuration templates.
   The following configuration template is used in this use case:

   {
     "source" : {
       "type" : "nosqldb_cloud",
       "endpoint" : "us-ashburn-1",
       "table" : "NDCSupload",
       "compartment" : "ocid1.compartment.oc1..aaaaaaaahcrgrgptoaq4cgpoymd32ti2ql4sdpu5puroausdf4og55z4tnya",
       "useDelegationToken" : true,
       "readUnitsPercent" : 90,
       "includeTTL" : true,
       "requestTimeoutMs" : 5000
     },
     "sink" : {
       "type" : "object_storage_oci",
       "format" : "json",
       "endpoint" : "us-ashburn-1",
       "namespace" : "",
       "bucket" : "Migrate_oci",
       "prefix" : "Delegation",
       "chunkSize" : 32,
       "compression" : "",
       "useDelegationToken" : true
     },
     "abortOnError" : true,
     "migratorVersion" : "1.6.0"
   }

2. From your Cloud Shell, navigate to the directory where you extracted the NoSQL Database Migrator utility.
3. Run the runMigrator command by passing the configuration file using the --config or -c option

   [~/nosql-migrator-1.6.0]$./runMigrator --config <complete/path/to/the/config/file>

4. The NoSQL Database Migrator utility proceeds with data migration. As you have set the useDelegationToken parameter to true, the Cloud Shell automatically authenticates using the delegation token while running the NoSQL Database Migrator utility. The NoSQL Database Migrator copies your data from the NDCSupload table to a JSON file in the Object Storage bucket Migrate_oci. Check the logs for successful data backup.

   [INFO] creating source from given configuration:
   [INFO] source creation completed
   [INFO] creating sink from given configuration:
   [INFO] sink creation completed
   [INFO] creating migrator pipeline
   [INFO] migration started
   [INFO] [OCI OS sink] : writing table schema to Delegation/Schema/schema.ddl
   [INFO] [OCI OS sink] : start writing records with prefix Delegation
   [INFO] migration completed.
   Records provided by source=4,Records written to sink=4,Records failed=0.
   Elapsed time: 0min 0sec 486ms
   Migration completed.

   Note:

   Data is copied to the file: Migrate_oci/NDCSupload/Delegation/Data/000000.json

   Depending on the chunkSize parameter in the sink configuration template, the source data can be split into several JSON files in the same directory.

   The schema is copied to the file: Migrate_oci/NDCStable1/Delegation/Schema/schema.ddl

This example shows how to use Oracle NoSQL Database Migrator with OKE Workload Identity Authentication to copy data from a JSON file in the OCI Object Storage to Oracle NoSQL Database Cloud Service table.

1. Prepare a Migrator configuration file, migrator-config.json with OCI OS source and NDCS sink. For templates, see Source Configuration Templates and Sink Configuration Templates.

   To use the OKE WIA to access OCI OS bucket and NDCS, set the useOKEWorkloadIdentity parameter to true in both source and sink configuration templates. Here, you will use source schema from the schema DDL file in the OCI OS bucket. Therefore, set useSourceSchema parameter to true in the sink configuration template.

   Note:

   While using OKE WIA, you can't generate the Migrator configuration file interactively. You must prepare the configuration file manually by referring to the source and sink configuration templates.

   {
     "source" : {
       "type" : "object_storage_oci",
       "format" : "json",
       "endpoint" : "us-ashburn-1",
       "namespace" : "",
       "bucket" : "Migrate_oci",
       "prefix" : "userSession",
       "useOKEWorkloadIdentity" : true
     },
     "sink" : {
       "type" : "nosqldb_cloud",
       "endpoint" : "us-ashburn-1",
       "table" : "users",
       "compartment" : "Training-NoSQL",
       "includeTTL" : true,
       "schemaInfo" : {
         "readUnits" : 100,
         "writeUnits" : 60,
         "storageSize" : 1,
         "useSourceSchema" : true
       },
       "useOKEWorkloadIdentity" : true,
       "writeUnitsPercent" : 90,
       "overwrite" : true,
       "requestTimeoutMs" : 5000
     },
     "abortOnError" : true,
     "migratorVersion" : "1.7.0"
   }

2. Create a configure map resource (configmap) to pass the migrator-config.json configuration input file to the Migrator container at runtime in the Kubernetes pod. The configmap mounts the Migrator configuration file in the file system of the container as a mount volume.

   #Command:
   kubectl create configmap oci-migrator-config --from-file=<Migrator configuration file> -n <namespace-name>

   #Example:
   kubectl create configmap oci-migrator-config --from-file=migrator-config.json -n migrator

   #Output:
   configmap/oci-migrator-config created

3. Create a Docker registry secret that includes the OCI credentials to use while pulling the Migrator Docker image from Container Registry into the Kubernetes pod.

   #Command:
   kubectl create secret docker-registry ocirsecret --docker-server=<region-key>.ocir.io --docker-username='tenancy-namespace/username' --docker-password='auth token' --docker-email='<user Email>' -n <namespace-name>

   #Example:
   kubectl create secret docker-registry ocirsecret --docker-server=iad.ocir.io --docker-username='idhx..xxwjzj/rx..xxxxh@oracle.com' --docker-password='<Auth token>' --docker-email='rx..xxxxh@oracle.com' -n migrator

   #Output:
   secret/ocirsecret created

4. Create a manifest file, which you will use to specify the Migrator Docker image. Ensure that you provide the values from the previous steps for namespace, Kubernetes service account name, Docker image name, Docker registry secret, and configmap. You can refer to the following sample manifest file:

   #migrator-deployment.yaml
   
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: nosql-migrator
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: nosql-migrator
     template:
       metadata:
         labels:
           app: nosql-migrator
       spec:
         serviceAccountName: migratorsa
         automountServiceAccountToken: true
         containers:
           - name: nosqlmigrator
             image: iad.ocir.io/idhx..xxwjzj/okemigrator:1.0
             imagePullPolicy: Always
             args: ["--config", "/config/migrator-config.json", "--log-level", "DEBUG"]
             volumeMounts:
               - name: config-volume
                 mountPath: /config  # Mount the file here
         imagePullSecrets:
           - name: ocirsecret
         volumes:
           - name: config-volume
             configMap:
               name: oci-migrator-config
   

5. Deploy the Migrator Docker image in the Kubernetes pod using the following command. OKE runs the Migrator utility in a container on one of the nodes of the Kubernetes cluster.

   #Command:
   kubectl create -f <manifest file> -n <namespace-name>

   #Example:
   kubectl create -f migrator-deployment.yaml -n migrator

   #Output:
   deployment.apps/nosql-migrator created

6. You can check the logs using the following commands:
   1. Fetch the pod's name on which the Migrator utility is running.

      #Command:
      kubectl get pods -n <namespace-name>

      #Example:
      kubectl get pods -n migrator

      #Output:
      NAME                            READY   STATUS    RESTARTS   AGE
      nosql-migrator-ccdbf549-6sjjg   1/1     Running   0          56s

   2. Use the pod's name to fetch the Migrator logs.

      #Command:
      kubectl logs -f <kubernetes cluster pod name> -n <namespace-name>

      #Example:
      kubectl logs -f nosql-migrator-ccdbf549-6sjjg -n migrator

      #Output:
      SLF4J(I): Connected with provider of type [org.apache.logging.slf4j.SLF4JServiceProvider]
      [INFO] Configuration for migration:
      {
        "source" : {
          "type" : "object_storage_oci",
          "format" : "json",
          "endpoint" : "us-ashburn-1",
          "namespace" : "idhkv1iewjzj",
          "bucket" : "Migrate_oci",
          "prefix" : "userSession",
          "useOKEWorkloadIdentity" : true
        },
        "sink" : {
          "type" : "nosqldb_cloud",
          "endpoint" : "us-ashburn-1",
          "table" : "users",
          "compartment" : "Training-NoSQL",
          "includeTTL" : true,
          "schemaInfo" : {
            "readUnits" : 100,
            "writeUnits" : 60,
            "storageSize" : 1,
            "useSourceSchema" : true
          },
          "useOKEWorkloadIdentity" : true,
          "writeUnitsPercent" : 90,
          "overwrite" : true,
          "requestTimeoutMs" : 5000
        },
        "abortOnError" : true,
        "migratorVersion" : "1.7.0"
      }
      [INFO] creating source from given configuration:
      [INFO] source creation completed
      [INFO] creating sink from given configuration:
      [INFO] sink creation completed
      [INFO] creating migrator pipeline
      [INFO] migration started
      [INFO] [cloud sink] : start loading DDLs
      [INFO] [cloud sink] : completed loading DDLs
      [INFO] [cloud sink] : start loading records
      [INFO] [OCI OS source] : start parsing JSON records from object: users/userSession/Data/users_1_5_0.json
      [INFO] [OCI OS source] : start parsing JSON records from object: users/userSession/Data/users_6_10_0.json
      [INFO] migration completed.
      Records provided by source=5, Records written to sink=5, Records failed=0, Records skipped=0.
      Elapsed time: 0min 1sec 15ms
      Migration completed.
      

   Note:

   You can iteratively run data migration as follows:

   1. Delete the nosql-migrator container using the following command:

      #Command:
      kubectl delete -f <manifest file> -n <namespace-name>

      #Example:
      kubectl delete -f migrator-deployment.yaml -n migrator

      #Output:
      deployment.apps "nosql-migrator" deleted

   2. Update the migrator-config.json configuration input file.
   3. Delete and recreate the configmap using the commands from Step 2.

      It is not required to create the Docker registry secret, and manifest file again.

   4. Deploy the Migrator Docker image in the Kubernetes pod using the manifest file (Step 5).

This example shows how to use Oracle NoSQL Database Migrator with session token authentication to copy data from Oracle NoSQL Database table to a JSON file in an OCI Object Storage bucket.

1. Prepare the configuration file (in JSON format) with Oracle NoSQL Database source and JSON file in the OCI OS bucket sink. For templates, see Source Configuration Templates and Sink Configuration Templates .
   To use the session token authentication to access OCI OS bucket, set the useSessionToken parameter to true in the sink configuration template. Correspondingly, specify the config path in the credentials parameter and the profile name in the credentialsProfile parameter.

   {
     "source" : {
       "type" : "nosqldb",
       "storeName" : "kvstore",
       "helperHosts" : ["<hostname>:<port>"],
       "table" : "users",
       "includeTTL" : true,
       "requestTimeoutMs" : 5000
     },
     "sink" : {
       "type" : "object_storage_oci",
       "format" : "json",
       "endpoint" : "us-ashburn-1",
       "namespace" : "idhkv1iewjzj",
       "bucket" : "Migrate_oci",
       "prefix" : "userSession",
       "chunkSize" : 32,
       "compression" : "",
       "useSessionToken" : true,
       "credentials" : "$/home/.oci/config",
       "credentialsProfile" : "SESSIONPROFILE"
     },
     "abortOnError" : true,
     "migratorVersion" : "<latest>"
   }

2. Open the command prompt and navigate to the directory where you extracted the NoSQL Database Migrator utility.
3. Run the runMigrator command by passing configuration file option. Use the --config or -c option to pass the configuration file as follows:

   ./runMigrator --config ./migrator-config.json

4. The Migrator utility proceeds with data migration. A sample output is shown below.
   With useSessionToken parameter to true, the Migrator utility automatically authenticates using the session token. The Migrator utility copies your data from users table to a JSON file in the OCI OS bucket named Migrate_oci. Check the logs for successful data backup.

   [INFO] creating source from given configuration:
   [INFO] source creation completed
   [INFO] creating sink from given configuration:
   [INFO] sink creation completed
   [INFO] creating migrator pipeline
   [INFO] [OCI OS sink] : writing table schema to userSession/Schema/schema.ddl
   [INFO] migration started
   [INFO] Migration success for source users_6_10. read=2,written=2,failed=0
   [INFO] Migration success for source users_1_5. read=3,written=3,failed=0
   [INFO] Migration is successful for all the sources.
   [INFO] migration completed.
   Records provided by source=5, Records written to sink=5, Records failed=0,Records skipped=0.
   Elapsed time: 0min 0sec 982ms
   Migration completed.
   

   Note:

   Depending on the chunkSize parameter in the sink configuration template, the Migrator utility splits the source data into several JSON files in the same directory. In this example, the Migrator utility copies data to users_1_5_0.json and users_6_10_0.json files in Migrate_oci/userSession/Data directory.

   The source table schema is copied to schema.ddl file in Migrate_oci/userSession/Schema directory.

This example shows the usage of Oracle NoSQL Database Migrator to copy data from a CSV file to Oracle NoSQL Database.

1. Open the command prompt and navigate to the directory where you extracted the Oracle NoSQL Database Migrator utility.
2. To generate the configuration file using Oracle NoSQL Database Migrator, execute the runMigrator command without any runtime parameters.

   [~/nosql-migrator-1.5.0]$./runMigrator

3. As you did not provide the configuration file as a runtime parameter, the utility prompts if you want to generate the configuration now. Type y.
   You can choose a location for the configuration file or retain the default location by pressing the Enter key.

   
   Configuration file is not provided. Do you want to generate
   configuration? (y/n) [n]: y
   Generating a configuration file interactively.
   
   Enter a location for your config [./migrator-config.json]: 
   ./migrator-config.json already exist. Do you want to overwrite?(y/n) [n]: y
   

4. Based on the prompts from the utility, choose your options for the Source configuration.

   
   Select the source: 
   1) nosqldb
   2) nosqldb_cloud
   3) file
   4) object_storage_oci
   5) aws_s3
   #? 3
   
   Configuration for source type=file
   Select the source file format: 
   1) json
   2) mongodb_json
   3) dynamodb_json
   4) csv
   #? 4
   

5. Provide the path to the source CSV file. Further, based on the prompts from the utility, you can choose to reorder the column names, select the encoding method, and trim the tailing spaces from the target table.

   
   Enter path to a file or directory containing csv data: [~/nosql-migrator-1.5.0]/course.csv
   Does the CSV file contain a headerLine? (y/n) [n]: n
   Do you want to reorder the column names of NoSQL table with respect to
   CSV file columns? (y/n) [n]: n
   Provide the CSV file encoding. The supported encodings are:
   UTF-8,UTF-16,US-ASCII,ISO-8859-1. [UTF-8]: 
   Do you want to trim the tailing spaces? (y/n) [n]: n
   

6. Based on the prompts from the utility, choose your options for the Sink configuration.

   
   Select the sink:
   1) nosqldb
   2) nosqldb_cloud
   #? 1
   Configuration for sink type=nosqldb
   Enter store name of the Oracle NoSQL Database: mystore
   Enter comma separated list of host:port of Oracle NoSQL Database: <hostname>:5000
   

7. Based on the prompts from the utility, provide the name of the target table.

   
   Enter fully qualified table name: course
   

8. Enter your choice to set the TTL value. The default value is n.

   
   Include TTL data? If you select 'yes' TTL value provided by the
   source will be set on imported rows. (y/n) [n]: n
   

9. Based on the prompts from the utility, specify whether or not the target table must be created through the Oracle NoSQL Database Migrator tool. If the table is already created, it is suggested to provide n. If the table is not created, the utility will request the path for the file containing the DDL commands for the schema of the target table.

   
   Would you like to create table as part of migration process?
   Use this option if you want to create table through the migration tool.
   If you select yes, you will be asked to provide a file that contains
   table DDL or to use schema provided by the source or default schema.
   (y/n) [n]: y
   Enter path to a file containing table DDL: [~/nosql-migrator-1.5.0]/mytable_schema.ddl
   Is the store secured? (y/n) [y]: n
   would you like to overwrite records which are already present?
   If you select 'no' records with same primary key will be skipped [y/n] [y]: y
   Enter store operation timeout in milliseconds. [5000]:
   Would you like to add transformations to source data? (y/n) [n]: n
   

10. Enter your choice to determine whether to proceed with the migration in case any record fails to migrate.

    
    Would you like to continue migration if any data fails to be migrated? 
    (y/n) [n]: n
    

11. The utility displays the generated configuration on the screen.

    
    Generated configuration is:
    {
      "source" : {
        "type" : "file",
        "format" : "csv",
        "dataPath" : "[~/nosql-migrator-1.5.0]/course.csv",
        "hasHeader" : false,
        "csvOptions" : {
          "encoding" : "UTF-8",
          "trim" : false
        }
      },
      "sink" : {
        "type" : "nosqldb",
        "storeName" : "mystore",
        "helperHosts" : ["<hostname>:5000"],
        "table" : "migrated_table",
        "query" : "",
        "includeTTL" : false,
        "schemaInfo" : {
          "schemaPath" : "[~/nosql-migrator-1.5.0]/mytable_schema.ddl"
        },
        "overwrite" : true,
        "requestTimeoutMs" : 5000
      },
      "abortOnError" : true,
      "migratorVersion" : "1.5.0"
    }
    

12. Finally, the utility prompts you to specify whether or not to proceed with the migration using the generated configuration file. The default option is y.
    Note: If you select n, you can use the generated configuration file to perform the migration. Specify the ./runMigrator -c or the ./runMigrator --config option.

    
    Would you like to run the migration with above configuration?
    If you select no, you can use the generated configuration file to
    run the migration using:
    ./runMigrator --config ./migrator-config.json
    (y/n) [y]: y
    
    

13. The NoSQL Database Migrator copies your data from the CSV file to Oracle NoSQL Database.

    
    creating source from given configuration:
    source creation completed
    creating sink from given configuration:
    sink creation completed
    creating migrator pipeline
    migration started
    [nosqldb sink] : start loading DDLs
    [nosqldb sink] : executing DDL: create table course (id INTEGER, name STRING, location STRING, fees INTEGER, PRIMARY KEY(id))
    [nosqldb sink] : completed loading DDLs
    [nosqldb sink] : start loading records
    [csv file source] : start parsing CSV records from file: course.csv
    migration completed. Records provided by source=4, Records written to sink=4, Records failed=0,Records skipped=0.
    Elapsed time: 0min 0sec 559ms
    Migration completed.
    

 java -jar lib/sql.jar -helper-hosts localhost:5000 -store kvstore

sql-> select * from course;
{"id":4,"name":"Telecommunication","location":"San Francisco","fees":2500}
{"id":1,"name":"Computer Science","location":"San Francisco","fees":2500}
{"id":2,"name":"Bio-Technology","location":"Los Angeles","fees":1200}
{"id":3,"name":"Journalism","location":"Las Vegas","fees":1500}
 
4 rows returned