1. Migration de données de MongoDB vers Oracle NoSQL Database Cloud Service
  2. Présentation des fichiers utilisés lors de la migration et des exemples de sortie

Présentation des fichiers utilisés lors de la migration et des exemples de sortie

Consultez les exemples de fichiers et de sorties de commande suivants utilisés lors de la migration des données.

A propos des fichiers JSON formatés MongoDB

Vous pouvez générer des fichiers JSON formatés MongoDB à partir de MongoDB pour migrer vos données. Voici un exemple de fichier JSON formaté MongoDB : 

[opc@oci-compute ~]$ head -3 ~/zips_orig.json 
{ "_id" : "01001", "city" : "AGAWAM", "loc" : [ -72.622739, 42.070206 ], "pop" : 15338, "state" : "MA" }
{ "_id" : "01002", "city" : "CUSHMAN", "loc" : [ -72.51564999999999, 42.377017 ], "pop" : 36963, "state" : "MA" }
{ "_id" : "01005", "city" : "BARRE", "loc" : [ -72.10835400000001, 42.409698 ], "pop" : 4546, "state" : "MA" }
[opc@oci-compute nosql-migrator-1.5.0]$
[opc@oci-compute ~]$ cat zips.json | wc -l29353
[opc@oci-compute ~]$

Exemple de commande permettant de fractionner le fichier JSON formaté MongoDB

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/.oci/config 
[DEFAULT]
user=user_OCID
fingerprint=user_fingerprint
tenancy=tenancy_OCID
region=region_ID
key_file=full_path_to_private_key_pem_file
[opc@oci-compute nosql-migrator-1.5.0]$

A propos des fichiers du langage de définition de données Oracle NoSQL Database Cloud Service

Lors de la migration de données à l'aide de la méthode de schéma personnalisé Oracle NoSQL Database Cloud Service, le nouveau schéma de table Oracle NoSQL Database Cloud Service doit être défini comme le schéma de la table source à l'aide d'un langage DDL. Les noms et types d'attribut de la nouvelle table doivent être identiques aux noms et types d'attribut de la table source.

Voici un exemple de fichier de langage de définition de données de schéma Oracle NoSQL Database Cloud Service. 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/movies.ddl CREATE TABLE IF NOT EXISTS movies (title String,year Number,info JSON, PRIMARY KEY(SHARD(title),year))
[opc@oci-compute nosql-migrator-1.5.0]$

A propos des fichiers d'identité OCI

Le fichier d'identité OCI contient des informations sur l'utilisateur effectuant une migration vers Oracle NoSQL Database Cloud Service. Voici un exemple de fichier d'identité OCI : 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/.oci/config 
[DEFAULT]
user=user_OCID
fingerprint=user_fingerprint
tenancy=tenancy_OCID
region=region_ID
key_file=full_file_path_to_PEM_file
[opc@oci-compute nosql-migrator-1.5.0]$

A propos de la sortie des migrations de schéma par défaut Oracle NoSQL Database

La migration vers MongoDB génère un fichier de sortie avec le schéma JSON de la configuration. Voici un exemple de sortie d'un schéma par défaut Oracle NoSQL Database Migration : 

[opc@oci-compute nosql-migrator-1.5.0]$ ./runMigrator --config ./migrator-config-mongodbjson-bucket.json --log-level debug
2023-02-15 17:42:46.955 [INFO] Configuration for migration:
{
 "source" : {
 "type" : "object_storage_oci",
 "format" : "mongodb_json",
 "endpoint" : "https://objectstorage.us-phoenix-1.oraclecloud.com",
 "namespace" : "namespace_name",
 "bucket" : "bucket_name",
 "prefix" : "zipssa",
 "credentials" : "/home/opc/.oci/config",
 "credentialsProfile" : "DEFAULT"
 },
 "sink" : {
 "type" : "nosqldb_cloud",
 "endpoint" : "https://nosql.us-phoenix-1.oci.oraclecloud.com",
 "table" : "zips",
 "schemaInfo" : {
 "defaultSchema" : true,
 "readUnits" : 50,
 "writeUnits" : 50,
 "storageSize" : 25
 },
 "compartment" : "compartment_name",
 "includeTTL" : false,
 "credentials" : "/home/opc/.oci/config",
 "credentialsProfile" : "DEFAULT",
 "writeUnitsPercent" : 90,
 "overwrite" : true,
 "requestTimeoutMs" : 5000
 },
 "abortOnError" : true,
 "migratorVersion" : "1.5.0"
}
2023-02-15 17:42:46.959 [INFO] creating source from given configuration:
2023-02-15 17:42:48.993 [INFO] source creation completed
2023-02-15 17:42:48.993 [INFO] creating sink from given configuration:
2023-02-15 17:42:49.871 [INFO] sink creation completed
2023-02-15 17:42:49.873 [INFO] creating migrator pipeline
2023-02-15 17:42:49.873 [INFO] migration started
2023-02-15 17:42:49.873 [INFO] [cloud sink] : start loading DDLs
2023-02-15 17:42:49.883 [INFO] [cloud sink] : executing DDL: CREATE TABLE IF NOT EXISTS zips (id STRING, document JSON, PRIMARY KEY(SHARD(id))),limits: [50, 50, 25]
2023-02-15 17:42:54.988 [INFO] [cloud sink] : completed loading DDLs
2023-02-15 17:42:55.081 [INFO] [cloud sink] : start loading records
2023-02-15 17:42:55.082 [DEBUG] [cloud sink] : tableLimits=[50, 50, 25]
2023-02-15 17:42:55.095 [INFO] [OCI OS source] : start parsing JSON records from object: zipssaa.json
2023-02-15 17:44:40.935 [INFO] Processed total of 5000 records
2023-02-15 17:46:32.044 [INFO] Processed total of 10000 records
2023-02-15 17:47:49.844 [INFO] [OCI OS source] : start parsing JSON records from object: zipssab.json
2023-02-15 17:48:23.154 [INFO] Processed total of 15000 records
2023-02-15 17:50:14.265 [INFO] Processed total of 20000 records
2023-02-15 17:52:05.377 [INFO] Processed total of 25000 records
2023-02-15 17:52:49.843 [INFO] [OCI OS source] : start parsing JSON records from object: zipssac.json
2023-02-15 17:53:42.149 [DEBUG] [cloud sink] : [Thread:18] wrote 7356 records
2023-02-15 17:53:42.154 [DEBUG] [cloud sink] : [Thread:17] wrote 7317 records
2023-02-15 17:53:42.176 [DEBUG] [cloud sink] : [Thread:16] wrote 7316 records
2023-02-15 17:53:42.199 [DEBUG] [cloud sink] : [Thread:19] wrote 7364 records
2023-02-15 17:53:42.207 [INFO] migration completed.
Records provided by source=29,353, Records written to sink=29,353, Records failed=0, Records skipped=0.
Elapsed time: 10min 52sec 326ms
Migration completed.
[opc@oci-compute nosql-migrator-1.5.0]$

A propos de la sortie de migration Oracle NoSQL Database de schéma personnalisé

La migration vers MongoDB génère un fichier de sortie avec le schéma JSON de la configuration. Voici un exemple de sortie d'un schéma personnalisé Oracle NoSQL Database Migration avec un exemple de transformation où _id est remplacé par id : 

opc@oci-compute nosql-migrator-1.5.0]$ ./runMigrator --config ./migrator-config-mongodbjson-bucket-src-schema.json --log-level debug
2023-02-22 16:59:32.864 [INFO] Configuration for migration:
{
  "source": {
    "type": "object_storage_oci",
    "format": "mongodb_json",
    "endpoint": "https://objectstorage.oci_region_ID.oraclecloud.com",
    "namespace": "namespace",
    "bucket": "bucket_name",
    "prefix": "source_json_filename_or_prefix",
    "schemaInfo": {
      "schemaObject": "collection_name_ddl"
    },
    "credentials": "/home/opc/.oci/config",
    "credentialsProfile": "DEFAULT"
  },
  "sink": {
    "type": "nosqldb_cloud",
    "endpoint": "https://nosql.oci_region_ID.oci.oraclecloud.com",
    "table": "new_Oracle_NoSQL_Database_Cloud_Service_table_name",
    "schemaInfo": {
      "useSourceSchema": true,
      "readUnits": 50,
      "writeUnits": 50,
      "storageSize": 25
    },
    "compartment": "compartment_name",
    "includeTTL": false,
    "credentials": "/home/opc/.oci/config",
    "credentialsProfile": "DEFAULT",
    "writeUnitsPercent": 90,
    "overwrite": true,
    "requestTimeoutMs": 5000
  },
  "transforms": {
    "renameFields": {
      "_id": "id"
    }
  },
  "abortOnError": true,
  "migratorVersion": "1.5.0"
}
2023-02-22 16:59:32.878 [INFO] creating source from given configuration:
2023-02-22 16:59:36.226 [INFO] source creation completed
2023-02-22 16:59:36.228 [INFO] creating sink from given configuration:
2023-02-22 16:59:37.328 [INFO] sink creation completed
2023-02-22 16:59:37.330 [INFO] creating migrator pipeline
2023-02-22 16:59:37.330 [INFO] migration started
2023-02-22 16:59:37.469 [INFO] [cloud sink] : start loading DDLs
2023-02-22 16:59:37.480 [INFO] [cloud sink] : executing DDL: CREATE TABLE zips ( id string, city string, loc json, pop number, state string, PRIMARY KEY ( id ) ),limits: [50, 50, 25]
2023-02-22 16:59:42.673 [INFO] [cloud sink] : completed loading DDLs
2023-02-22 16:59:42.754 [INFO] [cloud sink] : start loading records
2023-02-22 16:59:42.754 [DEBUG] [cloud sink] : tableLimits=[50, 50, 25]
2023-02-22 16:59:42.756 [INFO] [OCI OS source] : start parsing JSON records from object: zipssaa.json
2023-02-22 17:01:28.592 [INFO] Processed total of 5000 records
2023-02-22 17:03:19.698 [INFO] Processed total of 10000 records
2023-02-22 17:04:37.499 [INFO] [OCI OS source] : start parsing JSON records from object: zipssab.json
2023-02-22 17:05:10.810 [INFO] Processed total of 15000 records
2023-02-22 17:07:01.920 [INFO] Processed total of 20000 records
2023-02-22 17:08:53.031 [INFO] Processed total of 25000 records
2023-02-22 17:09:37.499 [INFO] [OCI OS source] : start parsing JSON records from object: zipssac.json
2023-02-22 17:10:29.807 [DEBUG] [cloud sink] : [Thread:17] wrote 7341 records
2023-02-22 17:10:29.810 [DEBUG] [cloud sink] : [Thread:16] wrote 7326 records
2023-02-22 17:10:29.832 [DEBUG] [cloud sink] : [Thread:18] wrote 7368 records
2023-02-22 17:10:29.854 [DEBUG] [cloud sink] : [Thread:19] wrote 7318 records
2023-02-22 17:10:29.860 [INFO] migration completed.
Records provided by source=29,353, Records written to sink=29,353, Records failed=0, Records skipped=0.
Elapsed time: 10min 52sec 524ms
Migration completed.
[opc@oci-compute nosql-migrator-1.5.0]$