Compreender os Arquivos Usados Durante a Migração e as Saídas de Amostra
Revise os arquivos de exemplo e as saídas de comando a seguir usados durante a migração de dados.
Sobre Arquivos JSON Formatados MongoDB
Você pode gerar arquivos JSON formatados MongoDB de MongoDB para migrar seus dados. Veja um exemplo de arquivo JSON formatado MongoDB:
[opc@oci-compute ~]$ head -3 ~/zips_orig.json
{ "_id" : "01001", "city" : "AGAWAM", "loc" : [ -72.622739, 42.070206 ], "pop" : 15338, "state" : "MA" }
{ "_id" : "01002", "city" : "CUSHMAN", "loc" : [ -72.51564999999999, 42.377017 ], "pop" : 36963, "state" : "MA" }
{ "_id" : "01005", "city" : "BARRE", "loc" : [ -72.10835400000001, 42.409698 ], "pop" : 4546, "state" : "MA" }
[opc@oci-compute nosql-migrator-1.5.0]$
[opc@oci-compute ~]$ cat zips.json | wc -l29353
[opc@oci-compute ~]$
Comando de amostra para dividir o arquivo JSON formatado em MongoDB
[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/.oci/config
[DEFAULT]
user=user_OCID
fingerprint=user_fingerprint
tenancy=tenancy_OCID
region=region_ID
key_file=full_path_to_private_key_pem_file
[opc@oci-compute nosql-migrator-1.5.0]$
Sobre Arquivos de Linguagem de Definição de Dados do Oracle NoSQL Database Cloud Service
Ao migrar dados usando o método de esquema personalizado do Oracle NoSQL Database Cloud Service, o novo esquema de tabela do Oracle NoSQL Database Cloud Service precisa ser definido de forma semelhante ao esquema da tabela de origem usando uma DDL (Data Definition Language). Os nomes e tipos de atributo da nova tabela devem ser iguais aos nomes e tipos de atributo da tabela de origem.
Veja um exemplo de um arquivo de linguagem de definição de dados de esquema de amostra do Oracle NoSQL Database Cloud Service.
[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/movies.ddl CREATE TABLE IF NOT EXISTS movies (title String,year Number,info JSON, PRIMARY KEY(SHARD(title),year))
[opc@oci-compute nosql-migrator-1.5.0]$
Sobre Arquivos de Identidade do OCI
O Arquivo de Identidade do OCI contém informações sobre o usuário que está executando uma migração para o Oracle NoSQL Database Cloud Service. Veja um exemplo de Arquivo de Identidade do OCI:
[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/.oci/config
[DEFAULT]
user=user_OCID
fingerprint=user_fingerprint
tenancy=tenancy_OCID
region=region_ID
key_file=full_file_path_to_PEM_file
[opc@oci-compute nosql-migrator-1.5.0]$
Sobre a Saída de Migrações do Oracle NoSQL Database do Esquema Padrão
A migração para MongoDB gera um arquivo de saída com o esquema JSON da configuração. Veja um exemplo de saída de um esquema padrão do Oracle NoSQL Database Migration:
[opc@oci-compute nosql-migrator-1.5.0]$ ./runMigrator --config ./migrator-config-mongodbjson-bucket.json --log-level debug
2023-02-15 17:42:46.955 [INFO] Configuration for migration:
{
"source" : {
"type" : "object_storage_oci",
"format" : "mongodb_json",
"endpoint" : "https://objectstorage.us-phoenix-1.oraclecloud.com",
"namespace" : "namespace_name",
"bucket" : "bucket_name",
"prefix" : "zipssa",
"credentials" : "/home/opc/.oci/config",
"credentialsProfile" : "DEFAULT"
},
"sink" : {
"type" : "nosqldb_cloud",
"endpoint" : "https://nosql.us-phoenix-1.oci.oraclecloud.com",
"table" : "zips",
"schemaInfo" : {
"defaultSchema" : true,
"readUnits" : 50,
"writeUnits" : 50,
"storageSize" : 25
},
"compartment" : "compartment_name",
"includeTTL" : false,
"credentials" : "/home/opc/.oci/config",
"credentialsProfile" : "DEFAULT",
"writeUnitsPercent" : 90,
"overwrite" : true,
"requestTimeoutMs" : 5000
},
"abortOnError" : true,
"migratorVersion" : "1.5.0"
}
2023-02-15 17:42:46.959 [INFO] creating source from given configuration:
2023-02-15 17:42:48.993 [INFO] source creation completed
2023-02-15 17:42:48.993 [INFO] creating sink from given configuration:
2023-02-15 17:42:49.871 [INFO] sink creation completed
2023-02-15 17:42:49.873 [INFO] creating migrator pipeline
2023-02-15 17:42:49.873 [INFO] migration started
2023-02-15 17:42:49.873 [INFO] [cloud sink] : start loading DDLs
2023-02-15 17:42:49.883 [INFO] [cloud sink] : executing DDL: CREATE TABLE IF NOT EXISTS zips (id STRING, document JSON, PRIMARY KEY(SHARD(id))),limits: [50, 50, 25]
2023-02-15 17:42:54.988 [INFO] [cloud sink] : completed loading DDLs
2023-02-15 17:42:55.081 [INFO] [cloud sink] : start loading records
2023-02-15 17:42:55.082 [DEBUG] [cloud sink] : tableLimits=[50, 50, 25]
2023-02-15 17:42:55.095 [INFO] [OCI OS source] : start parsing JSON records from object: zipssaa.json
2023-02-15 17:44:40.935 [INFO] Processed total of 5000 records
2023-02-15 17:46:32.044 [INFO] Processed total of 10000 records
2023-02-15 17:47:49.844 [INFO] [OCI OS source] : start parsing JSON records from object: zipssab.json
2023-02-15 17:48:23.154 [INFO] Processed total of 15000 records
2023-02-15 17:50:14.265 [INFO] Processed total of 20000 records
2023-02-15 17:52:05.377 [INFO] Processed total of 25000 records
2023-02-15 17:52:49.843 [INFO] [OCI OS source] : start parsing JSON records from object: zipssac.json
2023-02-15 17:53:42.149 [DEBUG] [cloud sink] : [Thread:18] wrote 7356 records
2023-02-15 17:53:42.154 [DEBUG] [cloud sink] : [Thread:17] wrote 7317 records
2023-02-15 17:53:42.176 [DEBUG] [cloud sink] : [Thread:16] wrote 7316 records
2023-02-15 17:53:42.199 [DEBUG] [cloud sink] : [Thread:19] wrote 7364 records
2023-02-15 17:53:42.207 [INFO] migration completed.
Records provided by source=29,353, Records written to sink=29,353, Records failed=0, Records skipped=0.
Elapsed time: 10min 52sec 326ms
Migration completed.
[opc@oci-compute nosql-migrator-1.5.0]$
Sobre a Saída da Migração do Oracle NoSQL Database do Esquema Personalizado
A migração para MongoDB gera um arquivo de saída com o esquema JSON da configuração. Veja um exemplo de saída de um esquema personalizado do Oracle NoSQL Database Migration com uma transformação de amostra em que _id
é alterado para id
:
opc@oci-compute nosql-migrator-1.5.0]$ ./runMigrator --config ./migrator-config-mongodbjson-bucket-src-schema.json --log-level debug
2023-02-22 16:59:32.864 [INFO] Configuration for migration:
{
"source": {
"type": "object_storage_oci",
"format": "mongodb_json",
"endpoint": "https://objectstorage.oci_region_ID.oraclecloud.com",
"namespace": "namespace",
"bucket": "bucket_name",
"prefix": "source_json_filename_or_prefix",
"schemaInfo": {
"schemaObject": "collection_name_ddl"
},
"credentials": "/home/opc/.oci/config",
"credentialsProfile": "DEFAULT"
},
"sink": {
"type": "nosqldb_cloud",
"endpoint": "https://nosql.oci_region_ID.oci.oraclecloud.com",
"table": "new_Oracle_NoSQL_Database_Cloud_Service_table_name",
"schemaInfo": {
"useSourceSchema": true,
"readUnits": 50,
"writeUnits": 50,
"storageSize": 25
},
"compartment": "compartment_name",
"includeTTL": false,
"credentials": "/home/opc/.oci/config",
"credentialsProfile": "DEFAULT",
"writeUnitsPercent": 90,
"overwrite": true,
"requestTimeoutMs": 5000
},
"transforms": {
"renameFields": {
"_id": "id"
}
},
"abortOnError": true,
"migratorVersion": "1.5.0"
}
2023-02-22 16:59:32.878 [INFO] creating source from given configuration:
2023-02-22 16:59:36.226 [INFO] source creation completed
2023-02-22 16:59:36.228 [INFO] creating sink from given configuration:
2023-02-22 16:59:37.328 [INFO] sink creation completed
2023-02-22 16:59:37.330 [INFO] creating migrator pipeline
2023-02-22 16:59:37.330 [INFO] migration started
2023-02-22 16:59:37.469 [INFO] [cloud sink] : start loading DDLs
2023-02-22 16:59:37.480 [INFO] [cloud sink] : executing DDL: CREATE TABLE zips ( id string, city string, loc json, pop number, state string, PRIMARY KEY ( id ) ),limits: [50, 50, 25]
2023-02-22 16:59:42.673 [INFO] [cloud sink] : completed loading DDLs
2023-02-22 16:59:42.754 [INFO] [cloud sink] : start loading records
2023-02-22 16:59:42.754 [DEBUG] [cloud sink] : tableLimits=[50, 50, 25]
2023-02-22 16:59:42.756 [INFO] [OCI OS source] : start parsing JSON records from object: zipssaa.json
2023-02-22 17:01:28.592 [INFO] Processed total of 5000 records
2023-02-22 17:03:19.698 [INFO] Processed total of 10000 records
2023-02-22 17:04:37.499 [INFO] [OCI OS source] : start parsing JSON records from object: zipssab.json
2023-02-22 17:05:10.810 [INFO] Processed total of 15000 records
2023-02-22 17:07:01.920 [INFO] Processed total of 20000 records
2023-02-22 17:08:53.031 [INFO] Processed total of 25000 records
2023-02-22 17:09:37.499 [INFO] [OCI OS source] : start parsing JSON records from object: zipssac.json
2023-02-22 17:10:29.807 [DEBUG] [cloud sink] : [Thread:17] wrote 7341 records
2023-02-22 17:10:29.810 [DEBUG] [cloud sink] : [Thread:16] wrote 7326 records
2023-02-22 17:10:29.832 [DEBUG] [cloud sink] : [Thread:18] wrote 7368 records
2023-02-22 17:10:29.854 [DEBUG] [cloud sink] : [Thread:19] wrote 7318 records
2023-02-22 17:10:29.860 [INFO] migration completed.
Records provided by source=29,353, Records written to sink=29,353, Records failed=0, Records skipped=0.
Elapsed time: 10min 52sec 524ms
Migration completed.
[opc@oci-compute nosql-migrator-1.5.0]$