1. Migración de datos de Amazon DynamoDB a Oracle NoSQL Database Service
  2. Descripción de los archivos utilizados durante la migración y las salidas de ejemplo

Descripción de los archivos utilizados durante la migración y las salidas de ejemplo

Revise los siguientes archivos de ejemplo y salidas de comandos de Amazon DynamoDB utilizados durante la migración de datos.

Acerca de los archivos JSON con formato de Amazon DynamoDB

Al importar datos de Amazon DynamoDB a Oracle NoSQL Database Cloud Service mediante archivos JSON con formato, se generará un archivo JSON con los registros migrados. A continuación, se muestra un ejemplo de archivo JSON con formato Amazon DynamoDB con 4.609 registros: 

[opc@jc-bastion-phx nosql-migrator-1.5.0]$ head -74 ~/dynamo/moviesnb.json 
{
  "Item": {
    "title": {
      "S": "Fantasia"
    },
    "year": {
      "N": "1940"
    },
    "info": {
      "M": {
        "actors": {
          "L": [
            {
              "S": "Leopold Stokowski"
            },
            {
              "S": "Deems Taylor"
            },
            {
              "S": "Corey Burton"
            }
          ]
        },
        "release_date": {
          "S": "1940-11-13T00:00:00Z"
        },
        "plot": {
          "S": "A collection of animated interpretations of great works of Western classical music."
        },
        "genres": {
          "L": [
            {
              "S": "Animation"
            },
            {
              "S": "Family"
            },
            {
              "S": "Fantasy"
            },
            {
              "S": "Music"
            }
          ]
        },
        "image_url": {
          "S": "http://ia.media.imdb.com/images/M/MV5BMTM2Nzk5ODU5NF5BMl5BanBnXkFtZTcwMjQ1ODYxMQ@@._V1_SX400_.jpg"
        },
        "directors": {
          "L": [
            {
              "S": "James Algar"
            },
            {
              "S": "Samuel Armstrong"
            },
            {
              "S": "Ford Beebe Jr."
            }
          ]
        },
        "rating": {
          "N": "7.8"
        },
        "rank": {
          "N": "3865"
        },
        "running_time_secs": {
          "N": "7500"
        }
      }
    }
  }
}
[opc@oci-compute nosql-migrator-1.5.0]$ grep Item ~/dynamo/moviesnb.json | wc -l4609
[opc@oci-compute nosql-migrator-1.5.0]$

Acerca de los archivos de lenguaje de definición de datos de Oracle NoSQL Database Cloud Service

Al migrar datos mediante el método de esquema personalizado de Oracle NoSQL Database Cloud Service, el nuevo esquema de tabla de Oracle NoSQL Database Cloud Service se debe definir de forma similar al esquema de la tabla de origen mediante un lenguaje de definición de datos (DDL). Los nombres y tipos de atributo de la nueva tabla deben ser los mismos que los nombres y tipos de atributo de la tabla de origen.

A continuación, se muestra un ejemplo de un archivo de lenguaje de definición de datos de esquema de ejemplo de Oracle NoSQL Database Cloud Service. 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/movies.ddl CREATE TABLE IF NOT EXISTS movies (title String,year Number,info JSON, PRIMARY KEY(SHARD(title),year))
[opc@oci-compute nosql-migrator-1.5.0]$

Acerca de los archivos de identidad de OCI

El archivo de identidad de OCI contiene información sobre el usuario que realiza una migración a Oracle NoSQL Database Cloud Service. A continuación, se muestra un archivo de identidad de OCI de ejemplo: 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/.oci/config 
[DEFAULT]
user=ocid1.user.oc1..user_OCID
fingerprint=user_fingerprint
tenancy=ocid1.tenancy.oc1..tenancy_OCID
region=region_ID
key_file=PEM_file_path
[opc@oci-compute nosql-migrator-1.5.0]$

Acerca de los archivos de configuración de Amazon DynamoDB

El archivo de configuración de Amazon DynamoDB contiene información sobre la cuenta de Amazon DynamoDB. A continuación le mostramos un ejemplo: 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/.aws/config 
[default]
region=region_ID
[opc@oci-compute nosql-migrator-1.5.0]$

Acerca de los archivos de credenciales de Amazon DynamoDB

Los archivos de credenciales de Amazon DynamoDB contienen información sobre Amazon DynamoDB. A continuación, se muestra un ejemplo del archivo de credenciales Amazon DynamoDB: 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/.aws/credentials 
[default]
aws_access_key_id = AWS_access_key_ID
aws_secret_access_key = AWS_secret_access_key
[opc@oci-compute nosql-migrator-1.5.0]$

Acerca de la salida de migraciones de Oracle NoSQL Database de esquema por defecto

La migración a Amazon DynamoDB genera un archivo de salida con el esquema JSON de la configuración. A continuación se muestra un ejemplo de salida de una migración de Oracle NoSQL Database de esquema por defecto: 

[opc@oci-compute nosql-migrator-1.5.0]$ ./runMigrator --config ./migrator-config-dynamodb-bucket_def_schema.json 
2023-12-06 05:53:31.389 [INFO] Configuration for migration:
{
 "source" : {
 "type" : "aws_s3",
 "format" : "dynamodb_json",
 "s3URL" : "https://bucket_URL.s3.us-west-2.amazonaws.com/AWSDynamoDB/xyz/data",
 "credentials" : "/home/opc/.aws/credentials",
 "credentialsProfile" : "default"
 },
 "sink" : {
 "type" : "nosqldb_cloud",
 "endpoint" : "https://endpoint_URL.us-phoenix-1.oci.oraclecloud.com",
 "table" : "movies",
 "schemaInfo" : {
 "defaultSchema" : true,
 "DDBPartitionKey" : "title:String",
 "DDBSortKey" : "year:Number",
 "readUnits" : 50,
 "writeUnits" : 50,
 "storageSize" : 25
 },
 "compartment" : "compartment_name",
 "includeTTL" : false,
 "credentials" : "/home/opc/.oci/config",
 "credentialsProfile" : "DEFAULT",
 "writeUnitsPercent" : 90,
 "overwrite" : true,
 "requestTimeoutMs" : 5000
 },
 "abortOnError" : false,
 "migratorVersion" : "1.5.0"
}
2023-12-06 05:53:31.392 [INFO] creating source from given configuration:
2023-12-06 05:53:36.487 [INFO] source creation completed
2023-12-06 05:53:36.487 [INFO] creating sink from given configuration:
2023-12-06 05:53:40.280 [INFO] sink creation completed
2023-12-06 05:53:40.281 [INFO] creating migrator pipeline
2023-12-06 05:53:40.281 [INFO] migration started
2023-12-06 05:53:40.281 [INFO] [cloud sink] : start loading DDLs
2023-12-06 05:53:40.288 [INFO] [cloud sink] : executing DDL: CREATE TABLE IF NOT EXISTS movies (title String,year Number,document JSON, PRIMARY KEY(SHARD(title),year)),limits: [50, 50, 25]
2023-12-06 05:53:45.494 [INFO] [cloud sink] : completed loading DDLs
2023-12-06 05:53:45.800 [INFO] [cloud sink] : start loading records
2023-12-06 05:53:45.803 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/abc.json.gz
2023-12-06 05:53:51.085 [WARN] failed to write record to the sink table {"title":"Borat: Cultural Learnings of America for Make Benefit Glorious Nation .… Primary key of 88 exceeded the limit of 64
2023-12-06 05:53:55.313 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/cde.json.gz
2023-12-06 05:54:10.201 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/efg.json.gz
2023-12-06 05:54:24.480 [WARN] failed to write record to the sink table {"title":"Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb",.… Primary key of 73 exceeded the limit of 64
2023-12-06 05:54:24.934 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/ghi.json.gz
2023-12-06 05:54:26.479 [WARN] failed to write record to the sink table {"title":"Don't Be a Menace to South Central While Drinking Your Juice in the Ho.… Primary key of 77 exceeded the limit of 64
2023-12-06 05:54:30.403 [WARN] failed to write record to the sink table {"title":"The Chronicles of Narnia: The Lion, the Witch and the Wardrobe","year".… Primary key of 67 exceeded the limit of 64
2023-12-06 05:54:39.401 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/123.json.gz
2023-12-06 05:54:53.579 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/456.json.gz
2023-12-06 05:55:08.177 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/789.json.gz
2023-12-06 05:55:22.603 [WARN] failed to write record to the sink table {"title":"Those Magnificent Men in Their Flying Machines or How I Flew from Lond.… Primary key of 109 exceeded the limit of 64
2023-12-06 05:55:22.688 [INFO] migration completed. Records provided by source=4,609, Records written to sink=4,604, Records failed=5, Records skipped=0. Elapsed time: 1min 42sec 400ms
Migration completed.
[opc@oci-compute nosql-migrator-1.5.0]$

Acerca de la salida de migración de Oracle NoSQL Database de esquema personalizado

La migración a Amazon DynamoDB genera un archivo de salida con el esquema JSON de la configuración. A continuación se muestra un ejemplo de salida de una migración de Oracle NoSQL Database de esquema personalizado: 

opc@oci-compute nosql-migrator-1.5.0]$ ./runMigrator --config ./migrator-config-dynamodb-bucket_custom-schema.json 
2023-12-06 17:38:12.604 [INFO] Configuration for migration:
{
 "source" : {
 "type" : "aws_s3",
 "format" : "dynamodb_json",
 "s3URL" : "https://bucket_URL.amazonaws.com/AWSDynamoDB/xyz/data",
 "credentials" : "/home/opc/.aws/credentials",
 "credentialsProfile" : "default"
 },
 "sink" : {
 "type" : "nosqldb_cloud",
 "endpoint" : "https://endpoint_URL.oci.oraclecloud.com",
 "table" : "movies",
 "schemaInfo" : {
 "schemaPath" : "/home/opc/movies.ddl",
 "readUnits" : 50,
 "writeUnits" : 50,
 "storageSize" : 25
 },
 "compartment" : "compartment_name",
 "includeTTL" : false,
 "credentials" : "/home/opc/.oci/config",
 "credentialsProfile" : "DEFAULT",
 "writeUnitsPercent" : 90,
 "overwrite" : true,
 "requestTimeoutMs" : 5000
 },
 "abortOnError" : false,
 "migratorVersion" : "1.5.0"
}
2023-12-06 17:38:12.680 [INFO] creating source from given configuration:
2023-12-06 17:38:17.488 [INFO] source creation completed
2023-12-06 17:38:17.489 [INFO] creating sink from given configuration:
2023-12-06 17:38:21.587 [INFO] sink creation completed
2023-12-06 17:38:21.588 [INFO] creating migrator pipeline
2023-12-06 17:38:21.588 [INFO] migration started
2023-12-06 17:38:21.588 [INFO] [cloud sink] : start loading DDLs
2023-12-06 17:38:21.595 [INFO] [cloud sink] : executing DDL: CREATE TABLE IF NOT EXISTS movies (title String,year Number,info JSON, PRIMARY KEY(SHARD(title),year)),limits: [50, 50, 25]
2023-12-06 17:38:26.793 [INFO] [cloud sink] : completed loading DDLs
2023-12-06 17:38:27.091 [INFO] [cloud sink] : start loading records
2023-12-06 17:38:27.093 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/abc.json.gz
2023-12-06 17:38:32.385 [WARN] failed to write record to the sink table{"year":2006,"title":"Borat: Cultural Learnings of America for Make Benefit Glor....Primary key of 88 exceeded the limit of 64
2023-12-06 17:38:36.613 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/cde.json.gz
2023-12-06 17:38:51.501 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/efg.json.gz
2023-12-06 17:39:05.777 [WARN] failed to write record to the sink table{"year":1964,"title":"Dr. Strangelove or: How I Learned to Stop Worrying and Lov....Primary key of 73 exceeded the limit of 64
2023-12-06 17:39:06.235 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/ghi.json.gz
2023-12-06 17:39:07.703 [WARN] failed to write record to the sink table{"year":1996,"title":"Don't Be a Menace to South Central While Drinking Your Jui....Primary key of 77 exceeded the limit of 64
2023-12-06 17:39:11.703 [WARN] failed to write record to the sink table{"year":2005,"title":"The Chronicles of Narnia: The Lion, the Witch and the Ward....Primary key of 67 exceeded the limit of 64
2023-12-06 17:39:20.701 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/123.json.gz
2023-12-06 17:39:34.881 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/456.json.gz
2023-12-06 17:39:49.477 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/678.json.gz
2023-12-06 17:40:03.903 [WARN] failed to write record to the sink table{"year":1965,"title":"Those Magnificent Men in Their Flying Machines or How I Fl....Primary key of 109 exceeded the limit of 64
2023-12-06 17:40:03.988 [INFO] migration completed.Records provided by source=4,609, Records written to sink=4,604, Records failed=5, Records skipped=0.
Elapsed time: 1min 42sec 393ms
Migration completed.
[opc@oci-compute nosql-migrator-1.5.0]$