1. ترحيل البيانات من Amazon DynamoDB إلى Oracle NoSQL Database Service
  2. فهم الملفات المستخدمة أثناء الترحيل ونماذج المخرجات

فهم الملفات المستخدمة أثناء الترحيل ونماذج المخرجات

راجع ملفات مثال Amazon DynamoDB ومخرجات الأوامر المستخدمة أثناء ترحيل البيانات.

حول Amazon DynamoDB ملفات JSON المنسقة

عند استيراد البيانات من Amazon DynamoDB إلى Oracle NoSQL Database Cloud Service باستخدام ملفات JSON المنسقة، سيتم إنشاء ملف JSON باستخدام السجلات التي تم ترحيلها. فيما يلي مثال على ملف Amazon DynamoDB بتنسيق JSON مع 4609 سجلات: 

[opc@jc-bastion-phx nosql-migrator-1.5.0]$ head -74 ~/dynamo/moviesnb.json 
{
  "Item": {
    "title": {
      "S": "Fantasia"
    },
    "year": {
      "N": "1940"
    },
    "info": {
      "M": {
        "actors": {
          "L": [
            {
              "S": "Leopold Stokowski"
            },
            {
              "S": "Deems Taylor"
            },
            {
              "S": "Corey Burton"
            }
          ]
        },
        "release_date": {
          "S": "1940-11-13T00:00:00Z"
        },
        "plot": {
          "S": "A collection of animated interpretations of great works of Western classical music."
        },
        "genres": {
          "L": [
            {
              "S": "Animation"
            },
            {
              "S": "Family"
            },
            {
              "S": "Fantasy"
            },
            {
              "S": "Music"
            }
          ]
        },
        "image_url": {
          "S": "http://ia.media.imdb.com/images/M/MV5BMTM2Nzk5ODU5NF5BMl5BanBnXkFtZTcwMjQ1ODYxMQ@@._V1_SX400_.jpg"
        },
        "directors": {
          "L": [
            {
              "S": "James Algar"
            },
            {
              "S": "Samuel Armstrong"
            },
            {
              "S": "Ford Beebe Jr."
            }
          ]
        },
        "rating": {
          "N": "7.8"
        },
        "rank": {
          "N": "3865"
        },
        "running_time_secs": {
          "N": "7500"
        }
      }
    }
  }
}
[opc@oci-compute nosql-migrator-1.5.0]$ grep Item ~/dynamo/moviesnb.json | wc -l4609
[opc@oci-compute nosql-migrator-1.5.0]$

حول ملفات لغة تعريف بيانات Oracle NoSQL Database Cloud Service

عند ترحيل البيانات باستخدام أسلوب مخطط قاعدة البيانات المخصص Oracle NoSQL Database Cloud Service، يجب تعريف مخطط قاعدة البيانات الجديد Oracle NoSQL Database Cloud Service على غرار مخطط الجدول المصدر باستخدام لغة تعريف البيانات (DDL). يجب أن تكون أسماء السمات وأنواعها للجدول الجديد مماثلة لأسماء السمات وأنواعها في الجدول المصدر.

فيما يلي مثال لملف لغة تعريف بيانات مخطط Oracle NoSQL Database Cloud Service. 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/movies.ddl CREATE TABLE IF NOT EXISTS movies (title String,year Number,info JSON, PRIMARY KEY(SHARD(title),year))
[opc@oci-compute nosql-migrator-1.5.0]$

حول ملفات هوية OCI

يحتوي ملف هوية OCI على معلومات حول المستخدم الذي يقوم بالترحيل إلى Oracle NoSQL Database Cloud Service. فيما يلي عينة لملف هوية OCI: 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/.oci/config 
[DEFAULT]
user=ocid1.user.oc1..user_OCID
fingerprint=user_fingerprint
tenancy=ocid1.tenancy.oc1..tenancy_OCID
region=region_ID
key_file=PEM_file_path
[opc@oci-compute nosql-migrator-1.5.0]$

حول ملفات تكوين Amazon DynamoDB

يحتوي ملف تكوين Amazon DynamoDB على معلومات حول حساب Amazon DynamoDB. إليك مثال: 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/.aws/config 
[default]
region=region_ID
[opc@oci-compute nosql-migrator-1.5.0]$

حول ملفات بيانات اعتماد Amazon DynamoDB

تحتوي ملفات بيانات اعتماد Amazon DynamoDB على معلومات حول Amazon DynamoDB. فيما يلي مثال لملف بيانات اعتماد Amazon DynamoDB: 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/.aws/credentials 
[default]
aws_access_key_id = AWS_access_key_ID
aws_secret_access_key = AWS_secret_access_key
[opc@oci-compute nosql-migrator-1.5.0]$

حول مخرجات عمليات ترحيل Oracle NoSQL Database للمخطط الافتراضي

يؤدي الترحيل إلى Amazon DynamoDB إلى إنشاء ملف مخرجات بمخطط JSON للتكوين. إليك مثال على المخرجات من مخطط قاعدة بيانات افتراضي Oracle NoSQL Database Migration: 

[opc@oci-compute nosql-migrator-1.5.0]$ ./runMigrator --config ./migrator-config-dynamodb-bucket_def_schema.json 
2023-12-06 05:53:31.389 [INFO] Configuration for migration:
{
 "source" : {
 "type" : "aws_s3",
 "format" : "dynamodb_json",
 "s3URL" : "https://bucket_URL.s3.us-west-2.amazonaws.com/AWSDynamoDB/xyz/data",
 "credentials" : "/home/opc/.aws/credentials",
 "credentialsProfile" : "default"
 },
 "sink" : {
 "type" : "nosqldb_cloud",
 "endpoint" : "https://endpoint_URL.us-phoenix-1.oci.oraclecloud.com",
 "table" : "movies",
 "schemaInfo" : {
 "defaultSchema" : true,
 "DDBPartitionKey" : "title:String",
 "DDBSortKey" : "year:Number",
 "readUnits" : 50,
 "writeUnits" : 50,
 "storageSize" : 25
 },
 "compartment" : "compartment_name",
 "includeTTL" : false,
 "credentials" : "/home/opc/.oci/config",
 "credentialsProfile" : "DEFAULT",
 "writeUnitsPercent" : 90,
 "overwrite" : true,
 "requestTimeoutMs" : 5000
 },
 "abortOnError" : false,
 "migratorVersion" : "1.5.0"
}
2023-12-06 05:53:31.392 [INFO] creating source from given configuration:
2023-12-06 05:53:36.487 [INFO] source creation completed
2023-12-06 05:53:36.487 [INFO] creating sink from given configuration:
2023-12-06 05:53:40.280 [INFO] sink creation completed
2023-12-06 05:53:40.281 [INFO] creating migrator pipeline
2023-12-06 05:53:40.281 [INFO] migration started
2023-12-06 05:53:40.281 [INFO] [cloud sink] : start loading DDLs
2023-12-06 05:53:40.288 [INFO] [cloud sink] : executing DDL: CREATE TABLE IF NOT EXISTS movies (title String,year Number,document JSON, PRIMARY KEY(SHARD(title),year)),limits: [50, 50, 25]
2023-12-06 05:53:45.494 [INFO] [cloud sink] : completed loading DDLs
2023-12-06 05:53:45.800 [INFO] [cloud sink] : start loading records
2023-12-06 05:53:45.803 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/abc.json.gz
2023-12-06 05:53:51.085 [WARN] failed to write record to the sink table {"title":"Borat: Cultural Learnings of America for Make Benefit Glorious Nation .… Primary key of 88 exceeded the limit of 64
2023-12-06 05:53:55.313 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/cde.json.gz
2023-12-06 05:54:10.201 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/efg.json.gz
2023-12-06 05:54:24.480 [WARN] failed to write record to the sink table {"title":"Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb",.… Primary key of 73 exceeded the limit of 64
2023-12-06 05:54:24.934 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/ghi.json.gz
2023-12-06 05:54:26.479 [WARN] failed to write record to the sink table {"title":"Don't Be a Menace to South Central While Drinking Your Juice in the Ho.… Primary key of 77 exceeded the limit of 64
2023-12-06 05:54:30.403 [WARN] failed to write record to the sink table {"title":"The Chronicles of Narnia: The Lion, the Witch and the Wardrobe","year".… Primary key of 67 exceeded the limit of 64
2023-12-06 05:54:39.401 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/123.json.gz
2023-12-06 05:54:53.579 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/456.json.gz
2023-12-06 05:55:08.177 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/789.json.gz
2023-12-06 05:55:22.603 [WARN] failed to write record to the sink table {"title":"Those Magnificent Men in Their Flying Machines or How I Flew from Lond.… Primary key of 109 exceeded the limit of 64
2023-12-06 05:55:22.688 [INFO] migration completed. Records provided by source=4,609, Records written to sink=4,604, Records failed=5, Records skipped=0. Elapsed time: 1min 42sec 400ms
Migration completed.
[opc@oci-compute nosql-migrator-1.5.0]$

حول مخرجات ترحيل Oracle NoSQL Database لمخطط قاعدة البيانات المخصص

يؤدي الترحيل إلى Amazon DynamoDB إلى إنشاء ملف مخرجات بمخطط JSON للتكوين. فيما يلي مثال لمخرجات مخطط قاعدة بيانات مخصص لترحيل Oracle NoSQL Database: 

opc@oci-compute nosql-migrator-1.5.0]$ ./runMigrator --config ./migrator-config-dynamodb-bucket_custom-schema.json 
2023-12-06 17:38:12.604 [INFO] Configuration for migration:
{
 "source" : {
 "type" : "aws_s3",
 "format" : "dynamodb_json",
 "s3URL" : "https://bucket_URL.amazonaws.com/AWSDynamoDB/xyz/data",
 "credentials" : "/home/opc/.aws/credentials",
 "credentialsProfile" : "default"
 },
 "sink" : {
 "type" : "nosqldb_cloud",
 "endpoint" : "https://endpoint_URL.oci.oraclecloud.com",
 "table" : "movies",
 "schemaInfo" : {
 "schemaPath" : "/home/opc/movies.ddl",
 "readUnits" : 50,
 "writeUnits" : 50,
 "storageSize" : 25
 },
 "compartment" : "compartment_name",
 "includeTTL" : false,
 "credentials" : "/home/opc/.oci/config",
 "credentialsProfile" : "DEFAULT",
 "writeUnitsPercent" : 90,
 "overwrite" : true,
 "requestTimeoutMs" : 5000
 },
 "abortOnError" : false,
 "migratorVersion" : "1.5.0"
}
2023-12-06 17:38:12.680 [INFO] creating source from given configuration:
2023-12-06 17:38:17.488 [INFO] source creation completed
2023-12-06 17:38:17.489 [INFO] creating sink from given configuration:
2023-12-06 17:38:21.587 [INFO] sink creation completed
2023-12-06 17:38:21.588 [INFO] creating migrator pipeline
2023-12-06 17:38:21.588 [INFO] migration started
2023-12-06 17:38:21.588 [INFO] [cloud sink] : start loading DDLs
2023-12-06 17:38:21.595 [INFO] [cloud sink] : executing DDL: CREATE TABLE IF NOT EXISTS movies (title String,year Number,info JSON, PRIMARY KEY(SHARD(title),year)),limits: [50, 50, 25]
2023-12-06 17:38:26.793 [INFO] [cloud sink] : completed loading DDLs
2023-12-06 17:38:27.091 [INFO] [cloud sink] : start loading records
2023-12-06 17:38:27.093 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/abc.json.gz
2023-12-06 17:38:32.385 [WARN] failed to write record to the sink table{"year":2006,"title":"Borat: Cultural Learnings of America for Make Benefit Glor....Primary key of 88 exceeded the limit of 64
2023-12-06 17:38:36.613 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/cde.json.gz
2023-12-06 17:38:51.501 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/efg.json.gz
2023-12-06 17:39:05.777 [WARN] failed to write record to the sink table{"year":1964,"title":"Dr. Strangelove or: How I Learned to Stop Worrying and Lov....Primary key of 73 exceeded the limit of 64
2023-12-06 17:39:06.235 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/ghi.json.gz
2023-12-06 17:39:07.703 [WARN] failed to write record to the sink table{"year":1996,"title":"Don't Be a Menace to South Central While Drinking Your Jui....Primary key of 77 exceeded the limit of 64
2023-12-06 17:39:11.703 [WARN] failed to write record to the sink table{"year":2005,"title":"The Chronicles of Narnia: The Lion, the Witch and the Ward....Primary key of 67 exceeded the limit of 64
2023-12-06 17:39:20.701 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/123.json.gz
2023-12-06 17:39:34.881 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/456.json.gz
2023-12-06 17:39:49.477 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/678.json.gz
2023-12-06 17:40:03.903 [WARN] failed to write record to the sink table{"year":1965,"title":"Those Magnificent Men in Their Flying Machines or How I Fl....Primary key of 109 exceeded the limit of 64
2023-12-06 17:40:03.988 [INFO] migration completed.Records provided by source=4,609, Records written to sink=4,604, Records failed=5, Records skipped=0.
Elapsed time: 1min 42sec 393ms
Migration completed.
[opc@oci-compute nosql-migrator-1.5.0]$