1. 將資料從 Amazon DynamoDB 移轉至 Oracle NoSQL Database Service
  2. 瞭解移轉期間使用的檔案和範例輸出

瞭解移轉期間使用的檔案和範例輸出

檢閱下列 Amazon DynamoDB 範例檔案，以及資料移轉期間使用的指令輸出。

關於 Amazon DynamoDB 格式化的 JSON 檔案

使用格式化的 JSON 檔案將資料從 Amazon DynamoDB 匯入至 Oracle NoSQL Database Cloud Service 時，將會以移轉的記錄產生一個 JSON 檔案。以下是一個範例 Amazon DynamoDB 格式的 JSON 檔案，其中包含 4,609 筆記錄： 

[opc@jc-bastion-phx nosql-migrator-1.5.0]$ head -74 ~/dynamo/moviesnb.json 
{
  "Item": {
    "title": {
      "S": "Fantasia"
    },
    "year": {
      "N": "1940"
    },
    "info": {
      "M": {
        "actors": {
          "L": [
            {
              "S": "Leopold Stokowski"
            },
            {
              "S": "Deems Taylor"
            },
            {
              "S": "Corey Burton"
            }
          ]
        },
        "release_date": {
          "S": "1940-11-13T00:00:00Z"
        },
        "plot": {
          "S": "A collection of animated interpretations of great works of Western classical music."
        },
        "genres": {
          "L": [
            {
              "S": "Animation"
            },
            {
              "S": "Family"
            },
            {
              "S": "Fantasy"
            },
            {
              "S": "Music"
            }
          ]
        },
        "image_url": {
          "S": "http://ia.media.imdb.com/images/M/MV5BMTM2Nzk5ODU5NF5BMl5BanBnXkFtZTcwMjQ1ODYxMQ@@._V1_SX400_.jpg"
        },
        "directors": {
          "L": [
            {
              "S": "James Algar"
            },
            {
              "S": "Samuel Armstrong"
            },
            {
              "S": "Ford Beebe Jr."
            }
          ]
        },
        "rating": {
          "N": "7.8"
        },
        "rank": {
          "N": "3865"
        },
        "running_time_secs": {
          "N": "7500"
        }
      }
    }
  }
}
[opc@oci-compute nosql-migrator-1.5.0]$ grep Item ~/dynamo/moviesnb.json | wc -l4609
[opc@oci-compute nosql-migrator-1.5.0]$

關於 Oracle NoSQL Database Cloud Service 資料定義語言檔案

使用 Oracle NoSQL Database Cloud Service 自訂綱要方法移轉資料時，必須使用「資料定義語言 (DDL)」來定義新的 Oracle NoSQL Database Cloud Service 表格綱要，與來源表格的綱要類似。新表格的屬性名稱與類型應與來源表格上的屬性名稱與類型相同。

以下是 Oracle NoSQL Database Cloud Service 範例綱要資料定義語言檔案的範例。 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/movies.ddl CREATE TABLE IF NOT EXISTS movies (title String,year Number,info JSON, PRIMARY KEY(SHARD(title),year))
[opc@oci-compute nosql-migrator-1.5.0]$

關於 OCI 識別檔案

OCI 識別檔案包含執行移轉至 Oracle NoSQL Database Cloud Service 之使用者的相關資訊。以下是 OCI 識別檔案範例： 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/.oci/config 
[DEFAULT]
user=ocid1.user.oc1..user_OCID
fingerprint=user_fingerprint
tenancy=ocid1.tenancy.oc1..tenancy_OCID
region=region_ID
key_file=PEM_file_path
[opc@oci-compute nosql-migrator-1.5.0]$

關於 Amazon DynamoDB 組態檔

Amazon DynamoDB 配置檔案包含 Amazon DynamoDB 帳號的相關資訊。以下為範例： 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/.aws/config 
[default]
region=region_ID
[opc@oci-compute nosql-migrator-1.5.0]$

關於 Amazon DynamoDB 證明資料檔案

Amazon DynamoDB 憑證檔案包含 Amazon DynamoDB 的相關資訊。以下是一個範例 Amazon DynamoDB 證明資料檔案： 

[opc@oci-compute nosql-migrator-1.5.0]$ cat /home/opc/.aws/credentials 
[default]
aws_access_key_id = AWS_access_key_ID
aws_secret_access_key = AWS_secret_access_key
[opc@oci-compute nosql-migrator-1.5.0]$

關於預設綱要 Oracle NoSQL Database 移轉輸出

移轉至 Amazon DynamoDB 會產生含有組態之 JSON 綱要的輸出檔案。以下是預設綱要「Oracle NoSQL Database 移轉」的輸出範例： 

[opc@oci-compute nosql-migrator-1.5.0]$ ./runMigrator --config ./migrator-config-dynamodb-bucket_def_schema.json 
2023-12-06 05:53:31.389 [INFO] Configuration for migration:
{
 "source" : {
 "type" : "aws_s3",
 "format" : "dynamodb_json",
 "s3URL" : "https://bucket_URL.s3.us-west-2.amazonaws.com/AWSDynamoDB/xyz/data",
 "credentials" : "/home/opc/.aws/credentials",
 "credentialsProfile" : "default"
 },
 "sink" : {
 "type" : "nosqldb_cloud",
 "endpoint" : "https://endpoint_URL.us-phoenix-1.oci.oraclecloud.com",
 "table" : "movies",
 "schemaInfo" : {
 "defaultSchema" : true,
 "DDBPartitionKey" : "title:String",
 "DDBSortKey" : "year:Number",
 "readUnits" : 50,
 "writeUnits" : 50,
 "storageSize" : 25
 },
 "compartment" : "compartment_name",
 "includeTTL" : false,
 "credentials" : "/home/opc/.oci/config",
 "credentialsProfile" : "DEFAULT",
 "writeUnitsPercent" : 90,
 "overwrite" : true,
 "requestTimeoutMs" : 5000
 },
 "abortOnError" : false,
 "migratorVersion" : "1.5.0"
}
2023-12-06 05:53:31.392 [INFO] creating source from given configuration:
2023-12-06 05:53:36.487 [INFO] source creation completed
2023-12-06 05:53:36.487 [INFO] creating sink from given configuration:
2023-12-06 05:53:40.280 [INFO] sink creation completed
2023-12-06 05:53:40.281 [INFO] creating migrator pipeline
2023-12-06 05:53:40.281 [INFO] migration started
2023-12-06 05:53:40.281 [INFO] [cloud sink] : start loading DDLs
2023-12-06 05:53:40.288 [INFO] [cloud sink] : executing DDL: CREATE TABLE IF NOT EXISTS movies (title String,year Number,document JSON, PRIMARY KEY(SHARD(title),year)),limits: [50, 50, 25]
2023-12-06 05:53:45.494 [INFO] [cloud sink] : completed loading DDLs
2023-12-06 05:53:45.800 [INFO] [cloud sink] : start loading records
2023-12-06 05:53:45.803 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/abc.json.gz
2023-12-06 05:53:51.085 [WARN] failed to write record to the sink table {"title":"Borat: Cultural Learnings of America for Make Benefit Glorious Nation .… Primary key of 88 exceeded the limit of 64
2023-12-06 05:53:55.313 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/cde.json.gz
2023-12-06 05:54:10.201 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/efg.json.gz
2023-12-06 05:54:24.480 [WARN] failed to write record to the sink table {"title":"Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb",.… Primary key of 73 exceeded the limit of 64
2023-12-06 05:54:24.934 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/ghi.json.gz
2023-12-06 05:54:26.479 [WARN] failed to write record to the sink table {"title":"Don't Be a Menace to South Central While Drinking Your Juice in the Ho.… Primary key of 77 exceeded the limit of 64
2023-12-06 05:54:30.403 [WARN] failed to write record to the sink table {"title":"The Chronicles of Narnia: The Lion, the Witch and the Wardrobe","year".… Primary key of 67 exceeded the limit of 64
2023-12-06 05:54:39.401 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/123.json.gz
2023-12-06 05:54:53.579 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/456.json.gz
2023-12-06 05:55:08.177 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/789.json.gz
2023-12-06 05:55:22.603 [WARN] failed to write record to the sink table {"title":"Those Magnificent Men in Their Flying Machines or How I Flew from Lond.… Primary key of 109 exceeded the limit of 64
2023-12-06 05:55:22.688 [INFO] migration completed. Records provided by source=4,609, Records written to sink=4,604, Records failed=5, Records skipped=0. Elapsed time: 1min 42sec 400ms
Migration completed.
[opc@oci-compute nosql-migrator-1.5.0]$

關於自訂綱要 Oracle NoSQL Database 移轉輸出

移轉至 Amazon DynamoDB 會產生含有組態之 JSON 綱要的輸出檔案。以下是自訂綱要「Oracle NoSQL Database 移轉」的輸出範例： 

opc@oci-compute nosql-migrator-1.5.0]$ ./runMigrator --config ./migrator-config-dynamodb-bucket_custom-schema.json 
2023-12-06 17:38:12.604 [INFO] Configuration for migration:
{
 "source" : {
 "type" : "aws_s3",
 "format" : "dynamodb_json",
 "s3URL" : "https://bucket_URL.amazonaws.com/AWSDynamoDB/xyz/data",
 "credentials" : "/home/opc/.aws/credentials",
 "credentialsProfile" : "default"
 },
 "sink" : {
 "type" : "nosqldb_cloud",
 "endpoint" : "https://endpoint_URL.oci.oraclecloud.com",
 "table" : "movies",
 "schemaInfo" : {
 "schemaPath" : "/home/opc/movies.ddl",
 "readUnits" : 50,
 "writeUnits" : 50,
 "storageSize" : 25
 },
 "compartment" : "compartment_name",
 "includeTTL" : false,
 "credentials" : "/home/opc/.oci/config",
 "credentialsProfile" : "DEFAULT",
 "writeUnitsPercent" : 90,
 "overwrite" : true,
 "requestTimeoutMs" : 5000
 },
 "abortOnError" : false,
 "migratorVersion" : "1.5.0"
}
2023-12-06 17:38:12.680 [INFO] creating source from given configuration:
2023-12-06 17:38:17.488 [INFO] source creation completed
2023-12-06 17:38:17.489 [INFO] creating sink from given configuration:
2023-12-06 17:38:21.587 [INFO] sink creation completed
2023-12-06 17:38:21.588 [INFO] creating migrator pipeline
2023-12-06 17:38:21.588 [INFO] migration started
2023-12-06 17:38:21.588 [INFO] [cloud sink] : start loading DDLs
2023-12-06 17:38:21.595 [INFO] [cloud sink] : executing DDL: CREATE TABLE IF NOT EXISTS movies (title String,year Number,info JSON, PRIMARY KEY(SHARD(title),year)),limits: [50, 50, 25]
2023-12-06 17:38:26.793 [INFO] [cloud sink] : completed loading DDLs
2023-12-06 17:38:27.091 [INFO] [cloud sink] : start loading records
2023-12-06 17:38:27.093 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/abc.json.gz
2023-12-06 17:38:32.385 [WARN] failed to write record to the sink table{"year":2006,"title":"Borat: Cultural Learnings of America for Make Benefit Glor....Primary key of 88 exceeded the limit of 64
2023-12-06 17:38:36.613 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/cde.json.gz
2023-12-06 17:38:51.501 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/efg.json.gz
2023-12-06 17:39:05.777 [WARN] failed to write record to the sink table{"year":1964,"title":"Dr. Strangelove or: How I Learned to Stop Worrying and Lov....Primary key of 73 exceeded the limit of 64
2023-12-06 17:39:06.235 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/ghi.json.gz
2023-12-06 17:39:07.703 [WARN] failed to write record to the sink table{"year":1996,"title":"Don't Be a Menace to South Central While Drinking Your Jui....Primary key of 77 exceeded the limit of 64
2023-12-06 17:39:11.703 [WARN] failed to write record to the sink table{"year":2005,"title":"The Chronicles of Narnia: The Lion, the Witch and the Ward....Primary key of 67 exceeded the limit of 64
2023-12-06 17:39:20.701 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/123.json.gz
2023-12-06 17:39:34.881 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/456.json.gz
2023-12-06 17:39:49.477 [INFO] [DDB S3 source] : start parsing JSON records from object: AWSDynamoDB/xyz/data/678.json.gz
2023-12-06 17:40:03.903 [WARN] failed to write record to the sink table{"year":1965,"title":"Those Magnificent Men in Their Flying Machines or How I Fl....Primary key of 109 exceeded the limit of 64
2023-12-06 17:40:03.988 [INFO] migration completed.Records provided by source=4,609, Records written to sink=4,604, Records failed=5, Records skipped=0.
Elapsed time: 1min 42sec 393ms
Migration completed.
[opc@oci-compute nosql-migrator-1.5.0]$