About Migrating Data from Amazon DynamoDB to Oracle NoSQL Database Cloud Service

Your organization has decided to migrate data from Amazon DynamoDB to Oracle NoSQL Database Cloud Service to keep data in a non-relational database and maintain high availability and scalability. This playbook walks you through the steps to migrate data from Amazon DynamoDB to Oracle NoSQL Database Cloud Service using the Oracle NoSQL Database Migrator.

Architecture

In this architecture, the Oracle NoSQL Database Migrator is installed on an OCI Compute Instance. We recommend using an OCI Always Free Compute instance to reduce costs. Amazon DynamoDB formatted JSON files are migrated from Amazon Simple Storage Service (Amazon S3) using an OCI Compute instance running the Oracle NoSQL Database Migrator.

Note:

When migrating data, Oracle reserves and manages the resources to meet your requirements and provisions capacity. Capacity is specified using read and write units for throughput and GB for storage units.

The following diagram illustrates this architecture.

Description of aws-dynamodb-nosql-architecture.png follows

Description of the illustration aws-dynamodb-nosql-architecture.png

aws-dynamodb-nosql-architecture.zip

This architecture supports the following components:

Region
An Oracle Cloud Infrastructure region is a localized geographic area that contains one or more data centers, called availability domains. Regions are independent of other regions, and vast distances can separate them (across countries or even continents).
Availability domain
Availability domains are standalone, independent data centers within a region. The physical resources in each availability domain are isolated from the resources in the other availability domains, which provides fault tolerance. Availability domains don’t share infrastructure such as power or cooling, or the internal availability domain network. So, a failure at one availability domain is unlikely to affect the other availability domains in the region.
Fault domain
A fault domain is a grouping of hardware and infrastructure within an availability domain. Each availability domain has three fault domains with independent power and hardware. When you distribute resources across multiple fault domains, your applications can tolerate physical server failure, system maintenance, and power failures inside a fault domain.
Virtual cloud network (VCN) and subnet
A VCN is a customizable, software-defined network that you set up in an Oracle Cloud Infrastructure region. Like traditional data center networks, VCNs give you complete control over your network environment. A VCN can have multiple non-overlapping CIDR blocks that you can change after you create the VCN. You can segment a VCN into subnets, which can be scoped to a region or to an availability domain. Each subnet consists of a contiguous range of addresses that don't overlap with the other subnets in the VCN. You can change the size of a subnet after creation. A subnet can be public or private.
Internet gateway
The internet gateway allows traffic between the public subnets in a VCN and the public internet.
Service gateway
The service gateway provides access from a VCN to other services, such as Oracle Cloud Infrastructure Object Storage. The traffic from the VCN to the Oracle service travels over the Oracle network fabric and never traverses the internet.
Identity and Access Management (IAM)
Oracle Cloud Infrastructure Identity and Access Management (IAM) is the access control plane for Oracle Cloud Infrastructure (OCI) and Oracle Cloud Applications. The IAM API and the user interface enable you to manage identity domains and the resources within the identity domain. Each OCI IAM identity domain represents a standalone identity and access management solution or a different user population.
Oracle NoSQL Database Cloud Service
Oracle NoSQL Database Cloud Service makes it easy for developers to build applications using document, columnar and key-value database models, delivering predictable single digit millisecond response times with data replication for high availability. The service offers ACID transactions, serverless scaling, comprehensive security, and low pay-per-use pricing for both on-demand and provisioned capacity modes, including 100% compatibility with on-premises Oracle NoSQL Database.

Before You Begin

Before you begin, check the versions of major software components used in this setup, download the required software package, and review the product documentation for later reference.

Review Requirements

Ensure the OCI Compute Instance running Oracle NoSQL Database Migrator has at least 1 OCPU with 16GB of RAM running Oracle Linux 8.
The OCI Compute Instance must be running Oracle NoSQL Database Migrator version 1.5.0 and Java 11.
Intermediate storage must include Amazon S3 Object Storage Bucket.

Download the Oracle NoSQL Database Migrator 1.5.0 binaries (V1033765-01.zip) under "Oracle NoSQL Database Migrator and Analytics Integrator":

Oracle NoSQL Database Migrator and Analytics Integrator

Review Documentation

These additional resources are helpful for context, details, and reference for your data migration.

Review Oracle NoSQL Database Cloud Service Resources

Review Amazon Resources

About Required Services and Roles

This solution requires the following products and roles:

Oracle Cloud Infrastructure Identity and Access Management
Oracle NoSQL Database Cloud Service
Oracle Cloud Infrastructure Compute
Amazon Simple Storage Service (Amazon S3)
Amazon DynamoDB
Amazon Identity and Access Management

These are the roles needed for each service.

Service Name: Role	Required to...
Oracle Cloud Infrastructure Identity and Access Management: `user`	Create API key to access OCI to create tables and import data to Oracle NoSQL Database Cloud Service.
Oracle NoSQL Database Cloud Service: `admin`	Create tables and import data.
Oracle Cloud Infrastructure Compute: `admin`	Create OCI compute instance to run Oracle NoSQL Data Migrator.
Oracle Cloud Infrastructure Compute VM: `opc`	Install and run Oracle NoSQL Data Migrator.
Amazon Simple Storage Service Bucket: `user`	Upload files and read files from Amazon Simple Storage Service.
Amazon DynamoDB: `Admin`	Explore DynamoDB tables and export tables to Amazon Simple Storage Service Bucket.
Amazon Identity and Access Management: `user`	Create access key to read files from Amazon Simple Storage Service Bucket.

See Oracle Products, Solutions, and Services to get what you need.

Considerations for Migration

Before you begin your migration, review these considerations.

When importing data from Amazon DynamoDB to Oracle NoSQL Database Cloud Service using DynamoDB formatted JSON files, you must specify dynamodb_json as the data format in the Oracle NoSQL Database Migrator configuration JSON file. If the target Oracle NoSQL Database Cloud Service table is using the default schema, the parameter DDBPartitionKey must be included in the sink definition. The parameter DDSortKey is optional.
The partition key and sort key in Amazon DynamoDB have larger character limits than their equivalent keys in Oracle NoSQL Database Cloud Service. In Amazon DynamoDB the partition key and sort key have limits of 2048 and 1024 bytes respectively, while in Oracle NoSQL Database Cloud Service these keys are limited to 64 bytes. Therefore, if one of the records being imported has a key longer than the limit in Oracle NoSQL Database Cloud Service, importing that record will fail with an error similar to: “primary or sort key of 88 exceeded the limit of 64”. For more information about limits and data types, see "Oracle NoSQL Documentation – Mapping of DynamoDB types to Oracle NoSQL Types" in the Review Documentation section.
Note the name and type for the partition key and sort key columns in the Amazon DynamoDB table before beginning the migration.