Migrate from Oracle NoSQL Database to OCI Object Storage Using Session Token Authentication

This example shows how to use Oracle NoSQL Database Migrator with session token authentication to copy data from Oracle NoSQL Database table to a JSON file in an OCI Object Storage bucket.

Use case

As a developer, you are exploring an option to back up Oracle NoSQL Database table data to OCI Object Storage (OCI OS). You want to use session token-based authentication.

In this demonstration, you will use the OCI Command Line Interface commands (CLI) to create a session token. You will manually create a Migrator configuration file and perform data migration.

Prerequisites

Identify the source and sink for the migration.
- Source: users table in Oracle NoSQL Database.
- Sink: JSON file in the OCI OS bucket
  
  Identify the region endpoint, namespace, bucket, and prefix for OCI OS. For the list of OCI OS service endpoints, see Object Storage Endpoints.
  - endpoint: us-ashburn-1
  - bucket: Migrate_oci
  - prefix: userSession
  - namespace: idhkv1iewjzj
    
    The namespace name for a bucket is the same as it’s tenancy’s namespace and is autogenerated when your tenancy is created. You can get the namespace name as follows:
    - From the Oracle NoSQL Database Cloud Service console, navigate to Storage > Buckets.
    - Select you Compartmentfrom the List Scope and select the bucket. The Bucket Details page displays the name in Namespace parameter.
    If you do not provide an OCI OS namespace name, the Migrator utility uses the default namespace of the tenancy.

Note: Ensure that you have the privileges to write objects in the OCI OS bucket. For more details on setting the policies, see Write to Object Storage.

Generate a session token by following these steps:
- Install and configure OCI CLI. See Quickstart.
- Use one of the following OCI CLI commands to generate a session token. For more details on the available options, see Token-based Authentication for the CLI.
```
#Create a session token using OCI CLI from a web browser:
oci session authenticate --region <region_name> --profile-name <profile_name>
```
```
#Example:
oci session authenticate --region us-ashburn-1 --profile-name SESSIONPROFILE
```
  or
```
#Create a session token using OCI CLI without a web browser:
oci session authenticate --no-browser --region <region_name> --profile-name <profile_name>
```
```
#Example:
oci session authenticate --no-browser --region us-ashburn-1 --profile-name SESSIONPROFILE
```
  In the command above,
  
  region_name: Specifies the region endpoint for your OCI OS. For a list of data regions supported in Oracle NoSQL Database Cloud Service, see Data Regions and Associated Service URLs.
  
  profile_name: Specifies the profile, which the OCI CLI command uses to generate a session token.
  
  The OCI CLI command creates an entry in the OCI config file at $HOME/.oci/config path as shown in the following sample:
```
[SESSIONPROFILE]
fingerprint=f1:e9:b7:e6:25:ff:fe:05:71:be:e8:aa:cc:3d:0d:23
key_file=$HOME/.oci/sessions/SESSIONPROFILE/oci_api_key.pem
tenancy=ocid1.tenancy.oc1..aaaaa ... d6zjq
region=us-ashburn-1
security_token_file=$HOME/.oci/sessions/SESSIONPROFILE/token
```
  The security_token_file points to the path of the session token that you generated using the OCI CLI command above.
  Note:
  - If the profile already exists in the OCI config file, the OCI CLI command overwrites the profile with session-token related configuration while generating the session token.
  - Specify the following in your sink configuration template:
    - The path to the OCI config file in the credentials parameter.
    - The profile used while generating the session token in the credentialsProfile parameter.
```
"credentials" : "$HOME/.oci/config"
"credentialsProfile" : "SESSIONPROFILE"
```
    The Migrator utility automatically fetches the details of the session token generated using the parameters above. If you don't specify the credentials parameter, the Migrator utility looks for the credentials file in the path $HOME/.oci . If you don't specify the credentialsProfile parameter, the Migrator utility uses the default profile name (DEFAULT) from the OCI config file.
  - The session token is valid for 60 minutes. To extend the session duration, you can refresh the session. For details, see Refreshing a Token.

Procedure

To migrate from Oracle NoSQL Database table to a JSON file in the OCI OS bucket:

Prepare the configuration file (in JSON format) with Oracle NoSQL Database source and JSON file in the OCI OS bucket sink. For templates, see Source Configuration Templates and Sink Configuration Templates.

To use the session token authentication to access OCI OS bucket, set the useSessionToken parameter to true in the sink configuration template. Correspondingly, specify the config path in the credentials parameter and the profile name in the credentialsProfile parameter.

{
  "source" : {
    "type" : "nosqldb",
    "storeName" : "kvstore",
    "helperHosts" : ["<hostname>:<port>"],
    "table" : "users",
    "includeTTL" : true,
    "requestTimeoutMs" : 5000
  },
  "sink" : {
    "type" : "object_storage_oci",
    "format" : "json",
    "endpoint" : "us-ashburn-1",
    "namespace" : "idhkv1iewjzj",
    "bucket" : "Migrate_oci",
    "prefix" : "userSession",
    "chunkSize" : 32,
    "compression" : "",
    "useSessionToken" : true,
    "credentials" : "$/home/.oci/config",
    "credentialsProfile" : "SESSIONPROFILE"
  },
  "abortOnError" : true,
  "migratorVersion" : "1.8.0"
}

Open the command prompt and navigate to the directory where you extracted the NoSQL Database Migrator utility.
Run the runMigrator command by passing configuration file option. Use the --config or -c option to pass the configuration file as follows:
```
./runMigrator --config ./migrator-config.json
```
The Migrator utility proceeds with data migration. A sample output is shown below.

With useSessionToken parameter to true, the Migrator utility automatically authenticates using the session token. The Migrator utility copies your data from users table to a JSON file in the OCI OS bucket named Migrate_oci. Check the logs for successful data backup.
```
[INFO] creating source from given configuration:
[INFO] source creation completed
[INFO] creating sink from given configuration:
[INFO] sink creation completed
[INFO] creating migrator pipeline
[INFO] [OCI OS sink] : writing table schema to userSession/Schema/schema.ddl
[INFO] migration started
[INFO] Migration success for source users_6_10. read=2,written=2,failed=0

[INFO] Migration success for source users_1_5. read=3,written=3,failed=0

[INFO] Migration is successful for all the sources.
[INFO] migration completed.
Records provided by source=5, Records written to sink=5, Records failed=0,Records skipped=0.
Elapsed time: 0min 0sec 982ms
Migration completed.
```
Note:

Depending on the chunkSize parameter in the sink configuration template, the Migrator utility splits the source data into several JSON files in the same directory. In this example, the Migrator utility copies data to users_1_5_0.json and users_6_10_0.json files in Migrate_oci/userSession/Data directory.

The source table schema is copied to schema.ddl file in Migrate_oci/userSession/Schema directory.

Verification

To verify your data backup, log in to the Oracle NoSQL Database Cloud Service console. Navigate through the menus, Storage > Object Storage & Archive Storage > Buckets. Access the files from the userSession directory in the Migrate_oci bucket. For the procedure to access the console, see Accessing the Service from the Infrastructure Console