Importing Data into Oracle NoSQL Database

Before You Begin

This 15-minute tutorial shows you how to configure an Internet of Things (IoT) application to import data from an Oracle Storage Cloud Service (OSCS) container into a table in Oracle NoSQL Database. You also learn how to upload comma separated values (csv) files to OSCS.

Background

Your Vision Heavy Machines IoT application and the Yellow Iron simulator application should be running for a minimum of two days to generate and store sensor data in the table in Oracle NoSQL Database. The batch analytics processors need this historical sensor data for training and then predicting the average engine temperature for the next day.

In this tutorial, you invoke an OSCS REST endpoint to upload a compressed csv file into an OSCS container, and then from your IoT application, you import the csv file into the table in Oracle NoSQL Database. The csv file contains historical sensor data for the training and scoring batch analytics processors.

Note: If your applications were running for two days or more, then you can directly move to the next tutorial.

What Do You Need?

• Access to an instance of Oracle Internet of Things Cloud Service with Analytics
• Any REST client plug-in added to your web browser such as RESTlet Client for Chrome
• The authentication URL, the service instance ID for your OSCS account, and the name of your container in OSCS
• The sensor-data.tar.gz file downloaded

Obtain an Authentication Token from OSCS

In this section, you send a GET request to the authentication URL and obtain an authentication token for your OSCS account.

1. In a web browser window, open a REST client plugin, and enter the following information:
   • Method: GET
   • URL: authentication URL for your account
   • Headers:
     • X-Storage-User: service instance ID: your username
     • X-Storage-Pass: your password

   Note: Example of an authentication URL: https://myinstance.storage.oraclecloud.com/auth/v1.0

2. Click SEND, and verify that in the RESPONSE section, a 200 OK message is displayed.
3. Note the value of X-Auth-Token obtained in the response. This is the authentication token.

Upload a Compressed File to an OSCS Container

In this section, you send a request to upload sensor-data.tar.gz to your OSCS container. You also verify the upload.

1. In a web browser window, open a REST client plugin, and enter the following information:
   • Method: PUT
   • URL: https://myinstancename/v1/AUTH_service instance ID/iota-user-uploads?extract-archive=tar.gz"
   • Headers:
     X-Auth-Token: Authenentication token

   Note: In the URL, iota-user-uploads is the name of the OSCS container.

2. In the Body section, select File from the drop-down list, click the Choose a file... button. Browse to the location where you saved the sensor-data.tar.gz file, select it, and click Open. Notice that your file gets added.

   

3. Click SEND, and verify that in the RESPONSE section, a 201 Created message is displayed.
4. To verify that the upload was successful, in the REST client plugin, enter the following information:
   • Method: GET
   • URL: https://myinstancename.storage.oraclecloud.com/v1/service instance ID/iota-user-uploads
   • Headers:
     X-Auth-Token: Authenentication token
5. Click SEND, and verify that in the RESPONSE section, the historical-sensor_data.csv file is listed.,

Import Data into Oracle NoSQL Database

In this section, you create a job in your IoT application that transfers the sensor data from the OSCS container to the Oracle NoSQL Database table.

1. Sign in to your instance of Oracle IoT Cloud Service.
2. On the Oracle IoT Cloud Service home page, click Menu , click Applications, and then click Vision Heavy Machines.
3. On the Vision Heavy Machines menu, click Analytics, and then click Import Data.
4. On the Applications | Vision Heavy Machines page, click Create Data Import Job.
5. On the Create Data Import Job page, enter the following and click Create:
   • Name: import-sensor-data
   • Description: import data from OSCS container to NoSQL table
6. On the import-sensor-data page, enter or select the following values:
   • Object: historical-sensor-data.csv
   • NoSQL Link: sensor-data-nosql
   • Data Format Type: CSV
   • Data Format Profile: Default
   • Header: Header is present in first record
7. On the import-sensor-data page, close to the Type field, click Start. Monitor the Import Progress bar until its value changes to 100%.

   

   If the import stops after transferring a few records, start the job again from where it stopped by entering the Offset value as the number of processed bytes. This value is displayed when the import progress stops.

Want to Learn More?

• About REST URLs
• Container REST Endpoints