1. Using Oracle Internet of Things Asset Monitoring Cloud Service
  2. Work with Your Assets
  3. Import Historical Asset Data

Import Historical Asset Data

If you have your pre-deployment device data in an external system, you can choose to import historical sensor and metric data into Oracle IoT Asset Monitoring Cloud Service and use the data to train your analytics artifacts, such as anomalies and predictions.

Importing historical data is useful for cold start scenarios where you don't have training data already available in Oracle IoT Asset Monitoring Cloud Service. You can also import data for proof-of-concept demonstrations, so that you can use the imported data to train your anomalies, predictions, and trends.

Note:

Pattern anomalies do not support imported historical data. To create pattern anomalies, you must generate training data in Oracle IoT Asset Monitoring Cloud Service.

Use the following steps to import historical sensor and metric data into Oracle IoT Asset Monitoring Cloud Service:

  1. In Oracle IoT Asset Monitoring Cloud Service, create and export an asset data template for your asset data.
  2. Populate the asset data template with sensor and metric data from the external system.
  3. Import the asset data into Oracle IoT Asset Monitoring Cloud Service. The data is available for training after validation and processing.

Export the Asset Data Template

The asset data template defines the schema for your asset data import. It includes fields for asset names and IDs, timestamps, sensor attributes, and metrics.

You should already have your asset type, sensor attributes, metrics, and assets created in Oracle IoT Asset Monitoring Cloud Service before creating the asset data template.

Note:

You can choose to import asset types, assets, and other entities into a new instance by importing a previously exported organization. See Export and Import Organizations for more information.
  1. Log in to Oracle IoT Asset Monitoring Cloud Service as an administrator.
    Only administrators have the privilege to export asset data templates from Oracle IoT Asset Monitoring Cloud Service.
  2. Click Menu (Menu icon), and then click Design Center.
  3. Select Asset Inventory from the Design Center sub-menu.
  4. Click the Asset Inventory Menu Asset Inventory Menu Icon and select Export Asset Data Template.
  5. Select the Entity Type (Asset Type) for the asset data template.
    For example, you may want to create an asset data template for forklift assets.
    The existing sensor attributes and metrics for the asset type appear in a tree-like structure under the asset type. If the asset type contains sub-asset types or associated asset types, the attributes for the sub-asset type are also shown.
  6. Deselect the sensor attributes and metrics that you do not wish to include in your asset data template.
    For example, if you do not have historical data for a particular attribute in your external system, you can exclude it from the asset data template.

    Export Asset Data Template Dialog

    You can click on the Sensor Attributes and Metrics nodes to expand or collapse them.
  7. Click Export.
    Save the exported csv (comma separated value) file to your local storage.
    The exported csv file contains the following fields:
    • ora_entity_type_name: The entity type (asset type) for which you created the template.
    • ora_entity_name: Name of the entity (asset). You must specify at least one of ora_entity_name, ora_entity_id, and ora_external_entity_id for each row of data that you populate in the asset data template.
    • ora_entity_id: Identifier (ID) of the entity (asset). You must specify at least one of ora_entity_name, ora_entity_id, and ora_external_entity_id for each row of data that you populate in the asset data template.
    • ora_external_entity_id: External Identifier of the entity (asset). An external identified would be the identifier of an imported asset in the external system from which it was imported. For example, the asset identifier of an asset in Oracle Fusion Cloud Maintenance.

      You must specify at least one of ora_entity_name, ora_entity_id, and ora_external_entity_id for each row of data that you populate in the asset data template.

    • ora_event_time: The event time against which the telemetry data is being reported. The epoch long time format and ISO 8061 format are supported.

      yyyy-MM-ddTHH:mm:ss.SSSZ. For example, 2007-07-16T19:20:30.45Z.

    • ora_sensor.name fields: The sensor attribute values for the attributes that you included in your template.
    • ora_metric.name fields: The metric values for the attributes that you included in your template.
The following example shows a sample, populated csv file segment. The user populates the asset data template with temperature and pressure sensor data for the Env_Sensor2 asset. Note that only one of ora_entity_name, ora_entity_id, and ora_external_entity_id is required for each row of data that you populate in the asset data template.
CSV File Segment: Described in text.

Import Asset Data for Sensors and Metrics

Once you have populated your asset data template with asset data, you can import the csv file, or a zip file containing one or more csv files, into Oracle IoT Asset Monitoring Cloud Service.

  1. Log in to Oracle IoT Asset Monitoring Cloud Service as an administrator.
    Only administrators have the privilege to import historical asset data into Oracle IoT Asset Monitoring Cloud Service.
  2. Click Menu (Menu icon), and then click Design Center.
  3. Select Asset Inventory from the Design Center sub-menu.
  4. Click the Asset Inventory Menu Asset Inventory Menu Icon and select Import Asset Data.
  5. Enter an Import Task Description to help you identify the import task later.
  6. Select the Entity Type (Asset Type) for the asset data that you are importing.
  7. Optionally change the number of data lines under Rejection Threshold.
    The Rejection Threshold specifies the threshold number of erroneous data lines in the imported file before the import is rejected. Typically, users populate the asset data template using an automated process, so if a certain number of data lines are erroneous, it is very likely that the rest of the lines are erroneous too. Oracle IoT Asset Monitoring Cloud Service halts the import once the specified threshold is reached.
    A data line may be rejected for various reasons. For example, the data line might have missing or incorrect entity information.
  8. (Optional) Deselect Review errors before processing if you wish the file to be auto processed or rejected without prompting you to review the errors, if any.
    Review errors before processing lets you review the error details in case there are validation errors in the imported file. If the number of erroneous rows are below the threshold, you can choose to process or reject the remaining rows.
    If you deselect Review Errors Before Processing, then if the number of errors is below the rejection threshold, the import is auto-processed. Else the import is rejected.
  9. Click Choose File and select the csv file or zip file to be imported.
    The maximum file size cannot exceed 150 MB. You can choose to compress multiple csv files in a single zip file.
  10. Click Continue.
    A notification appears confirming that the upload request was sent.
    The imported file is next validated and processed after which the imported data is available for training. If you are working in other areas of the application, you get periodic notifications about the status of the import. You are also notified if data errors are found in the import file.
  11. (Optional) Click the Data Import Log tab to monitor the status of the import.
    The Initiated column reflects the time when the import request was initiated.
    The Status column reflects the current status of the import. The status column may reflect one of the following:
    • Pending Validation: This is the status just after you have initiated the import.
    • Validating Data: The data validation is in progress for the imported file.
    • Processing Trained Data: The data processing stage follows the validation stage.
    • Historical Data Available: The training data is available in the system.
    • Some Data Errors Found: Indicates that there were data errors while processing. You can click the Show Details icon Show Details icon to view the error details. Click Action Required Action Required icon to accept or reject the remaining rows.
    • Request Rejected: The reason for request rejection is enumerated. For example, the asset data template had a bad schema, or the number of erroneous rows exceeded the rejection threshold. You can click the Show Details icon Show Details icon to view the rejection details.
    1. Click the Show Details icon Show Details icon to view details about the import, such as any error or rejection details.
    2. Click OK.
    3. Click Action Required Action Required icon to complete any user-pending tasks, such as the following:
      • Process historical data ignoring errors
      • Reject Request
      If you have chosen to review errors before processing, and if the number of erroneous rows are below the rejection threshold, then you can choose to process the historical data in the remaining rows. You can also choose to reject the import request.
    4. Click OK.

After you have imported historical sensor and metric data, you can use the data to train your analytics artifacts, such as anomalies and predictions.

Note that historical sensor data will not show up in the digital twin view of your asset. So, you cannot see data plots for imported historical data.