Ingesting data using the Stream API

The Streaming API is used to ingest data directly from source systems into Oracle Audience Segmentation.

In this topic:

• Ingesting data using the Stream API
• Introduction
• Use cases
• Explore the API
• API Performance
• Request URL
• Request body details
• Response details
• Examples
• Date and timestamp formats
• Data types
• Character limits
• View Summary and Errors

Introduction

The Stream API ingests data into Oracle Audience Segmentation. After running data warehouse jobs, the data can then be used.

Note: The Near real time ingestion API ingests data directly into Oracle Audience Segmentation's data warehouse data objects. This is the preferred option if you need to action your data faster, as data is ingested in real time. Learn more.

Use cases

• Send data updates directly into Oracle Audience Segmentation from your source system. If your source system allows your customers to update their personal profiles, you can send data updates each time your customers save their profile changes.

• After customers register for an event, send customer information directly into Oracle Audience Segmentation.

• After a customer makes a purchase, send product/order information directly into Oracle Audience Segmentation.

Explore the API

The embedded I/O doc below enables you to explore the API. The I/O doc explains the parameters for each method and provides templates for your calls. You cannot make live API calls from the tool, however. Note that the embedded doc includes reference to APIs that are available only to Unity customers. APIs available to Oracle Audience Segmentation customers are described in this documentation.

Open the link below in a new tab to see the I/O doc in a three-pane format.

https://oracleunityapis.docs.apiary.io

API Performance

• The maximum request size is 1 MB.

• Multiple records, up to a thousand, can be included in the request.

• Processing Time for streaming ingest API calls are usually sub-second.

Request URL

POST /api-stream/v1/<tenantAccessKey>/data/entities

Request body details

In the request body, include the data you want streamed directly into Oracle Audience Segmentation. The format is:

{
  "<Unity_Data_Object_resourceName>": [{
    "<attribute>": value,
    "<attribute>": value,
    ...
}

Where...

• <Data_Object_resourceName> represents the existing resource name in Oracle Audience Segmentation to ingest data. You can retrieve your data object's resource name using the Retrieve all data objects endpoint. The resource name is returned in the resourceName property.

  Example: For the "customer" data object, the resourceName is "customers". If you are ingesting data into the "event" data object, the resourceName is "events".

• Along with <Data_Object_resourceName>, your payload must include the sourceAttribute for any declared Oracle Audience Segmentation data objects.

  Example: To stream data into the "customer" data object, the payload must include the sourceAttribute, which is SourceCustomerID . If you are ingesting data into the "event" data object, the sourceAttribute is SourceEventID.

• <attribute> represents a data object's attribute name to ingest data into. See the Data Objects API for more information on retrieving your data object attribute details.

  Example: To stream data into the "customer" data object, specify an existing attribute name, for example "FirstName".

Response details

Successful requests will return a 200 level response code with a message indicating the status of the request, and the number of records ingested.

{
    "message": "Records written",
    "recordCount": 3
}

Examples

Ingest customer data from a customer unsubscribe operation:

{
  "customers": [
    {
      "SourceCustomerID": 100,
      "FirstName": John,
      "LastName": Doe,
      "Email": john.doe@oracle.com,
      "okToEmail": no,
      "ModifiedTS": 2021-01-1316: 10: 42
    }
  ]
}

Ingest multi-line product data:

{
  "product": [
    {
      "SourceProductID": 100,
      "SourceID": "20210426prodLaunch",
      "Name": StoneWashedJeans,
      "Description": "These jeans are on sale\r\nOnly on boxing day"
    }
  ]
}

Ingest data into a partitioned "order" data object , into the partition "202102", where the data object's referenceField is OrderEntryTS:

{
  "orders": [
    {
      "SourceID": "Source1",
      "SourceOrderID": "002893",
      "OrderEntryTS": "2021-02-02 09:40:01"
    }
  ]
}

Notes:

• Ingest multi-line data using the character "\r\n".

• Ingesting multi-line data is supported with string data types.

Date and timestamp formats

When streaming date and timestamp information into Oracle Audience Segmentation, use one of the following supported formats:

Data types

Refer to Data types for more information about preparing your Unity data prior to ingestion.

Character limits

When streaming data into Oracle Audience Segmentation, note the character limits for each data type:

• String: Up to 500 characters.

• Text: Up to 8000 characters.

If the character limit for a data type is exceeded, the Stream API will not return an error. Instead, data that exceeds the character limit will be truncated in Oracle Audience Segmentation.

View Summary and Errors

Oracle Audience Segmentation enables us to view the error records for the data streamed into Oracle Audience Segmentation using the streaming API. You can search failed records with a specific error code or error message and view the error details. Also, you can filter records based on any particular error code or message. You can view the stream runs summary for a given time range, such as total records, passedCount, errorCount, and jobId.

Learn more

Ingesting data using the Near real time ingestion API

Ingesting Empty or Null Data

Ingest Jobs