Data Ingest

Data ingest is the process of collecting and classifying user data in the Oracle Data Cloud platform. The data ingest process entails extracting user's attributes from your online, offline, and mobile sources and then mapping the collected attributes into categories in your taxonomy via classification rules.

For example, if a user is shopping for a smartphone on an electronics website, mobile site, or retail store, they could be tagged with an in-market=smartphone attribute (a key-value pair called a phint). When this data is imported into the Oracle Data Cloud platform, the phint can be mapped to a Smartphone category in your private taxonomy via a classification rule that states "if in-market=smartphone, then category = Smartphone". After your online, offline, and mobile data is ingested into your taxonomy, you can deliver it to multiple media execution platforms for targeting, optimization, and modeling.

If you are just learning about Oracle Data Cloud and want to understand the benefits of ingesting your data into our platform, read about the following ingest methods to determine which one is best for you:

Online ingest: Collect and organize your page and online user attributes (for example, product pages, purchase intent, add-to-cart actions, and conversions) by deploying an Oracle Data Cloud core tag.
Offline onboard: Onboard data from a data warehouse, a customer relationship management (CRM) system, or any other structured offline source using offline match integration.
User data API: Transfer user data into the Oracle Data Cloud platform using a real-time server-side API. Use this method if you are a DMP client that does not have sufficient space for storing user data.
Mobile ingest: Collect data from your mobile website and mobile hybrid apps by deploying an Oracle Data Cloud mobile core tag.
Mobile app ingest: Collect data from your mobile native and hybrid apps using the Oracle Data Cloud Android and iOS SDKs.

Data Providers Onboarding EU Data. To ingest data for user profiles located in the European Union (EU), you must have signed Oracle's General Data Protection Regulation (GDPR) Consent agreement. Contact your Oracle account representative to obtain and sign the agreement.

Online ingest checklist

Create a container

Use the container tool in the Oracle Data Cloud platform to generate a container. You can also use container API to automate the creation of containers and their site IDs.

A container provides JavaScript and HTML code that collects the user and page-level attributes that you need to identify. The container transfers the data to the Oracle Data Cloud platform, where it then can be normalized into a standard taxonomy using classification tools.

Scope the data to be transferred to the platform

Look at your website and identify the data you want to extract from your web pages and pass into the container. Typically, you will extract the following technical elements from your page to represent human readable user attributes:

HTML DOM properties and page keywords: The container includes JavaScript code that automatically collects properties such as document.referrer and document.title. Click here for more information on HTML DOM properties.
Native variables: Properties that exist in your rendered HTML pages, such as product SKU numbers.
Other data: Data such as form fields and shopping cart items that do not natively exist in the DOM

Contact your Solutions Consultant to:

Identify your business goals for the DMP.
Scope your sites.
Design a data collection strategy based on your objectives.
Identify the native variables on your site that should be collected.
Identify traffic, conversion, and other analytics variables.
Learn how to use the container's bk_addPageCtx() function to collect data from your forms, shopping cart, and other sources.

If you are a DMP client, see Oracle Data Cloud core tag implementation for detailed onboarding configuration instructions.

If you are a data provider, see Becoming a Data Provider.

Configure the container for deployment

Add key-value pairs (referred to as phints) to the container for the elements you want to extract from your website. For example, a phint for an automobile site could be make=chevy.

Every website has different data that is critical for classifying and targeting your users. You need to add phints to your container so that your data can be mapped to categories in your taxonomy.

Deploy the container

Insert the container tag code directly above the closing </body> tag of the pages in your website.

After you deploy your container tag and create classification rules, your site and user data begins to flow into categories in your Oracle Data Cloud platform taxonomy.

Classify your data

Select a method for mapping the data extracted from your site to categories in the taxonomy. Categories, which are the building blocks of the Oracle Data Cloud platform, represent collections of users. For example, a category named "Red" would represent all of the users who have been tagged with the attribute "Red". The taxonomy is a hierarchical tree used to organize the categories.

The Oracle Data Cloud platform provides the following methods for classifying your data:

Taxonomy Manager:Use this method if you are a DMP client who wants to manually create the data mapping rules and categories.
Category and rule APIs: If you are a DMP client, use these method if you want to programmatically create data mapping rules and categories.
Classification via the Oracle Data Cloud classification and taxonomy team. Use this method if you are a DMP client or data provider who wants to work with Oracle Data Cloud on mapping your user data. Contact your Customer Success Manager or a Solutions Consultant for more information..

After your data is classified, you can target, model, optimize, and analyze your data or monetize it.

If you are an Oracle Data Cloud platform client, you can deliver your data to execution platforms or you can share it with other DMP partners.
If you are data provider, you can sell your data in the Oracle Data Marketplace.

To work with Oracle Data Cloud on mapping your user data, you will create a data map that our taxonomy and classification teams use to organize your data into a taxonomy. Your data map defines the following:

The set of keys used in your ingested online data
The possible set of values for each key, and make the values human readable, if necessary
The hierarchical relationships, if any, between a set of keys

Monitor your data ingest

Check if your tag is calling the platform. Use ingest and delivery monitoring to verify that your tag is firing calls.
Check if your inventory is growing. Use the ingest and delivery monitoring to verify that the amount of inventory per category is increasing daily.
Check your 30-day inventory. Use the Audience Builder to view the estimated number of unique users in your categories based on current configurations. This data is updated once daily at about midnight GMT. You can also use the categories API to automate your inventory check for unique users.

You need to verify that your data is being collected and classified correctly and that your site is generating the expected amount of inventory.

For more details, see: Becoming a Data Provider.

User data API checklist

Transfer your user data into the Oracle Data Cloud platform using a real-time server-side API. Use this method if you are a DMP client that does not have sufficient space for storing user data. You can perform an ID swap and then send user data to be secured on an Oracle Data Cloud server, which will function as your cloud profile store. You can call the user data API anytime to transfer additional attributes and deliver user data out of the system to execution platforms.

To get your developer keys:

Log in to partner.bluekai.com, click Tools, and then click the Web Service Key Tool link.

The Web Service Key Tool displays your web service user key (bkuid), which is your unique ID for accessing the user data API. You will pass your user key and the request signature as arguments when you make calls to the user data API.
Click Show Private Key to display your web service authentication key (bksecretkey).

The user data API uses this private cryptographic key and a HMAC-SHA256 encryption algorithm to generate your message signature. After the user data API receives your signed message, it authenticates the request and compares the calculated message signature result against the value received in the request. If the two message signatures are identical, the message is considered authentic and the request is processed and returned to you.
Use the ping API to validate that you can use your Oracle Data Cloud developer keys to send authenticated message requests to the user data API.
Read authenticating Oracle Data Cloud API calls to learn how to send authenticated requests to the user data API.

To send data into the Oracle Data Cloud platform, your user data API calls must be authenticated using your developer keys.

Create a container that includes an ID swap tag

If you are a current DMP client or partner that has already deployed an Oracle Data Cloud core tag on your site, add a phint to your Oracle Data Cloud core tag for passing your UUIDs to the platform.
If you are a new client or partner, use the container tool in the Oracle Data Cloud platform to generate a container, and then add a phint to your container for passing your UUIDs. You can use the container API to programmatically create additional containers and their site IDs.
If you already ID swapping and storing the Oracle Data Cloud UUID (BKUUID), you can pass the BKUUID in your ID swap tag.

The ID swap provides the platform with your UUIDs. After an ID swap has been triggered for a user, you can use the user data API to transfer their attributes into the Oracle Data Cloud platform.

If you are using ID swaps to pass your UUIDs, read ID swapping for details on creating and deploying ID swap tags.

Deploy the ID swap tag

Place your ID swap tag on each page in your network.
Fire the ID swap tag on each unique user once every 30 days.

The platform confirms that your UUIDs are being collected and stored in the Oracle Data Cloud platform.

Monitor the ID swap tag

Verify that your ID swap tag is sending your UUIDs.

Paste your ID swap tag in a web browser, and use Firebug or another web inspector to verify that your web page is passing your UUIDs. The platform checks if your ID swap tag was fired.
Verify that your inventory of UUIDs is accumulating.
Use the the Taxonomy Manager or the category and rule APIs to create a category that represents your ID swap site and a rule that maps the firing of your ID swap tag to the new category.
Use the inventory trend report to view the number of ID swaps being executed daily.
Use the site hit report to compare the inventory figures to the number of site hits the ID swap tag is generating.
Use the Audience Builder in the Oracle Data Cloud platform or the categories API to view the estimated number of unique users seen in your ID swap category. The inventory figures in your taxonomy may not initially be accurate but they show that your inventory of UUIDs is ramping up.

Verify that your ID swaps are working so that when your users visit your site, The platform receives their UUIDs and you can then transfer their attributes into the Oracle Data Cloud platform.

Classify your data

Select a method for mapping your user data to categories in the taxonomy. Categories, which are the building blocks of the Oracle Data Cloud platform, represent collections of users. For example, a category named "Red" would represent all of the users who have been tagged with the attribute "Red". The taxonomy is a hierarchical tree used to organize the categories.

The Oracle Data Cloud platform provides three methods for classifying your data:

Taxonomy Manager: Use this method if you are a DMP client who wants to manually create the data mapping rules and categories.
Category and Rule APIs: Use these methods if you are DMP client who wants to programmatically create the data mapping rules and categories.
Classification via the Oracle Data Cloud classification and taxonomy team. Use this method if you are a DMP client or data provider who wants to work with Oracle on mapping your user data. This method requires the purchase of Oracle consulting services.

After your user data has been transferred and classified, you target, model, optimize, and analyze your it, or you can monetize it.

If you are a DMP client, you can deliver your user data to execution platforms, or you can share it with other DMP partners.
If you are data provider, you can sell your user data in the Oracle Data Marketplace.

To work with Oracle Data Cloud on mapping your user data, contact your Solutions Consultant or Customer Success Manager. You will create a data map that our taxonomy and classification teams will use to organize your data into a taxonomy. Your data map will do the following:

Define the set of keys used in your user data.
Define the possible set of values for each key, and make the values human readable, if necessary.
Defines the hierarchical relationships, if any, between a set of keys.

Call the user data API

Use Python or another programming language to write a server-side script that passes your unique user IDs and key-value pairs (phints) categorizing your users.

The user data API documentation includes a Python script that demonstrates how to generate an authentication signature, construct a user data API request, and make an HTTP call to the platform.

Monitor your data ingest

Check if your inventory is growing. Use the inventory trend report to verify that the amount of inventory per category is increasing daily.
Check your 30-day inventory. Use the Audience Builder in the platform UI to view the estimated number of unique users in your categories based on current configurations. You can use the categories API to programmatically check your inventory of your unique users.
Verify that your user data is collected and classified correctly and that it is generating the expected amount of inventory.

Mobile ingest checklist

Create a container with an Oracle Data Cloud mobile core tag

Use the container tool in the Oracle Data Cloud platform to generate a container.
Apply the mobile configuration template to the container. See creating containers for more information. You can use containers API to programmatically create additional containers and their site IDs.

The Oracle Data Cloud mobile core tag includes a set of JavaScript functions that collect user and phone attributes from your mobile properties and transfers them into the Oracle Data Cloud platform.

If you are working with a solutions architect, they will provide you with instructions for creating your container and using it to pass site and user data.

Scope the data to be transferred to the platform

Look at your mobile properties and identify the data you want to extract and pass into the Oracle Data Cloud mobile core tag. Typically, you will extract the following technical elements from your mobile web and mobile hybrid apps to represent human readable user attributes:

HTML DOM properties and page keywords: The container includes JavaScript code that automatically collects properties such as document.location, document,referrer, document.title. Click here for more information on HTML DOM properties.
Native variables: Properties that exist in your rendered HTML pages, such as product SKU numbers.
Other data: Data such as form fields and shopping cart items that do not natively exist in the DOM

If you are working with a solutions architect, they will work with you to understand your business goals for the DMP, and then scope your sites and design a data collection strategy based on your objectives. They will help you identify the native variables in your mobile properties that should be collected. They will also identify traffic, conversion, and other analytics variables and show you how to use the bk_addPageCtx() function in the container to collect data from your forms and your shopping cart.

Configure the container for deployment

Add key-value pairs (referred to as phints) to the container for the user IDs (mobile web)/device IDs (mobile apps) and the user attributes you want to extract from your mobile properties. For example, a phint for a user attribute on an automobile site could be “Make=Chevy”.

Every mobile website or mobile app has different data that is critical for classifying and targeting your users. You need to add phints to your container so that your data can mapped to categories in a taxonomy.

Deploy the container

Insert the container tag code directly above the closing <body> tag of the pages in your mobile website (mobile view for hybrid apps).

After you deploy your container, your site and user data begins to flow into the Oracle Data Cloud platform.

Classify your data

Select a method for mapping the data extracted from your mobile properties to categories in the taxonomy. Categories, which are the building blocks of the Oracle Data Cloud platform, represent collections of users. For example, a category named "Red" would represent all of the users who have been tagged with the attribute "Red". The taxonomy is a hierarchical tree used to organize the categories.

The Oracle Data Cloud platform provides the following methods for classifying your data:

Self-classification platform: Use this method if you are a DMP client who wants to manually create the data mapping rules and categories.
category and rule APIs: Use these methods if you are DMP client who wants to programmatically create the data mapping categories and rules.
The Oracle Data Cloud classification and taxonomy team: Use this method if you are a DMP client or mobile data provider who wants to work with Oracle Data Cloud on mapping your user data. This method requires the purchase of Oracle consulting services.

After your data has been classified, you target, model, optimize, and analyze your data, or you can monetize it.

If you are a DMP client, you can deliver your data to execution platforms, or you can share it with other DMP partners.
If you are mobile data provider, you can sell your data in the Oracle Data Marketplace.

To work with Oracle Data Cloud on mapping your mobile data, contact your Solutions Consultant or Customer Success Manager. You will create a data map that our taxonomy and classification teams will use to organize your data into a taxonomy. Your data map will do the following:

Define the set of keys used in your ingested mobile data.
Define the possible set of values for each key, and make the values human readable, if necessary.
Defines the hierarchical relationships, if any, between a set of keys.

Monitor your data ingest

Check if your tag Calling the platform. Use the site hits report in the Oracle Data Cloud platform to verify that your tag is firing calls to the plat.
Check if your inventory is growing. Use the inventory trend report to verify that the amount of inventory per category is increasing daily.
Check your 30-day inventory. Use the Audience Builder in the Oracle Data Cloud platform to view the estimated number of unique users in your categories based on current configurations. You can use the categories API to programmatically check your inventory of your unique users.

You need to verify that your mobile data is being collected and classified correctly and that your site is generating the expected amount of inventory.

DMP clients: Oracle Data Cloud core tag implementation
Mobile data providers: data provider implementation

Important: For hybrid apps, you can use the Oracle Data Cloud iOS and Android SDKs to collect and transfer mobile data instead of the Oracle Data Cloud mobile core tag. For native apps, you must use these SDKs.

Mobile app ingest checklist

Get your site ID

You can get your site ID using one of the following methods:

Use the container tool in the platform UI to look up your site ID.
Use the containers API to programmatically retrieve your site ID.

You need to set your site ID in the platform SDKs to associate the data being extracted from your site with your DMP.

Scope the data to be transferred to the platform

Look at your mobile apps and identify the data you want to extract and pass into a platform SDK. Typically, you will extract the following technical elements from your mobile native and mobile hybrid apps to represent human readable user attributes:

HTML DOM properties and page keywords: The container includes JavaScript code that automatically collects properties such as document.location, document,referrer, document.title. Click here for more information on HTML DOM properties.
Native variables: Properties that exist in your rendered HTML pages, such as product SKU numbers.
Other data: Data such as form fields and shopping cart items that do not natively exist in the DOM.

If you are working with a solutions architect, they will work with you to understand your business goals for the DMP, scope your sites, design a data collection strategy based on your objectives, and help you identify the native variables in your mobile properties that should be collected. They will also identify traffic, conversion, other analytics variables, and show you how to collect data from your forms and your shopping cart.

Detailed onboarding configuration instructions are located here if you are a DMP client or here if you are a data provider.

Collect data from your native and hybrid apps

Use the Oracle Data Cloud iOS SDK and the Oracle Data Cloud Android SDK to collect data from your native and hybrid apps.

Classify your data

The Oracle Data Cloud platform provides three methods for classifying your data:

Self-classification platform: Use this method if you are a DMP client who wants to manually create the data mapping rules and categories.
Self-classification API: Use this method If you are DMP client who wants to programmatically create the data mapping rules and categories.
Classification via the Oracle Data Cloud classification and taxonomy team. Use this method if you are a DMP client or mobile data provider who wants to work with Oracle Data Cloud on mapping your user data. This method requires the purchase of Oracle consulting services.

After your data has been classified, you target, model, optimize, and analyze your data, or you can monetize it.

If you are a DMP client, you can deliver your data to execution platforms, or you can share it with other DMP partners.
If you are mobile data provider, you can sell your data in the Oracle Data Marketplace.

Define the set of keys used in your ingested mobile data.
Define the set of values for each key and make the values human readable if necessary.
Defines the hierarchical relationships, if any, between a set of keys.

Monitor your data ingest

Check if your tag is calling the platform. Use ingest and delivery monitoring to verify that your tag is firing calls.
Check if your inventory is growing. Use ingest and delivery monitoring to verify that the amount of inventory per category is increasing daily.
Check your 30-day inventory. Use the Audience Builder to view the estimated number of unique users in your categories based on current configurations. You can use the categories API to programmatically check your inventory of your unique users.

You need to verify that your data is being collected and classified correctly and that your site is generating the expected amount of inventory.

Learn more

Managing your taxonomy

Creating containers

Oracle Data Cloud core tag implementation