Server data transfer

Server data transfer (SDT) is a server-side delivery method for transferring user data from the Oracle Data Cloud platform into your system. After an ID swap has been performed on a user, the platform can deliver data on that user to your server-side profile store — without firing a pixel. Alternatively, your user data can be saved in hourly or daily batch files that you can download via SFTP or an Amazon S3 bucket.

SDT is Oracle Data Cloud's preferred data transfer mechanism and provides the following benefits:

• Increases efficiency by only delivering users known on your site.
• Eliminates use of site bandwidth to transfer user data.
• Provides user data from online, offline, and mobile sources.
• Enables user data to be queued or processed asynchronously.

Note: This document mentions campaigns. Campaigns are no longer created explicitly in the platform UI, but they are still created automatically and used internally to deliver and manage data. Each campaign has a unique campaign ID.

Prerequisites

• You must supply Oracle with an SSH public key (only the public side of the key).
• You can receive user profile data via a real-time POST requests to your server, or you can download batch files from an SFTP server.
• To receive data via real-time POST requests, you have a server-side profile store and you can receive data server-side.
  • Your server must be able to process ≥ 500 requests per second. 6,000 requests per second is recommended.
  • Your server must be able to process ≥ 45 MB per second.
  • You can parse a JSON POST request sent to your server (not needed if you receive user data in a batch file).
• You can ID swap with the Oracle Data Cloud platform. You can pass unique user IDs from your site or mobile app to the platform via a Oracle Data Cloud core tag, image pixel, or an SDK).
• You have signed Oracle's General Data Protection Regulation (GDPR) Right to Use agreement if you plan on receiving user profiles located in the European Union (EU). Contact your Oracle Account Representative to obtain and sign this agreement.
• You have developer resources ready to work on the SDT integration.

Creating and deploying the ID swap tag

An ID swap is the exchange of unique user IDs (UUIDs) between your site and the Oracle Data Cloud platform when a user performs an action, such as visiting a web page. The platform synchronizes your UUIDs to the network of user profiles that are linked together in the Oracle ID Graph, which is used to manage IDs and user attributes for all customers. This synchronization enables you to receive data via SDT in the following scenarios:

• Desktop sites: Depending on your environment, you make calls to an Oracle Data Cloud core tag (recommended) or image pixel. The Oracle Data Cloud core tag contains HTML code and built-in JavaScript functions for sending UUIDs and user attributes and it can be deployed directly on your site or in a tag management system. The image pixel is typically used in environments that require pixels for making tag calls, such as display media.
• Mobile sites and mobile hybrid apps: Deploy a Oracle Data Cloud core tag for mobile sites.
• Mobile apps (native and hybrid): Implement the Oracle Data Cloud Android and iOS SDKs in your apps.

Real-time SDT

Use the real-time SDT endpoint to receive user data as it is collected on users via HTTP POST or GET requests. User data is transferred directly to your server after a qualifying user is ID-synced on your network or the Oracle Data Cloud network.

By default, user data is sent without authentication. The data is sent to an endpoint you selected in a specific JSON format that includes a common UUID.

Using a POST request is required for SDT real-time implementation. It can include multiple categories per user and it does not have any restrictions on data length like the legacy GET request.

To use SDT real-time endpoint:

1. Configure your server for receiving user data. This includes implementing a method for parsing the JSON POST requests sent to your server.
2. Contact your Customer Success Manager or Solutions Consultant and provide them with the following:
   
   • Data transfer URL. The URL used by the Oracle Data Cloud platform back end servers to send user data to your server. You may have multiple URLs depending on how you plan on receiving data linked to different ID types (cookies or Mobile Advertising IDs [MAIDs]) and from different countries and regions.
   • Supported ID Types.If you plan on using separate endpoints for different ID types, specify which ones may be delivered to endpoint. For example, you may have separate cookie and MAID endpoints.
   • Supported Countries/Regions. If you plan on using separate endpoints for different countries and regions, specify which ones may be delivered to endpoint. For example, you may have separate United States (US); Europe, Middle East, and Africa (EMEA); and Asia-Pacific (APAC) endpoints.

SDT real-time data transfer process

Data is transferred to your server in the following sequence:

1. The Oracle Data Cloud platform sends the platform UUID and campaign parameters to your server.
2. If the data is successfully received by your endpoint, the endpoint must return an “HTTP 2xx” response code (200, 202, or 204). If the endpoint returns “HTTP 2xx,” the data is considered to have been received.
3. The server looks up the platform UUID or your UUID, associates new data attributes to user, and returns an HTTP response code. If your endpoint could not process or accept the SDT data, your endpoint must return a response code other than “HTTP 2xx” to avoid being charged for the data. Response codes other than “HTTP 2xx” will cause the SDT system to discard the data and begin throttling subsequent deliveries.

   Important: The default timeout for HTTP requests sent to your SDT endpoint is 2 seconds.

4. When the platform receives an error from your endpoint, it discards the data attempted for delivery and then updates an error rate percentage for your endpoint. This error rate is used in a calculation that determines the throttling for new data deliveries to your endpoint. The throttling calculation is as follows:
           delivery rate = (100 - error rate) + 5
   For example, if the error rate is 10%, the platform throttles the data delivery rate down to 95%. The error rate is updated in real-time; therefore, if an endpoint fails, the platform instantly throttle the data delivery rate down to the minimum, which is 5%.

JSON POST format

The JSON-formatted data will contain a CATEGORIES array that lists the CategoryID and Timestamp for each campaign win as shown in the following example:

{
	"DeliveryTime": "Fri May 07 08:24:48 PDT 2016",
	"DestinationId": 1,
	"PixelCount": 1,
	"Pixels": [{
		"BkUuid": "6KMp1LAq99eQPyOu",
		"BKClear": 1,
		"CampaignId": yourCampaignID,
		"Categories": [{
			"Id": 5915,
			"Utc": 1375750288
		}, {
			"Id": 5962,
			"Utc": 1462609486
		}],
		"PartnerUuid": "YOUR_UUID",
		"PixelUrl": " https://stags.bluekai.com/site/YOUR_SITE_ID?limit=0&account=AO+",
		"UtcSeconds": 1462609486
	}]
}

The following table describes the parameters in the JSON-formatted data contained in the HTTP POST requests.

Note: Accepting mobile IDs enables you to offer marketers and advertisers the ability to target mobile app users based on their online behavior. Oracle Data Cloud can deliver user data into your platform that is linked to mobile advertising IDs (MAID), which are also referred to as "device IDs" when derived from mobile apps. The platform can send you user data that is linked to IDFAs, hashed IDFAs, hashed Android IDs, Google Advertising IDs, and hashed Google Advertising, IDs (hashed IDs can be sent with either SHA-1 or MD5 encryption). For more information on how to receive MAIDs in your platform, click here.

Whitelisting the new SFTP server IP Address

In some cases when SDT delivers files to a customer-owned endpoint, you may need to whitelist our SFTP server. For customers who need to whitelist the SFTP server, we recommend that you whitelist the public IP ranges for the Oracle Cloud Infrastructure Phoenix data center (us-phoenix-1). Whitelisting these ranges ensures future compatibility if we move to a new server and the current IP address changes.

See Oracle Coud Infrastructure Address Ranges for current information about the IP address ranges you can use.

If your are unable to whitelist the Phoenix data center IP ranges or prefer not to, you can whitelist sftp.bluekai.com: 129.146.213.6. Although we do not currently plan to move from this IP address, we reserve the right to change it in the future.

SDT batch

When you use the SDT batch endpoint, user data is saved to hourly or daily batch files that are formatted according to your preferences. You download the batch files from an Oracle Data Cloud server or an Amazon S3 bucket and then import them into your system. Select this option if you have limited data storage capacity or if you do not have the resources to implement SDT on a server.

To receive user data using SDT batch:

1. Configure your system so that it can automatically receive batch files:
   • If you are using SFTP, you need an automated method for logging in to an SFTP account on the Oracle Data Cloud server and downloading files via SFTP.
   • If you are using S3, you need to an automated method for exporting the batch files from an Amazon S3 Bucket.
   • If you are using S3, create an S3 user for this purpose, and grant the user write access to the bucket where your batch files will be stored.
2. Contact your Customer Success Manager or Solutions Consultant and provide them with the following information:

   • Supported ID Types.If you plan on using separate batch files for different ID types, specify which ones may be delivered to endpoint. For example, you may have separate cookie and MAID files.

     Supported Countries/Regions. If you plan on using separate batch files for different countries and regions, specify which ones may be included in the file. For example, you may have separate US, EMEA, and APAC files.

   • SSH public key :Only the public side of the key is required.

   • SDT batch method: Select SFTP or S3. If you are using SFTP, Oracle Data Cloud provides an SFTP account and credentials.
   • S3 credentials: If you are using S3, provide the bucket name, user access key, and secret key to Oracle Data Cloud. We use these credentials to upload your batch files to your Bucket.
   • Batch cadence: The interval at which your batch will be batched, which is either hourly or daily.
   • Batch field delimiter: The character you will use to separate fields in your batch file. The batch output file includes one line per user record and the fields in each record are separated by a single delimiter. Oracle recommends the tab, space, or vertical bar delimiters because some fields internally include one or more delimiters that could affect the parsing of your files, such as the Categories field. Colon, comma, and semicolon separators are also supported.
   • Batch fields: Select one or more of the supported fields to be included in your batch files and the exact order in which you want them listed.

Supported batch fields

Batch file naming conventions

Where: foldername uses the same name as the SFTP name or location.

To use SDT batch to send ad targeting on users in the target audience:

1. Log in to your SFTP account on the Oracle Data Cloud server and download the batch files. Alternatively, export the batch files from your Amazon S3 bucket.
2. Import the batch files into your system and process the user data.
3. Contact My Oracle Support (MOS) to confirm that you are receiving and processing the user data.

Deliver user data

To deliver user data into your platform:

1. The Oracle Data Cloud platform creates a special regex pixel that associates your ID swap pixel's domain with your SDT endpoint. This association enables the platform to identify you as an SDT partner. The platform looks for and matches a defined pattern in the URL path of your ID swap pixel (subdomain or parameter) to initiate an SDT. The platform typically identifies a match based on the site ID in the pixel.

   Note: If you are an AudienceOn mobile partner, Oracle Data Cloud sets up a campaign to continuously transfer all third-party data to you. The platform uses a regex pixel to associate the campaign with your data transfer URL.

2. The client delivers audience data, which automatically starts the SDT data delivery. Your regex pixel will automatically be linked to the client's campaign.
3. If you are using SDT real-time, parse the JSON-formatted data included in the HTTP POST request or the data in the URL of the GET request to send ad targeting to the users.

Mapping SDT data

Link the SDT data you are receiving with the audience segment objects in your platform using one of the following methods:

• Audience injection: The automated method for creating and mapping audience objects between the Oracle Data Cloud platform and execution platforms. The Oracle Data Cloud platform programmatically creates audience objects in your platform via your APIs and your APIs return the object used in your platform for storing and targeting users (this object is typically referred to as a user audience, segment, or list). After this automated mapping has occurred, you can add the users you receive via SDT to your audience objects.
• Managed Mapping. The preferred alternative mapping method if you cannot support an audience injection integration. Managed mapping uses real-time email notifications and a mapping UI to notify you when a client's data needs to be mapped in your platform and automatically start delivery when you are done.
• Manual Mapping: The following manual methods may be used; however, they are not the preferred mapping solutions:
  • Campaign-level mapping: This manual method typically includes having the client create and name the audience/segment object in your platform and then specify the campaign ID to be associated with that audience.
  • Audience sharing (taxonomy API): In this category-level mapping method, the client shares the audience with you, which enables you to get the audience name and composition from your seat. You can also call the categories API, pass the category IDs into the parentId field, enable the showReceivedAudienceCategories flag to return the categories in the shared audiences, and enable the fullpath flag to return the full taxonomy path of the categories.
  • Audience sharing (audience API): In this category-level mapping method, the client shares the audience with you. This will generate an email notification that includes the client's name and the name of their audience. This enables you to get the audience composition from your Oracle Data Cloud platform seat. You can also call the audiences API to do the following:
    • Make a GET list call with the name of the audience you received from the client passed into the name_or_id field. The Audience API will return the audience ID.
    • Make a GET read call with the audience ID. The audiences API response will include a segments object that includes the audience composition.
  • Category whitelisting: Whitelist the categories included in your data delivery, which enables you to get the names and IDs of the client's categories.

Oracle Data Cloud Opt-Out Policies

Oracle requires that all partners who receive third-party data from the Oracle Data Cloud platform remove data for users who have opted out of third-party, interest-based advertising. Oracle Data Cloud provides partners with downloadable lists of device and browser identifiers for opted-out users. We update the opt-out lists daily to include identifiers for users who have opted out recently.

Partner requirements

If you receive third-party data from Oracle Data Cloud, you must delete device and browser identifiers for opted-out users from your data. Deletion of these identifiers ensures that opted-out users are not targeted again for advertising based on data provided by Oracle Data Cloud.

We provide you with links to downloadable lists of identifiers related to the data types that you transact on. For example, if you use ID-swapped partner-based user IDs, Oracle Data Cloud cookie IDs, and MAIDs, you receive links for your partner IDs, for Oracle Data Cloud cookie IDs, and for MAIDs.

We update the opt-out lists daily. You are required to access or download your Oracle Data Cloud opt-out lists and delete data at least once a week. A weekly cadence guarantees that data for opted-out users is removed in a timely manner.

Why is this a requirement?

If you receive third-party data from Oracle Data Cloud, you are contractually obligated to delete data for opted-out users. This requirement enables us to comply with local and global data privacy regulations, including the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA).

In addition, industry groups such as the Network Advertising Initiative (NAI) and the Digital Advertising Alliance (DAA) require members to provide an opt-out mechanism for users who choose not to receive targeted advertising.

How Oracle Data Cloud generates and delivers opt-out feeds

Oracle Data Cloud collects opt-outs for each data type from the sources listed in the table below. We prepare lists of the identifiers for opted-out browsers and devices and save them as text files in locations that you access by using secure URLs. Your account manager provides you with URLs for the locations of your opt-out files. You can download your files for processing from that location. You receive a separate URL for each of your delivery endpoints.

We update the opt-out files every day and overwrite the previous version in each location. The new files contain the previous opt-outs appended with an incremental update of opted-out identifiers that have been collected in the previous 24 hours. We maintain separate opt-out lists for each ID type.

The lists for partner IDs contain identifiers for only that specific partner. The lists for other data types contain identifiers for all opted-out users. You must develop a process for extracting the IDs that pertain to your business.

The following table lists the opt-out sources, data retention period, and other information about each data type.

MAID Device Type IDs

Opt-out process security

Oracle Data Cloud stores opt-out list files in secure Amazon S3 buckets. Oracle Data Cloud securely shares the bucket locations externally via an AWS pre-signed URL. These AWS S3 pre-signed URLs then redirect the user requests to the opt-out list files.

The final endpoint URL provided to partners contains a payload (an S3 path specific to that partner's attributes) that is securely tokenized by using a random string secret key.

Frequently asked questions

What if our server will be off-line for any period of time for maintenance or other scheduled down times?
If you know your server will be down for a period of time, you should set your data delivery to Idle during that period. Contact your Client Service Manager with any questions regarding this issue.

Why is a pixel still required for SDT?
The Oracle Data Cloud platform uses the pixel URL as an identifier to trigger a server-to-server response. The platform uses a regex pixel to look for and match a defined pattern from the buyer to initiate a server-side data transfer. The pattern can be any part of the pixel URL path (subdomain or parameter). By using this method of "activating" a server-side data transfer, the platform enables you to provide clients with a pixel exactly like the current pixel transfer workflow for use within our system. When the platform sees a pixel that matches with a defined SDT format, it automatically uses SDT.

Can the ID-swap pixel be placed within an iframe?
Yes, the ID swap pixel will still function within an iframe.

Is there a default time span for expiration?
All cookies should be expired after 30 days. However, we recommend that buyers match expiration to ad campaign recency wherever possible. For example, if you are buying Travel to reach an audience traveling within the next seven days, you should expire the cookies after that 7-day period.

Which element that we receive in the JSON POST actually defines the data (CampaignId, CategoryId)?
Data is represented by CategoryId. A user can possess multiple CategoryIds, and there may be multiple CampaignIds in the JSON POST. A campaign in our system is an entity that is set up to target an audience. An audience consists of data categories. Therefore, a user might qualify for one or more campaigns, and each campaign may contain one or more category IDs.

Do you re-send all of the segments for a user with every POST?
No, we only send a partial update. Only those segments for which data has not been transferred within the last 30-days are sent for a particular user.

Can you send the data to multiple data transfer URLs?
The platform send the data to only one endpoint. You typically parse the data, as needed, based on Campaign ID and/or the Pixel URLs that are returned within the JSON POST. We are able to append specific parameters to the 'Pixel URL' on a campaign-by-campaign basis.

What happens when a user indicates they do not want to be tracked by the Oracle Data Cloud platform? How is that information communicated to us? Do we need to delete the data from our database for that user?
When a user opts out, the platform deletes all category-level information about that user in its profile store and persists the opt out with an 'ignore flag' stored server-side. The platform may also store the users' opt-out status as a cookie in users' browser cache. After an opt out has propagated through the platform, it no longer transmits information about that user, including opt-out status.

Oracle Data Cloud provides you with a daily feed of opted-out users so you can remove users already delivered to you from targeting in your platform.

Technical Support

Technical support during the setup of your SDT integration will be provided by an Oracle Data Cloud Integration Support Engineer (ISE). After your integration has been activated, you can contact your ISE for further assistance. If you receive 3rd-party data, a partner manager will be assigned to you, and you contact them for support.

Learn more

Data delivery