2Import Commerce Data

About Commerce Integrations

Oracle Adaptive Intelligent Apps for CX integrates with Oracle commerce applications and non-Oracle commerce applications. Depending on your integration, a different extent of manual configuration is necessary for both data import, scheduling, and getting your widgets set up for click tracking.

Note: For all integration approaches, before or during the data import process, personal information about consumers is stripped out, encrypted, or anonymized to protect consumer privacy.

The following sections summarize the different supported integrations.

Oracle Commerce Cloud

The integration with Oracle Commerce Cloud is registered as a trusted integration and requires minimal setup. After connecting to Oracle Commerce Cloud, automated processes import and synchronize consumer, product, brand, and other data. For widget configuration, you download predefined templates and upload them to Oracle Commerce Cloud where they are ready to collect real-time clickstream data for recommendations.

See Import Data from Oracle Commerce Cloud for more information.

Oracle Commerce Platform

The integration with Oracle Commerce Platform provides plug-ins to extract consumer, product catalog, and order data into flat files. The plug-ins strip any personally identifiable information about consumers from the files, and transform the data into the adaptive intelligent data model. Because the plug-ins transform the data model, you don't need to map fields or write your own scheduling jobs.

See Import Data from Oracle Commerce Platform for more information.

Other Commerce Applications

Oracle Adaptive Intelligent Apps for CX supports integrating with any application platform that can access its REST API to upload or retrieve data and record consumer responses. Integration requires field mapping, coding API requests to upload and synchronize data, and creating scheduling jobs. The application provides samples of widget configuration code you can use to copy into your widgets.

See Import Data from Other Commerce Applications for more information.

Email Integrations

Use the Email Templates page to design widgets for use in your email campaigns. You select the layout and styles you want, generate new images for open-time optimization, and copy the HTML into your email templates.

See About Email Widget Templates for more information.

Import Data from Oracle Commerce Cloud

To import data from Oracle Commerce Cloud, you must perform the following steps:

  1. In the Oracle Commerce Cloud Service administration area, register Oracle Adaptive Intelligent Apps for CX as follows:

    1. On the Settings page, click Web APIs.

    2. On the Registered Applications page, click Register Application.

    3. Enter a unique name, such as Oracle AIApps CX.

    4. Click Save.

  2. Locate and copy your API key as follows:

    1. In the Registered Applications list, click the name you added, and then click Click to Reveal.

    2. Select and copy the entire value in the Application Key field, and then click Cancel to close the window.

      Paste the value to a temporary location so you can access it later.

  3. Obtain the value for the service endpoint URL and save it temporarily so you can access it in the next step.

    The service endpoint URL is your agent URL, such as https://www.mystore.com, as shown in the following figure.

    Connect pop-up window showing the fields for API key and service endpoint URL

  4. Connect to Oracle Commerce Cloud as follows:

    1. On the Connections page in Oracle Adaptive Intelligent Apps for CX, click Oracle Commerce Cloud.

    2. Enter the API key and the URL value you obtained.

    3. Click Test Connection.

    4. Click Continue.

  5. Select the Enabled check box for each site that you want enabled for adaptive intelligence.

  6. For each enabled site, select whether the site must require consent from shoppers before collecting any data about them, such as their clicks and order history.

    This setting determines data protection rules. For more information about shopper consent, see Site-Enabled Shopper Consent.

  7. Click Save.

The automated process will start to receive data from your site. As it receives data, your products, brands, categories, and promotions are reflected on the Insights page.

Import Data from Oracle Commerce Platform

Importing data from Oracle Commerce Platform requires setup steps. You can choose one of the following approaches:

  • Download and use plug-ins to extract your data into a data warehouse and upload to Oracle Storage Cloud.

  • Extract your data manually and implement REST API calls for initial and ongoing uploads.

This section describes the first option, as illustrated in the following figure. For information about using the REST API directly, refer to REST API for Oracle Adaptive Intelligent Apps for CX.

The Oracle Commerce data warehouse plug-in extracts catalog, order, and profile data and saves it into flat files. The adaptive intelligence plug-in reads the data files, strips out personally identifiable information about consumers, and sends the output to the Oracle Storage Cloud server. A scheduled job moves the data to the adaptive intelligence database. When you start the plug-in, it looks for new or changed data files in the output subfolders that you created for the Oracle Commerce data warehouse plug-in.

Overview of initial data flow from Oracle Commerce Platform. On the Oracle Commerce Platform server, use the data warehouse plug-in to extract data files. Use the adaptive intelligence plug-in to move the data to Oracle Storage Cloud. Data moves to the adaptive intelligence database using its own scheduler.

For more information and configuration steps, see Summary of Configuration Steps.

Configure the Oracle Commerce Data Warehouse Plug-in

Perform the following steps in preparation for using the plug-in for extracting data:

  1. If you don't already use a data warehouse, create a database schema on a separate server to avoid placing strain on the production server.

    Refer to Creating the Data Warehouse Schema in the Oracle Commerce Business Intelligence Installation and Configuration guide, for details.

  2. Download and install the Oracle Commerce Data Export Utility.

    1. Accept the license agreement on the Oracle Commerce Data Export Utility Downloads page on the Oracle Technology Network, and then click the download link.

    2. Extract the contents of the Oracle-Commerce-Data-Export-Utility.zip file to a temporary location.

    3. Extract the contents of the config.zip and src.zip files to the Oracle Commerce data warehouse server.

  3. On the Oracle Commerce data warehouse server, locate the following three configuration files in the /config/atg/reporting/datawarehouse/service folder:

    • CatalogFileLogger.properties

    • OrderFileLogger.properties

    • ProfileFileLogger.properties

  4. For each of these three files, change the value for defaultRoot to an absolute path and remove the carrot (^) character.

    For example, you might change the value for ProfileFileLogger.properties from defaultRoot^=/atg/dynamo/service/DWDataCollectionConfig.defaultRoot to defaultRoot=/app/oracle/product/aiacs/atg_output/profile.

    Note: Ensure that you keep the default values for all other properties in the files.
  5. Create the three subfolders to match each of the defaultRoot values you set in the properties files. For example:

    • /app/oracle/product/aiacs/atg_output/catalog

    • /app/oracle/product/aiacs/atg_output/order

    • /app/oracle/product/aiacs/atg_output/profile

Configure the Commerce Platform Server

To properly retrieve data from Commerce Platform, you must perform the steps outlined in the Oracle Commerce Data Export Utility attachment available on My Oracle Support (document ID 2254809.1).

Perform the following steps in addition to the steps in that document:

  1. Enable SHA-256 and MD5 formats for email addresses by adding the following two lines to the JSONOutputCustomizer.properties file.

    secureLoggingPropertyExceptions=\
    user.email
  2. Increase the maxDepthMap OrderRepository value in the ProcessorPersistenceConfiguration.properties file from 4 to 5.

  3. Add the following lines to the OrderResponseGenerator.properties file to exclude shipping and payment groups from Commerce.
    atg.commerce.order.CommerceIdentifier#paymentGroups,\
    atg.commerce.order.CommerceIdentifier#shippingGroups,\
  4. Verify that the ExportLogRotationMessageSink.properties file defines three lines for output folders (one for each entity type). For example:

    defaultRootByFilePrefix=\
    order_=/app/oracle/product/aiacs/ATG_TEST/atg_output/order,\
    catalog_=/app/oracle/product/aiacs/ATG_TEST/atg_output/catalog,\
    user_=/app/oracle/product/aiacs/ATG_TEST/atg_output/profile
  5. Verify that the DeploymentDWDataCollectionConfig.properties file is set as enabled and specifies a valid log folder. Refer to Configuring an Asset Management Server in the Commerce Business Intelligence Installation and Configuration Guide for details.

  6. Verify that the OrderSubmitLoader component is configured with scheduling properties. Refer to the Data Loader Components section in the Commerce Business Intelligence Installation and Configuration Guide for details.

Configure the Adaptive Intelligence Plug-In

The following steps must be performed by a user with the Operations Manager role.

  1. Sign in to Oracle Adaptive Intelligent Apps for CX.

  2. On the Connections page click Oracle Commerce Platform.

  3. Enter your storefront URL and select the check box to proceed, and then click Save.

    This step associates the application with your Cloud Storage Cloud location, creates your unique credentials, and prepares the built-in scheduler to retrieve data.

    Connect page for Oracle Commerce Platform showing the option to edit the storefront URL, reset the authentication URL, and buttons to download the plug-in and its properties file

  4. Click Download Plug-In and save the ATGIngestionClient.jar file to a temporary location.

  5. Click Download Properties, and save the aiacs.properties file to the same location.

  6. Move the JAR file to a folder on the Commerce Platform server, for example:
    /app/oracle/product/aiacs/ATG_TEST/atg_aiacs/client/target
  7. Move the properties file to the same folder or a separate folder, for example:
    /app/oracle/product/aiacs/ATG_TEST/atg_aiacs/client/etc
  8. Edit the properties file to change the default values for your environment as shown in the following table.

    Parameter Default Value Description
    order.properties.filename OrderFileLogger.properties Name of the properties file used to load consumers' product order data.
    profile.properties.filename ProfileFileLogger.properties Name of the properties file used to load consumer profile data.
    catalog.properties.filename CatalogFileLogger.properties Name of the properties file used to load product catalog data.
    properties.file.dir {enter folder} Full path to the folder containing the configuration files for the data warehouse plug-in, for example, /app/oracle/product/atg/ATG/home/servers/atg_dw_loader_lockserver/localconfig.

    Based on the assumption that the data warehouse properties file always stay in the /atg/reporting/datawarehouse/service folder, the additional path for this property should be from the root folder to /atg. For example, if the ATG properties files are at absolute path: D:/Data/ATG/impl/config/atg/reporting/datawarehouse/service, then properties.file.dir should be set to D:/Data/ATG/impl/config.

    Important: Ensure that you don't change the value for the cloudstorage.url property. This value is the autogenerated authentication URL containing the host name of the Oracle Storage Cloud server. It also contains the name of the container used for storing data from Oracle Commerce Platform, and an encrypted access key. To update the encrypted key, reset the URL and download the updated properties file.

Start the Adaptive Intelligence Plug-In

The adaptive intelligence plug-in runs using a Java command that takes three parameters:
  • Name of the properties file

  • Polling period in minutes

  • Properties file folder, an optional parameter that you specify if you have moved the aiacs.properties file to a different folder

To start the plug-in and run it in the background, type a command similar to these examples:

  • UNIX

    java -jar -Dnbo.dir=etc target/ATGIngestionClient.jar aiacs.properties 15 &
  • Windows

    java -jar -Dnbo.dir=etc target\ATGIngestionClient.jar aiacs.properties 15
where:
  • -Dnbo.dir=etc specifies the location of the properties file if it isn't in the same location as the JAR file.

  • 15 is the polling interval in minutes for the thread to pause

Tip: For convenience, you can create a script that will generate a stop script for the process, as shown in this example.
java -jar target/ATGIngestionClient.jar aiacs.properties 15 & aiacsPID=$!
echo $aiacsPID
echo "#!/bin/bash" > aiacs_stop.sh
echo "kill $aiacsPID" >> aiacs_stop.sh
wait

Extracting and Importing Product Price, Path, and Inventory Data

Certain data from Oracle Commerce Platform isn't automatically synchronized as part of the scheduled updates. This data includes image URL, inventory stock levels, and product prices. However, you can use the REST API to import this information by using the product IDs you get to retrieve and update data for them. For example, you can use the POST method to update a product array with groups of attributes and use them in your call to retrieve price, inventory, or path data.

The following table lists some of the attributes you can use in the /offers/rest/v1/products POST method to update existing products.

Data REST API Attributes

Price information

listPrice

salePrice

currency

currencySymbol

Image URL

imageURL

imagePath

Note that imageURL takes precedence over imagePath. ImagePath is only used if site is configured with siteBasePath.

Product URL

productURL

productPath

Note that productURL takes precedence over productPath. ProductPath is only used if site is configured with siteBasePath.

Inventory

stockLevel

The following example payload updates the unsynchronized attributes for products 1000315 and 10065696.

{
  "products": [
    {
      "productId": "1000315",
      "imageURL": "https://www.example.com/img/shoes.jpg",
      "productURL": "https://www.example.com/shoes/product/1000315",
      "stockLevel": 1000,
      "currency": "USD",
      "currencySymbol": "$",
      "listPrice": 29.99
      "salePrice": 19.99
    },
    {
      "productId": "10065696",
      "imageURL": "https://www.example.com/img/boots.jpg",
      "productURL": "https://www.example.com/shoes/product/10065696",
      "stockLevel: 500,
      "currency": "USD",
      "currencySymbol": "$",
      "listPrice": 63.00,
    }
  ]
}

Refer to REST API for Oracle Adaptive Intelligent Apps for CX for more information:

Load Initial Data

Loading initial data imports all consumer profiles, product orders, and catalog details, such as products, promotions, and brands. The process strips personal information about consumers before uploading data by extracting only data mapped to the adaptive intelligence tables. Uploaded data is stored in the output folder you specified when configuring the data warehouse plug-in.

The first time you load data, you must trigger the initial load process in the Commerce Platform component browser by invoking the doWalk method to generate the data files. Subsequent updates are performed using scheduled jobs. Perform the steps outlined in the Oracle Commerce Data Export Utility attachment available on My Oracle Support (document ID 2254809.1).

Important: You must load catalog data before loading order data. Loading in reverse sequence will fail because the order entity type is dependent on the catalog entity type.

Schedule Data Extracts

Configure Oracle Commerce Platform to extract at a frequency that results in files that are typically no larger than 100MB. You can configure rotation using one of these two methods:

  • Automatic rotation based on the number of entries written to the file (export logger property)

  • Scheduled rotation, such as once per hour (component property)

The export loggers are enabled by default and set to rotate automatically after 10000 entries. To reduce the number of entries for automatic rotation, change the value for the dataItemThreshold property in the following properties files to a lower number, such as 1000:

  • ProfileFileLogger.properties

  • OrderFileLogger.properties

  • CatalogFileLogger.properties

To schedule rotation, refer to Configuring a Schedulable Component section of the Oracle Commerce Platform Programming Guide. This guide contains information about using the scheduler and schedule properties and invoking rotation manually on any of the FileLogger components using the rotate() method.

Important: Certain data, such as price, image URLs, and inventory, isn't automatically synchronized as part of scheduled updates. Use the REST API to import this data. Refer to Extract and Import Product Price, Path, and Inventory Data for more information.

Import Data from Other Commerce Applications

Before you load data, you must understand the data structure, data types, and columns to ensure your data properly maps to the adaptive intelligence data model.

To import data from non-Oracle commerce applications, you must follow these general steps:

  1. Map your catalog data model to the adaptive intelligence canonical model.

  2. Create a custom routine to export the data to map into flat files.

  3. Create custom code using the REST API requests to upload data.

  4. Write scheduling jobs to synchronize data on an ongoing basis.

See Field Mapping for information about the canonical data model and required fields for data loading.

See REST API for Oracle Adaptive Intelligent Apps for CX for more information about using the REST API.

Import Data from Files

If you want to import data from Google Shopping or other application whose data you exported, you can use the Data Source page to select your ingestion method for each object you specify. You can select one or a combination of ingestion methods.

  1. On the Data Sources page, set the ingestion method for each data object type when establishing a connection with the data source.

    Tip: If for some reason you need to change the ingestion method later, you can return the Manage Connection page and click Edit.
  2. Get the URL to use to upload data to Oracle Storage Cloud using a call similar to this cUrl example:

    curl -i -X PUT "https://storage_url.oraclecloud.com/v1/Storage-123/idcs_123/cx/offers/tp/google/test.zip?temp_url_sig=a1b2c3&temp_url_expires=9999999999" --upload-file gsdata-03-09-18.zip

The following table lists the object types and source types for importing data.

Note: The Google Shopping ingestion method groups products, brands, and categories into a single catalog file.

Object Type Supported Source Type Comment
CATALOG REST, GOOGLE Includes product, brand, and category data. Derived from the catalog data file.
SITE REST, TSV Must load before any other data unless a site is already defined and if only one site exists in your implementation.
CONSUMER REST, TSV Must import before order data.
ORDER REST, TSV Implicitly includes order lines.
PRODUCT

BRAND

CATEGORY

REST

Individually these entities can only be ingested through the REST API, however, they are collectively ingested with the catalog.

PROMOTION REST Not supported as file import.

Refer to Import Google Shopping Data and Tab-Delimited File Formats for more information.

Import Google Shopping Data

You can use the Data Source page to select Google Shopping as your ingestion method for each object you specify. The Google Shopping ingestion method expects that products, brands, and categories are grouped logically together in a single catalog file before it's uploaded and ingested.

To import Google Shopping data:

  1. On the Data Sources page, in the Available section, click Other Commerce Application.

  2. In the Ingestion Method list, select Google Shopping for each data object you want to import.

Prepare Your Data Files

Although Google enables saving your data in several file formats, you must use a tab-delimited format (either TXT or TSV) when importing data from Google Shopping. The TXT or TSV file is typically then compressed as a GZ, ZIP, or BZ2 file.

It's best practice to create the file using Microsoft Excel, and then save the tab-delimited, plain text file in an ASCII format. Other best practices include:
  • The first line of the file is the header and must contain attribute names as provided in Tab-Delimited File Formats

  • One item per line

  • Don't include trailing tabs at the end of lines

  • Don't include any tabs or line breaks in the attribute values

  • For group attributes, use colon-separated sub-attributes

  • For multivalue attributes, separate each value using a comma wherever applicable

Tab-Delimited File Formats

This topic includes some sample files in tab-delimited format to use for the following data:
  • Site

  • Consumers

  • Orders

  • Catalog

Note that brands and categories are included with products in the catalog data file. Data for promotions isn't supported at this time.

Sites Data
siteId	enabled 	name    	storeFrontURL	consentRequirement	catalogId
siteUS	TRUE    	Site US 	http://mysite	REQUIRED          	CloudCatalog
Consumer Data
consumerId	email	emailAddressSHA256                               	receiveEmail	shippingAddressCountry	shippingAddressState	shippingAddressCity	shippingAddressPostalCode	billingAddressCountry	billingAddressState	billingAddressCity	billingAddressPostalCode	shopperConsent	shopperConsentTimestamp
200002	      	67ba0160562dc07a54f233d8e2684379ab56466b7195be6b6652e60ad1f	TRUE	US	NH	Exeter	3833	US	NH	Portsmouth	23801	GRANTED	2018-03-01T16:00-08:00
290000	      	c48db6b6ebea3392b512eb824f471ad30112324a694bc7af85580d38456	FALSE	US	NY	Syracuse	13202	US	NY	Buffalo	14201	GRANTED	2018-03-01T16:00-08:00
300000	      	b827dbc28170d8507faa215c5b42493bc3e2cb5cfde86d3fbe6777a3d41	FALSE	US	NY	Syracuse	13202	US	NY	Buffalo	14201	GRANTED	2018-03-01T16:00-08:00
500000	plain-text@email.com	                                          	FALSE	US	NY	Syracuse	13202	US	NY	Buffalo	14201	GRANTED	2018-03-01T16:00-08:00
Order Data
orderId	site	status	orderCreationDate	consumerId	promotions	shippingMethod	shippingAddressCountry	shippingAddressState	shippingAddressCity	shippingAddressPostalCode	currency	orderAmount	shippingAmount	taxAmount	billingAddressCountry	billingAddressState	billingAddressCity	billingPostalCode	orderLineId	productId	productQuantity	productListPrice	productSalePrice	productOnSale
xo10002	siteUS	NO_PENDING_ACTION	2011-03-03T16:00-08:00	se-570030		Ground	US	NH	Exeter	3833	USD	301.5	6.5	0	US	NH	Portsmouth	3801	xci1000001	prod10003	1	47.5	0	FALSE
xo10002	siteUS	NO_PENDING_ACTION	2011-03-03T16:00-08:00	se-570030		Ground	US	NH	Exeter	3833	USD	301.5	6.5	0	US	NH	Portsmouth	3801	xci1000002	prod20013	1	129	0	FALSE
xo10002	siteUS	NO_PENDING_ACTION	2011-03-03T16:00-08:00	se-570030		Ground	US	NH	Exeter	3833	USD	301.5	6.5	0	US	NH	Portsmouth	3801	xci1000003	prod20001	1	89	0	FALSE
xo10002	siteUS	NO_PENDING_ACTION	2011-03-03T16:00-08:00	se-570030		Ground	US	NH	Exeter	3833	USD	301.5	6.5	0	US	NH	Portsmouth	3801	xci1000004	prod20011	1	29.5	0	FALSE
xo10005	siteUS	NO_PENDING_ACTION	2011-03-06T16:00-08:00	se-570031		Ground	US	NY	Syracuse	13202	USD	110.49	6.5	0	US	NY	Buffalo	14201	xci1000008	xprod2501	1	54	0	FALSE
xo10005	siteUS	NO_PENDING_ACTION	2011-03-06T16:00-08:00	se-570031		Ground	US	NY	Syracuse	13202	USD	110.49	6.5	0	US	NY	Buffalo	14201	xci1000008	xprod1045	1	88	49.99	TRUE
xo10006	siteUS	NO_PENDING_ACTION	2011-03-06T16:00-08:00	se-570031		Ground	US	NY	Syracuse	13202	USD	106.5	6.5	0	US	NY	Buffalo	14201	xci1000010	xprod1038	1	100	0	FALSE
Google Shopping Catalog Data
id 	title	description	link 	price	sale price	sale price effective date 	brand	condition 	image link 	gtin	product type		adwords grouping 	adwords labels		adwords redirect	availability 	item group id	color	google product category  	mpn	size 	gender	age group 	shipping(price)	min qty
1548	 Suede Loafers	Men's dress shoe, soft suede, handmade upper and lower	http://www.mysite.com/products/shoes/mens/1548.html	149.99 USD	84.99 USD	2018-03-08T05:00-0000/2018-03-11T05:00-0000	Mom's	new	http://www.mysite.com/products/images/1548.jpg	40015481	Shoes	Men's Dress Shoes			in stock				null					1
1548	 Suede Loafers	Men's dress shoe, soft suede, handmade upper and lower	http://www.mysite.com/products/shoes/mens/1548.html	149.99 USD	84.99 USD	2018-03-08T05:00-0000/2018-03-11T05:00-0000	Mom's	new	http://www.mysite.com/products/images/1548.jpg	40015481	Shoes	Men's Dress Shoes			in stock				null					1
1548	 Suede Loafers	Men's dress shoe, soft suede, handmade upper and lower	http://www.mysite.com/products/shoes/mens/1548.html	149.99 USD	84.99 USD	2018-03-08T05:00-0000/2018-03-11T05:00-0000	Mom's	new	http://www.mysite.com/products/images/1548.jpg	40015481	Shoes	Men's Dress Shoes			in stock				null					1
1548	 Suede Loafers	Men's dress shoe, soft suede, handmade upper and lower	http://www.mysite.com/products/shoes/mens/1548.html	149.99 USD	84.99 USD	2018-03-08T05:00-0000/2018-03-11T05:00-0000	Mom's	new	http://www.mysite.com/products/images/1548.jpg	40015481	Shoes	Men's Dress Shoes			in stock				null					1
1548	 Suede Loafers	Men's dress shoe, soft suede, handmade upper and lower	http://www.mysite.com/products/shoes/mens/1548.html	149.99 USD	84.99 USD	2018-03-08T05:00-0000/2018-03-11T05:00-0000	Mom's	new	http://www.mysite.com/products/images/1548.jpg	40015481	Shoes	Men's Dress Shoes			in stock				null					1
1548	 Suede Loafers	Men's dress shoe, soft suede, handmade upper and lower	http://www.mysite.com/products/shoes/mens/1548.html	149.99 USD	84.99 USD	2018-03-08T05:00-0000/2018-03-11T05:00-0000	Mom's	new	http://www.mysite.com/products/images/1548.jpg	40015481	Shoes	Men's Dress Shoes			in stock				null					1

Pause the Ingestion Queue and Delete Data Files

You can pause or resume data ingestion or delete data files that are in the queue waiting to be imported without having to disable ingestion. For example, you might delete a file in the queue because it has old or bad data that you want to correct and then upload again. You can't delete a file that's in process or already ingested. To prevent ingestion from starting on a file in the queue, you can pause ingestion by disabling it.

To pause the ingestion queue or delete files not yet loaded:

  1. On the Data Sources page for your connected commerce application, click View Status > Data Ingestion Status.

  2. Click the object type whose ingestion queue you want to view.

  3. To disable the ingestion queue, click the Disable toggle.

  4. To delete a file, click the Delete icon in the row of the file not yet processed.

Tip: Click Refresh to see any new files uploaded to Oracle Storage Cloud.

View Past Loads

If you are loading files from a third-party commerce application, you can view the history of file loads within a specified period. Viewing history can help you diagnose data load issues and give you a view into the performance of past loads. Using a specified period, you can view the start and end times and other details that can help you with troubleshooting.

  1. On the Data Sources page for your connected commerce application, click View Status > Data Ingestion Status.

  2. Click the object type whose file load history you want to view.

  3. Click the History tab.

Tip: Click Refresh to see the latest data loads.

Field Mapping

This topic lists the required and optional fields in the canonical data model that you will need to map to the schema of your commerce application.

If you're using the data loader plug-in provided by Oracle for extracting data from Oracle Commerce Platform, this field mapping is informational only. If you've adapted your schema, or if you're extracting data from a non-Oracle commerce application, consider which data elements you require for import and map your data accordingly.

The following objects contain the data elements you're most likely to want to import:

  • Sites

  • Products

  • Product Categories

  • Brands

  • Consumers

  • Orders and Order Lines

  • Promotions

Refer to the following sections for the fields to use for data ingestion, their data type, and their equivalent in Oracle Commerce Cloud and Oracle Commerce Platform. Refer to the REST API for Oracle Adaptive Intelligent Apps for CX for additional information.

Note: The field names from Oracle Commerce Cloud and Oracle Commerce Platform are for information purposes only. They may be useful when determining the mapping to use for another commerce application data model.
Sites

The following table describes the available fields for site data. Site data must exist prior to loading other data.

*=Required field.

Field Name Data Type Description
siteId String The unique identifier for the site.
siteName String The name for the site.
minStockThreshold Integer The minimum stock level allowed for products to be recommended.
minPriceThreshold Number The minimum price allowed for products to be recommended.
consentTimeThreshold Integer The consent time threshold to us across all sites in miliseconds.
catalogId String The name or list of names of the catalog of products, brands, and product categories.
enabled Boolean When multiple sites are configured, true if the site is enabled for adaptive intelligence.
storeFrontURL String

The base path of the commerce server for relative URLs.

consentRequirement String The status of the shopper consent requirement for the site: UNKNOWN', 'REQUIRED', 'NOT_REQUIRED'
Product Fields

The following table describes the available fields for product data.

*=Required field.

Field Name Data Type Description Commerce Cloud Commerce Platform (ATG) Google Shopping
productId * String The unique identifier for the product. id id id
name * String The product name. displayName displayName title
description String The product description. description description description
imageURL String The image URL of the product. Derived by prepending the site base path to the imagePath value. N/A N/A imageLink
imagePath String The path to the product image (excluding the http://). For example, /ccstore/v1/images/?source=/file/v6308646299235676733/products/myProduct.jpg. Required for email integrations. primaryFullImageUrl largeImage > url N/A
productURL String The URL to the page that shows the product. Required for email integrations. (Derived from productPath with prepended site base path) N/A link
productPath String The path to the page that shows the product (excluding the http://). route N/A N/A
brandName String Name of the product brand. brand brand brand
tags String A comma-separated list of keywords associated with the product. For example, a T-shirt might have these tags: blue,cotton,short sleeve,men's,outdoor. N/A N/A N/A
categories Array A list of categories that the product belongs to. parentCategories (comma-separated) parentCategories ("item-ref" in the URL shape, not "id") productType
newProduct Boolean If true, the payload will be filtered to include only new products. N/A N/A N/A
inStock Boolean If true, the payload will be filtered to include only products currently in stock. derived from inventory > totalStockLevel N/A derived from availability
stockLevel Integer Number of product items currently in stock. inventory > totalStockLevel N/A N/A
active Boolean If true, then product is active. active N/A N/A
activeDate Integer The date that the product becomes active. arrivalDate dateAvailable > time N/A
expiryDate Integer The date on which the product expires from the catalog. After this date, the product will no longer be recommended. N/A N/A expirationDate
currency String The currency code for the product price in ISO format, for example, USD. priceListGroup > currencyCode N/A price.currency
currencySymbol String The currency symbol for the product price, for example, $. priceListGroup > symbol N/A N/A
listPrice Number The list price of the product. Used as the minimum price if maxListPrice is set to a value higher than the list price. listPrices N/A price.value
maxListPrice Number The maximum list price of the product. N/A N/A N/A
salePrice Number The sale price of the product. Used as the minimum price if maxSalePrice is set to a value higher than the sale price. salePrices N/A salePrice.value
maxSalePrice Number The maximum sale price of the product. N/A N/A N/A
salePriceStartDate String The start date of the product sales period in ISO-8601 format. priceListGroup > startDate N/A salePriceEffectiveDate
salePriceEndDate String The end date of the product sales period in ISO-8601 format. priceListGroup > endDate N/A N/A
saleStatus String The status of the product related to sale. Allowable values: ON_SALE, NOT_ON_SALE, SOME_ON_SALE N/A N/A N/A
boostOrConstrainLevel Integer The whole number representing the adjusted level in the range -1 to +1 that overrides default recommendations. N/A N/A N/A
neverOffer Boolean If true, prevent the product from being recommended. N/A N/A N/A
sites Array Comma-separated list of site identifies associated with the product. N/A N/A
additionalAttributes Array List of additional attributes. N/A N/A N/A
Category Fields

The following table describes the available fields for category data.

*=Required field.

Field Name Data Type Description Commerce Cloud Commerce Platform (ATG)
categoryId * String The unique identifier of the category. id id
parentCategoryId String The identifier of the parent category. Derived from childCategories parentCatalog > id (comma-separated)
code String The code representing the category. N/A N/A
name * String The display name of the category. displayName displayName
description String The category description as it appears on the storefront. description description
imageURL String The image URL of the category. N/A N/A
imageAltText String The hover text for the category image. N/A N/A
productsInCategory String A comma-separated list of product identifiers currently in this category. From childProducts jsons From fixedChildProducts jsons
boostOrConstrainLevel Integer The whole number representing the adjusted level in the range -1 to +1 that overrides default recommendations. N/A N/A
neverOffer Boolean If true, prevent the product from being recommended. N/A N/A
sites Array Comma-separated list of site identifies associated with the category. N/A N/A
additionalAttributes Array List of additional attributes. N/A N/A
Brand Fields

The following table describes the available fields for brand data.

Note: Brand is optional as an object. If brand information is passed with products, the brand will be created if it doesn't already exist.

*=Required field.

Field Name Data Type Description Commerce Cloud Commerce Platform (ATG)
brandId * String The unique identifier for this brand. brand brand
Name * String The brand name. brand brand
imageURL String The image URL of the brand. N/A N/A
imageAltText String The hover text for the brand image. N/A N/A
boostOrConstrainLevel Integer The whole number representing the adjusted level in the range -1 to +1 that overrides default recommendations. N/A N/A
neverOffer Boolean If true, prevent the product from being recommended. N/A N/A
sites Array Comma-separated list of site identifies associated with the brand. N/A N/A
additionalAttributes Array List of additional attributes. N/A N/A
Consumer Fields

The following table describes the available fields for consumer data.

*=Required field.

Field Name Data Type Description Commerce Cloud Commerce Platform (ATG)
consumerId * String The unique identifier of the consumer. id id
shippingAddressCity String The city of the consumer's shipping address. shippingAddress > city N/A
shippingAddressCountry String The ISO 3166–1 country code of the consumer's shipping address. shippingAddress > country N/A
shippingAddressState String The state or region of the consumer' shipping address. shippingAddress > state N/A
shippingAddressPostalCode String The postal code of the consumer's shipping address. shippingAddress > postalCode N/A
receiveEmail Boolean Whether the consumer is willing to receive notifications, true or false. receiveEmail receiveEmail
emailAddressMD5 String The consumer's email address, MD5 hashed. email email
emailAddressSHA256 String The consumer's email address, SHA-256 hashed. email email
billingAddressCountry String The consumer's billing address country code in ISO 3166–1 alpha-2 format. paymentGroups > {0} > country paymentGroups > {0} > country
billingAddressState String The consumer's billing address state. paymentGroups > {0} > state paymentGroups > {0} > state
billingAddressCity String The consumer's billing address city. paymentGroups > {0} > city paymentGroups > {0} > city
billingAddressPostalCode String The consumer's billing address postal code. paymentGroups > {0} > postalCode paymentGroups > {0} > postalCode
shopperConsent String Value indicating the shopper's consent status. Allowable valuesare GRANTED, NOT_GRANTED, and NOT_REQUIRED. N/A N/A
shopperConsentTimestamp Integer Time the consumer set the shopper consent status in milliseconds. N/A N/A
additionalAttributes Array List of additional attributes. N/A N/A
Order Fields

The following table describes the available fields for order data. All fields are optional.

Some important notes about orders and order lines:

  • Don't include order lines with negative product quantities (these are used by merchants as refunds)

  • Where possible, provide order data in chronological order.

  • Break large files into smaller files while still ensuring complete orders with order lines are together within one file.

Field Name Data Type Description Commerce Cloud Commerce Platform (ATG)
orderId String The unique identifier of the order. id id
orderCreationDate Integer The date the order was created in ISO 8601 format. Provide orders in chronological order. creationTime creationTime
consumerId String The unique identifier of the consumer (registered or guest profile ID). profileId profileId
orderLineCount Integer The number of order lines. commerceItems commerceItems
promotions String The comma-delimited list of promotion IDs used on the order. From /ccagent/v1/orders/ + orderId + ?includeResult=full endpoint adjustments > pricingModel > id
status String Free-form status of the order. INCOMPLETE is a reserved status. state stateAsString
shippingMethod String The shipping method of the order. shippingGroups > {0} > shippingMethod shippingGroups > {0} > shippingMethod
currency String The currency code in ISO 4217 format of the amounts of the order. Inherited by all order lines. priceInfo > currencyCode priceInfo > currencyCode
orderAmount Number (Double) The total order amount. priceInfo > total priceInfo > total
shippingAmount Number (Double) The shipping amount. priceInfo > shipping priceInfo > shipping
taxAmount Number (Double) The tax amount. priceInfo > tax priceInfo > tax
shippingAddressCountry String The shipping address country code of the order in ISO 3166–1 alpha-2 format. shippingGroups > {0} > shippingAddress > country shippingGroups > {0} > shippingAddress > country
shippingAddressState String The shipping address state of the order. shippingGroups > {0} > shippingAddress > state shippingGroups > {0} > shippingAddress > state
shippingAddressCity String The shipping address city of the order. shippingGroups > {0} > shippingAddress > city shippingGroups > {0} > shippingAddress > city
shippingAddressPostalCode String The postal code of the order. shippingGroups > {0} > shippingAddress > postalCode shippingGroups > {0} > shippingAddress > postalCode
billingAddressCountry String The billing address country code of the order in ISO 3166–1 alpha-2 format. paymentGroups > {0} > country paymentGroups > {0} > country
billingAddressState String The billing address state of the order. paymentGroups > {0} > state paymentGroups > {0} > state
billingAddressCity String The billing address city of the order. paymentGroups > {0} > city paymentGroups > {0} > city
billingAddressPostalCode String The postal code of the order. paymentGroups > {0} > postalCode paymentGroups > {0} > postalCode
site String The name identifier of the site. Leave blank if only one site is enabled. N/A N/A
orderLines Array The order lines of the order. Self-constructed comma-separated list from commerceItems Self-constructed comma-separated list from commerceItems
OrderLine Fields

The following table describes the available fields for order line data. All fields are optional.

Field Name Data Type Description Commerce Cloud Commerce Platform (ATG)
orderLineId String The unique identifier of the order line.

Order line IDs can be scoped within an order. Two different orders can have the same order line IDs. For example, order A could have order lines 1,2,3 and order B could have order lines 1,2,3,4.

During data ingestion, unique IDs are created by concatenating the order ID with the order line with a dash delimiter. For example, A-1, A-2, A-3, B-1, B-2, B-3, B4

commerceItems > id commerceItems > id
orderId String The unique identifier of the parent order of the order line. id id
productId String The unique identifier of the product of the order line. commerceItems > productId commerceItems > productId
productQuantity Integer (Double) The product quantity. Note that this is rounded to the nearest whole integer. Negative product quantities that indicate refunds are ignored. commerceItems > quantity commerceItems > quantity
currency String The currency code of the amounts of the order. commerceItems > priceInfo > currencyCode commerceItems > priceInfo > currencyCode
productListPrice Number (Double) The product unit list price. commerceItems > priceInfo > listPrice commerceItems > priceInfo > listPrice
productSalePrice Number The product unit sale price if the product is on sale. commerceItems > priceInfo > salePrice commerceItems > priceInfo > salePrice
productOnSale Boolean If true, the product is on sale. commerceItems > priceInfo > onSale commerceItems > priceInfo > onSale
shippingAddressCountry String The shipping address country code of the order line. shippingGroups > {0} > shippingAddress > country N/A
shippingAddressState String The shipping address state of the order line. shippingGroups > {0} > shippingAddress > state N/A
shippingAddressCity String The shipping address city of the order. shippingGroups > {0} > shippingAddress > city N/A
billingAddressCountry String The billing address country code of the order. N/A N/A
billingAddressState String The billing address state of the order. N/A N/A
billingAddressCity String The billing address city of the order. N/A N/A
Promotion Fields

The following table describes the available fields for promotion data. All fields are optional.

Field Name Data Type Description Commerce Cloud Commerce Platform (ATG)
promotionId String The unique identifier for the promotion. id id
typeCode String The promotion type code. type type
type String The promotion type. templateValues > discount_type_value templateValues > discount_type_value
includedProducts String A comma-separated list of product identifiers in the promotion. Derived from XML contents pmdlRule attribute PSC_value > includedProducts
productCount Integer The number of products in the promotion. N/A N/A
name String The promotion name. N/A N/A
description String The promotion description. The value for this field is the text displayed in the widget. description description
enabled Boolean If true, the promotion is enabled. enabled enabled
startDate Integer The date the promotion starts. startDate startDate
endDate Integer The date the promotion ends. endDate endDate
imageURL String The image URL of the promotion. N/A N/A
boostOrConstrainLevel Integer The whole number representing the adjusted level in the range -1 to +1 that overrides default recommendations. N/A N/A
neverOffer Boolean If true, prevent the product from being recommended. N/A N/A
sites Array Comma-separated list of site identifies associated with the promotion. N/A N/A
additionalAttributes Array List of additional attributes. N/A N/A

View Data Ingestion Status

Viewing data ingestion status enables you to monitor the status of data being pulled into the application directly from your commerce system or manually from data files. You can view the number of objects ingested by object type as shown in the following screen capture.

To view data ingestion status:

  1. On the Data Sources page for your connected commerce application, click View Status > Data Ingestion Status.

    Screen capture showing the Data Ingestion Status page

    Tip: Refresh the browser to see the latest data loads. Most data is polled every fifteen minutes. Inventory and stock levels from Oracle Commerce Cloud are checked more frequently, typically every five minutes.
  2. If you are loading files from a third-party commerce application, you can:

    • Click the link in the Start Time column for the object type you want to view the latest load details for that object type.

    • Click an object type to view the files currently in the queue and the history of file loads within a specified period.

    If you're importing data from Oracle Commerce Cloud, you will see only the basic information.

The following table describes the status columns.

Column Description
Data Object The object types appropriate for the connected commerce application. These may vary depending source of the data. For example, for Google Shopping data, products, brands, and categories are combined into a single Catalog object type. Another example is for ingestion of tab-delimited (TSV) data files, where the table lists site data, a prerequisite to other data.

If you ingest data using the data file loader, for example for TSV files, you can view more details about historical data loads.

Last Loaded The date and time of the last successful data load, even if no data is new or changed.
Total Records The total number of records in the corresponding data load.
Skipped Records The number of records excluded from the data load, typically in compliance of any special regulations preventing the ingestion of personal information.
Errors For data loads whose errors are five percent or more of the total records, the number of errors. Errors under five percent are not reported and don't prevent a successful data load.
Status Indicates either completion of the data load (errors under five percent) or failure (errors exceeding five percent).

View Clickstream Status

Viewing clickstream status enables you to monitor the status of events within the last seven days for a specified site. You can view data for various events, such as the number of times users viewed products or logged on to a commerce site.

To view clickstream data:

  1. On the Data Sources page for your connected commerce application, click View Status > Clickstream Status.

  2. Select the site for the clickstream data you want to view.

    Screen capture showing the Clickstream Status page

  3. To view the latest clickstream data, click Refresh.

Tip: If no clickstream events were ingested for an event type within the last seven days, the count shows as zero, even if they were ingested previously. The zero count makes it easy to view which event types have no recent ingestion activity.

The following table describes the clickstream event types.

Event Type Description
Any Page Load Number of times consumers arrive on any page with a page ID.
BlueKai Profile Match Number of distinct consumers who accessed the site and have an existing BlueKai ID associated with their consumer ID.
Category Page Load Number of times consumers loaded a category page on the site.
Content Clicks Number of times consumers clicked a link inside any content area on the site
Home Page Load Number of times consumers loaded the Home page on the site.
Item Added to Cart Number of times consumers added items to their carts.
Item Removed from Cart Number of times consumers removed items from their carts.
Order Confirmed Number of times a product order was confirmed and purchased on the site.
Product Page Load Number of times consumers loaded the Home page on the site.
Search Results Page Load Number of times consumers loaded the Search Results page on the site.
Site Entry Number of times any consumers entered the site.
User Login Number of times consumers logged in to the site.
User Logout Number of times consumers logged out of the site.