1. Batch Processing and Web Services Guide
  2. Introduction

1 Introduction

This chapter provides a general overview of integration options for Customer Engagement. This information includes processing, standards, components and business logic, and messaging interfaces.

Overview

This chapter includes the following information:

Note:

In the examples in this guide, user details, names, addresses, emails, or telephone numbers represent a fictitious sample. Any similarity to actual persons, living or dead, is purely coincidental and not intended in any manner.

Anonymous Customer

An anonymous customer is a customer for whom there is no personal information (name, address, phone, or email) recorded. This type of customer is identified by his/her customer ID and Alternate ID. All other types of information can be recorded, including accounts, loyalty points, and transactions.

Processing Mode: Batch XML Files

Batch XML files are used for processing messages in Customer Engagement.

The first line in the file should be similar to the following:

<?xml version="1.0" encoding="UTF-8" ?>

Batch Processing

A bucket in Oracle OCI Object Storage (OS) is created for each customer environment to facilitate uploading XML files for batch uploading. Access to the bucket is through a Pre-Authenticated Request (PAR) retrieved from an ORCE web service added in Release 20.0 called File Services.

A PAR is essentially a URL to a resource in OS that requires no additional authentication to use. The URL can be used to upload or download files from OS, depending on the parameters used in the create PAR request.

Calls to the File Services API use the same authorization model as other ORCE web services. The request must have the Authorization header with encoded value for Basic (deprecated) or OAuth Bearer authorization. The ORCE user or application created in OCI IAM must have an Api Access user Api Access security role.

(For detail on the File Services service query parameters and message body, see Introduction).

For more information about Object Storage, see: https://www.oracle.com/cloud/storage/object-storage-faq.html

Batch files must have an xml extension. The client has the option of uploading individual XML files to Object Storage, which will be imported by ORCE in an ad-hoc manner. There is no guarantee of order or precedence when uploading files individually.

Alternately, the client can zip multiple XML files in a flat folder structure and then upload the zip to Object Storage. ORCE will process the contents of the zip as a single fileset. The names of the XML files within the zip must follow the naming convention shown below to ensure they are processed in the proper order (see the following table).

The valid data type must be included in the name. The following table indicates the proper file names by data type. The wildcard (*) indicates that additional information (for example, client name, timestamp, and so on), can be prefixed and\or suffixed to file names, but they must contain the strings as shown below. Filenames must NOT contain spaces.

Table 1-1 XML File Name Conventions

File Naming Convention Example: client = ABC; Datestamp as Suffix

*itemmaster*.xml

ABC_itemmaster_11:01:01.xml

*hier*.xml

ABC_hier_11:01:01.xml

*card*.xml

ABC_card_11:01:01.xml

*store*.xml

ABC_store_11:01:01.xml

*card*.xml

ABC_card_11:01:01.xml

*customer*.xml

ABC_customer_11:01:01.xml

*poslog*.xml

ABC_poslog_11:01:01.xml

*awards*.xml

ABC_awards_11:01:01.xml

*points*.xml

ABC_points_11:01:01.xml

*user*.xml

ABC_user_11:01:01.xml
Empty XML Tags

When processing batch files, do not include empty XML tags (i.e. <Name></Name>). These empty tags will not prevent the processing of the file, but will generate warnings that are placed in the log files. These warnings may unnecessarily increase the size of the log files.

Symbol Parsing

There are certain symbols that, if used as part of an XML value in a file, will confuse an XML parser into thinking you are trying to do something other than what you are really trying to do. There are two methods that can be used to tell the Parser how to process these symbols: CDATA and Entity. The table below shows the symbols and how to use both methods.

Symbol CDATA* Entity**

<

![CDATA[ ... < ... ]]

&lt;

>

![CDATA[ ... > ... ]]

&gt;

&

![CDATA[ ... & ... ]]

&amp;

’

![CDATA[ ... ' ... ]]

&apos;

"

![CDATA[ ... " ... ]]

&quot;

* The complete value must be included inside the inner brackets.

** Only the symbol is replaced by the entity.

The method you will use in a batch file depends on which type of request you are trying to process.

You may use the CDATA method in any batch file.

You can only use the entity method with the following types:

  • Customer (Add, Update, Card Customer Association)

  • Transaction (Post, Calculate Proforma Points)

  • Stored Value (All)

Date/Time Format

The date/time format used by the Customer Engagement processor is YYYY-MM-DDTHH:MM:SS as shown in the figure to the right.

Any date/time element included in an XML batch file must be in this format.

Figure 1-1 Time Separator

Time separator

Customer Engagement Services Components and Business Logic

Components

The Customer Engagement Services framework is made up of Business Logic components. All of the components are Java classes that implement a common interface. This allows new components to be created based on specific customer needs. The new components can then be plugged in to an existing instance of Customer Engagement without recompiling and redeploying an entire build.

Business Logic

The Business Logic is the heart of Customer Engagement Services. The Business Logic reads the batch file and determines, based on the type of file, what processing needs to be performed on the information found.

Batch Process Tracking

The Batch Process Tracking pages in the user interface enable you to review the activity totals for each batch XML process on a daily, weekly, or monthly basis, including the average time to process a thousand records for each batch type.

Figure 1-2 Batch Process Tracking Page

Batch Process Tracking Page

See the User Guide for more information on these pages.

Batch Import Review

When batch files are processed and errors occur, it is often difficult to determine exactly what happened so the problem can be corrected. Looking in the /failed folder will tell you which records failed, but not why. Looking in the log file can be frustrating if the cause of the error is not obvious.

To help you determine the success or failure of file processing, and the reasons for failures, warnings, and reposts, you can use the Batch Import Review page in Customer Engagement. This page is available through the System menu. The BatchImportAdmin role controls access.

Figure 1-3 Batch Import Review List

Batch Import Review List

Table 1-2 Batch Import Review Page Key

Item Description

1

Search options - Enables you to filter the displayed Reports based on name, date, and/or the results of the processing. See Filter for more information.

2

Actions - Click Actions > Refresh to refresh the list. Click Actions > Currently Processing Files to view a list of the files currently being processed. See "Currently Processing Files" for more information.

3

Batch Import Review List - A list of Batch Import file sets that were processed. See below for more information.

The Batch Import Review list displays the following information for each fileset import Report:

  • Report Name - Identifies a unit of processed files.

  • Record Type - The type of records included in the fileset, such as Award Request, Customer, Item, LoyaltyRequest, MerchandiseHierarchy, RetailLocation, Transaction, or User. See the related chapters in this guide for each record type.

  • Repost Attempt - The total number of attempts at reposting for the fileset. Controlled by System Configuration settings:

    • The Enable Reposting of Customer Transactions property controls whether to make another attempt to repost a transaction if there is a problem with identifying the customer.

    • The Repost Customer Transaction Period property controls how many minutes to wait before attempting a repost, and the total number of repost attempts.

    • The Reject Transactions if Cust Not Found property controls whether to mark the record as a Success or a Failure once the number of repost attempts for a record exceeds this limit.

    See the Customer Engagement Implementation Guide for information on System Configuration properties.

  • Successes - The number of records that were imported successfully.

  • Warnings - The number of records that generated warnings during processing. A warning is not necessarily a failure. Possible warnings include:

    • Customer record not found for a Poslog record.

    • Original transaction not found for post void.

    • Unable to process loyalty data for transaction because no account found.

    • Multiple warnings found.

  • Reposts - The number of records that were earmarked for reposting at the next attempt at processing.

  • Failures - The number of records that could not be processed. Examples of reasons why a failure might occur include:

    • An unknown attribute type in the record.

    • The customer already exists.

    • The transaction already exists.

    • A void request references an original transaction that could not be found.

  • Examined - The number of records that were processed. Might be less than the total number of records in the included files if, for example, processing ended unexpectedly.

  • Start Time - The date and time when processing started for the fileset.

  • End Time - The date and time when processing ended for the fileset.

Currently Processing Files

The Currently Processing Files window displays a count of the files currently in a processing state. A list showing the first ten files ordered by the date and time when processing started is displayed. The list contains the following information:

  • File Name. Identifies the name of the file being processed.

  • Start Time. The date and time when processing started for the file.

  • Examined. The number of records currently processed.

Click Refresh to refresh the list. Click Close to close the window.

Filter

The Batch Import Review list page includes the following fields for filtering.

Figure 1-4 Batch Import Review Filter Options

This image shows the Batch Import Review Filter options.

  • File Name - Filters the displayed fileset reports based on a full or partial File Name.

  • Completed Date Range - Filters the displayed fileset reports based on the beginning of a date range. Defaults to two months before the current date.

  • To - Filters the displayed fileset reports based on the ending of a date range. Defaults to the current date.

  • Successes - Filters the displayed fileset reports based on whether there were any records successfully processed.

  • Warnings - Filters the displayed fileset reports based on whether there were any warnings generated during processing.

  • Reposts - Filters the displayed fileset reports based on whether there were any reposts.

  • Failures - Filters the displayed fileset reports based on whether there were any failures.

  • Examined - Filters the displayed fileset reports based on whether there were any records examined.

Complete each field required to filter the list and click Search.

Report Detail

Click a Report fileset listed at the Batch Import Review list page to open the Report Detail window. You can use this window to review additional details about the processing of the fileset, including errors and warnings.

Figure 1-5 Report Detail Window

This image shows the Report Detail window.

The Report Detail window displays the following information:

  • Report Name - Identifies a unit of processed files.

  • Record Type - The type of records included in the fileset, such as Award Request, Customer, Item, MerchandiseHierarchy, RetailLocation, Transaction, or User.

  • Application Server - The name of the application server where the fileset was available for processing.

  • Start Time - The date and time when processing began.

  • End Time - The date and time when processing ending.

  • Repost Attempt - The total number of attempts at reposting for the fileset.

  • Repost Schedule - The number of minutes to wait before attempting to repost a transaction, and the total number of attempts to make. From the Repost Customer Transaction Period property. Included only for a Transaction record type.

  • Processed Files - Lists the names of each processed file for the fileset. This menu does not filter the information displayed in the window.

  • Failure Summary - Lists the names of each file for which there is a Failure Summary. This menu does not filter the information displayed in the window.

  • Processing Result - Includes the following totals:

    • Total Records Found - The total number of records found for processing in the fileset.

    • Successes - The total number of records successfully processed.

    • Warnings - The total number of warnings generated. A warning is not necessarily a failure. Possible warnings include:

      • Customer record not found for a Poslog record.

      • Original transaction not found for post void.

      • Unable to process loyalty data for transaction because no account found.

      • Multiple warnings found.

    • Reposts - The number of records that were earmarked for reposting at the next attempt at processing.

    • Failures - The number of records that could not be processed. Examples of reasons why a failure might occur include:

      • An unknown attribute type in the record.

      • The customer already exists.

      • The transaction already exists.

      • A void request references an original transaction that could not be found.

    • Examined - The number of records that were processed. Might be less than the total number of records in the included files if, for example, processing ended unexpectedly.

  • Failures tab - Select this tab to have the top panel display a list of links to each failure that occurred. The name of the file that produced the failure is included in the link, as well as the error that occurred and the line number that produced the error. Click the link to position the lower panel to the area in the XML where the failure occurred. This tab is selected by default.

  • Warnings tab - Select this tab to have the top panel display a list of links to each warning that was generated. The name of the file that produced the warning is included in the link, as well as the line number that produced the warning. Click the link to position the lower panel to the area in the XML where the warning occurred.

  • Reposts tab - Select this tab to have the top panel display a list of links to each repost that occurred. The name of the file that produced the repost is included in the link, as well as the line number that produced the repost. Click the link to position the lower panel to the area in the XML where the reposted information occurred.

Note:

The actual XML for a failure, warning, or repost is not displayed for a message that contains customer, user, or transaction data.

Batch Sync Processing

You can use scheduled batch jobs to keep data in Customer Engagement in sync with an integrated system, such as Responsys or Retail Insights. These jobs extract data from Customer Engagement for transmission to the integrating, system, or import data. The data includes:

  • Marketing:

    • Award Accounts

    • Award Coupons

    • Award Transfer Notifications

    • Customers

    • Customer Activity

    • Customer Attributes

    • Customer Deletions

    • Loyalty Accounts

    • Loyalty Points Transfer Notifications

    • Promotions

    • Promotion Award Coupons

    • Promotion Deals

    • Promotion Deal Attributes

    • Promotion Serialized Coupons

    • Promotion Targets

    • Segments

    • Segment Attributes

  • BI/Analytics:

    • Attribute Metadata

    • Customers

    • Customer Address

    • Customer Attributes

    • Customer Segment Categories

    • Households

    • Household Customer Associations

    • Loyalty Accounts

    • Loyalty Account Activity

    • Loyalty Award Transactions

    • Loyalty Programs

    • Segments

    • Segment Attributes

    • Segment Categories

    • Segment Customer Associations

See Marketing Batch Sync Processing and BI/Analytics Batch Sync Processing for more information.

You can also configure a batch import of loyalty activity history through the File Transfers option. See Introduction for background.