Implementing the Push Model for Loading Data

Overview

Oracle Enterprise Command Center Framework supports real-time updates to the dashboard data using a special model of data load: Push Load. Changes in the source system are immediately captured by the Push model by triggering a data load.

The Push model is essential for business use cases that have high-frequency transactions and need to render the dashboards with up-to-date information.

The Push model skips concurrent processing overhead and allows business users to leverage near real-time data so they can make quicker and prompt decisions. An asynchronous HTTP POST Call is required to perform a PUSH request.

Implementing Push Load (Using ECC Client JAVA API)

Create the data set with load type: "Data Load".

Example of Creating a Data Set
Do not include any package or procedure for the data set.

Create the PushRequestHeader object for generating the ECC Client connection.

PushRequestHeader pushRequestHeaderObj = new PushRequestHeader();
        pushRequestHeaderObj.setSystemName(SystemName.EBS);
        pushRequestHeaderObj.setAdditionalParams(additionalParams);
        pushRequestHeaderObj.setCookies(cookies);
        pushRequestHeaderObj.setCustomLoadParams(null);
        pushRequestHeaderObj.setDatasetKey(eccDataset);
        pushRequestHeaderObj.setEccAppShortName(eccAppName);
        pushRequestHeaderObj.setEccServiceBaseURL(eccWebEntyUrl);
        pushRequestHeaderObj.setExternalRequestId(extnRequestId);
        pushRequestHeaderObj.setLastRunDate(new Date());
        pushRequestHeaderObj.setLoadType(loadType);
        pushRequestHeaderObj.setLogLevel(logLevel);
        pushRequestHeaderObj.setLanguages(pLanguages);
        pushRequestHeaderObj.setResetCollection(resetCollectionFlag);

The EccPushClient object is required to push data. Use the call: EccClientUtil.getPushClientService(SystemName.EBS).getPushClient to get the eccPushClient object.
Ways to generate a Push request:
1. Push request using Java objects as the source during a transaction:
  1. Identify the set of attribute keys for the records to be sent by the PUSH request and create the RecordSet object.
  2. Identify the operation to be performed for the set of records. Possible values are the following using LoadOperationType Enum: UPDATE, DELETE, INSERT, UPSERT, REPLACE.
2. Use of a Result Set for a batch operation:
  
  Identify the result set and give the following call passing loadOperation and batchSize of the records to be process in a job. You also need to create a GFMGenerator class object so that gfmToken can be created by the ECC Client to post each batch request.

Tracking data load status

The data load status can be checked using the ECC Client method call, passing the job ID retrieved from the response of the data load request.

Notes for Developers

Please ensure that the ecc-client.zip file and the Jackson libraries from (/u01/R122_EBS/fs1/EBSapps/comn/java/lib) from the central environment are in the build path of the code changes for compilation. The files are listed below:

/u01/R122_EBS/fs1/EBSapps/comn/java/lib/jackson_annotations.zip
/u01/R122_EBS/fs1/EBSapps/comn/java/lib/jackson_core.zip
/u01/R122_EBS/fs1/EBSapps/comn/java/lib/jackson_databind.zip
/u01/R122_EBS/fs1/EBSapps/comn/java/lib/ecc-client.zip

Import the following classes into the client call:

import oracle.ecc.client.Cookie;
import oracle.ecc.client.SystemName;
import oracle.ecc.client.impl.EccPushClient;
import oracle.ecc.client.model.LoadOperationType;
import oracle.ecc.client.model.LoadType;
import oracle.ecc.client.model.LogLevel;
import oracle.ecc.client.model.PushRequestHeader;
import oracle.ecc.client.model.ResponseHeader;
import oracle.ecc.client.util.EccClientUtil;
import oracle.ecc.client.util.GFMGenerator;

To generate GFMToken, use the following code:

GFMGenerator gfmGeneratorObj = new GFMGenerator(connection);
String gfmToken = gfmGeneratorObj.generateNewGFMToken();

To get the list of supported ECC languages, use the following function and class: method:

ArrayList<String> getSupportedLanguages(String eccServiceBaseURL,Map<String, List<String>> additionalParams, SystemName systemName, LogLevel logLevel)
class: oracle.ecc.client.util.EccClientUtil

Note thatgfmToken needs to be added to additionalParamsas described in the table below.

Define attributes of PushRequestHeader:

Attributes of PushRequestHeader
Parameter	Mandatory	Meaning	Possible Values	Impact
SystemName	Yes	The Source System of PUSH Request	EBS	It is important to identify that the request is from the Oracle E-Business Suite system, and the request will be verified using the Oracle E-Business Suite Security Manager.
externalRequestID	Yes	The Source ID of the push request	Long value up to 100 characters	This will be useful for auditing requests from a source.
eccAppShortName	Yes	ECC Application Short Name	Any string matching the ECC application short name	The Push request is executed for the respective application data set.
datasetKey	Yes	ECC Dataset Key	Any data set key of the application identified above	The Push request is executed for the respective data set.
loadType	Yes	Identifies if the load is PUSH or INCREMENTAL OR FULL	LoadType.DATA_LOAD	The value `DATA_LOAD` ensures that the load request is PUSH.
eccServiceBaseURL	Yes	This value is ECC ENTRY URL which is /ecc appended to the Oracle E-Business Suite web entry URL.	The value of the Oracle E-Business Suite entry URL can be identified using profile value "APPS_FRAMEWORK_AGENT"	Sends the request to the appropriate URL.
customLoadParams	No	This is made available to send other parameters to the job.	null	Not in use currently.
cookies	Yes (if no gfmToken used)		EBS Session cookie	Security
additionalParams	Yes (if no cookies sent)	Can be used to pass additional parameters to the URL	(See below)	Security
logLevel	No	Logging level for ECC data load request	LogLevel Enum in ECC Client Default value: ERROR
pLanguages	No	Languages for which data is sent	EBS Session language for which the PUSH request update is triggered Default value: “US”
lastRunDate	Yes ( For Full Load and Incremental Load)	The last run date from which the data has been sent for DATA LOAD		This ensures that the last run date is updated in ECC for the data set.
resetCollectionFlag	Yes (For Full Load only)		true, false	The data set collection is reset and only the new data will be available after the data load.

A possible value for additionalParams is:

gfmToken
Map<String, List<String>> additionalParams = new HashMap<String, List<String>>();
if (null != gfmToken)
additionalParams.put("gfmString", Arrays.asList(gfmToken));

Code Samples

Code sample for PushRequestHeader:

EccPushClient eccPushClientObj = EccClientUtil.getPushClientService(SystemName.EBS).getPushClient(pushRequestHeaderObj);

Code sample for Push Request using Java Objects as source:

List<Map<String,Object>> recordList = new ArrayList<Map<String,Object>>();
List<String> attributeKeyList = new ArrayList<String>();
attributeKeyList.add("ECC_SPEC_ID");
attributeKeyList.add("BRANCH");
attributeKeyList.add("CLASS");
attributeKeyList.add("STUDENT_NAME");
/* US_COURSES is a translated attribute for language US*/
attributeKeyList.add("US_COURSES");
attributeKeyList.add("JOIN_DATE");
Map<String,Object> record = new HashMap<String,Object>();
record.put("ECC_SPEC_ID", "2");
record.put("BRANCH", "CSC");
record.put("CLASS", "2");
record.put("STUDENT_NAME", "RAJNI KANTH");
/* COURSES is multi-value attribute. Hence values are '|' separated */
record.put("US_COURSES", "CS|ICT");
/* dates should be send in canonical format */
record.put("JOIN_DATE", new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'").parse("2018-01-01T00:00:00Z"));
recordList.add(record);
                                                        
RecordSet recordSetObj= EccClientUtil.createRecordSet(attributeKeyList,recordList);
recordSetObj.setOperation(LoadOperationType.REPLACE);
ResponseHeader responseHeaderObj = eccPushClientObj.pushData(recordSetObj);

A code sample for the Use of Result Set for a batch operation:

List<ResponseHeader> responseHeaderList = eccPushClientObj.pushData(LoadOperationType.REPLACE, rs, 1000,gfmGeneratorObj);

The recommended size of a batch job is 1000.

Sample SQL query to generate the result set (similar to the query for PULL model):
```
select * from (select ECC_SPEC_ID,BRANCH,CLASS,STUDENT_NAME,COURSES,JOIN_DATE,LANGUAGE_CODE 
from ECC_CLIENT_ENROL_TEST ) t  
PIVOT ( max(COURSES) as COURSES for LANGUAGE_CODE in ('US' \"US\")) 
```
- If the resetCollection is true, then lastRunDateis required. Else, the request will fail.
- If etl_languages is null when resetCollection is true, it defaults to the base language.
- If reset_collection_flag is set to true, then it is similar to Full Load (PUSH).
- If reset_collection_flag is set to false(default) and last_run_date is defined, it is the same as Incremental Load Push.