Additional Full Load Features

Online Full Load

Oracle Enterprise Command Center Framework supports online full load to allow users to access the dashboard functionality even during full data load execution. This feature can be utilized to greatly reduce the downtime of ECC dashboards because of full data loads. Online full load works as described in the flowchart below:

Online Full Load Flowchart

the picture is described in the document text

  1. A user submits a full load for a data set with a job ID.

  2. If the onlinefullload flag is disabled, then the existing full load logic is executed.

  3. If the onlinefullload flag is not enabled for the given data set, then the existing full load logic is executed.

  4. If the onlinefullload flag is enabled for the steps above, then the logic for online full load logic is executed.

The steps for online full load are:

  1. Create a new collection with the name <data set>-<job ID> and write data to it.

  2. Create a backup after the writing of the data is completed, with backup name <data set>-<job ID>.

  3. Delete the newly-created collection with the name <data set>-<job ID>

  4. Deleted the old collection with the name <data set>.

  5. Restore the backup with the name <dataset>-<job ID> to collection name <dataset>.

  6. Deleted the backup with the name <dataset>-<job ID> and invalidate the cache for the data set with name <data set>.

Any errors until step 4 will rollback the changes to the data set and users will still be able to use the current version of the data set. Errors after step 4 will not rollback the changes as the data set is deleted in step 4.

Enabling Online Full Load

Follow the steps below to enable online full load:

  1. Set the below properties in configuration properties:

    • Set the property ecc.onlinefulload.isEnabled to true for enabling the online full load. By default, the property is set to false.

    • Add the data sets requiring online full load to property ecc.onlinefulload.enabled.datasets = <dataset1>,<dataset2>. Each data set key has to be separated by ','. There are no data sets mentioned by default.

    • Add the location for creating temporary backup in property ecc.onlinefulload. staging.location = <location of temporary backup for onlinefulload>. No location is mentioned by default.

  2. Restart the ECC managed server to apply the changes.

  3. Validate changes by submitting a full load for one of the data sets enabled for online data load.

Sequential Full Load

Sequential load employs a staggered data load mechanism by segregating the data load execution based on unique values of the attribute set as a sequential load controller. The Sequential Full Load feature serializes the data load by languages, generating fine-grained queries that yield better performance compared to the conventional data load process that uses coarse-grained queries.

This feature an improve the data load performance as it provides better flexibility for customers to load only a subset of essential data rather than the full data.

How Sequential Full Load Works

A conventional full load resets all the data to reflect the state of data in Oracle E-Business Suite, whereas an incremental load identifies the last run date at the data set level, and then fetches the incremental changes from that date to update the Enterprise Command Center data set.

Sequential full load is a composite of full data loads for all the identifiers that form the data set and it sequences the full loads per identifier. Sequential full load ensures that the last run date is set at an identifier level so that you can run an incremental load considering this last run date.

Sequential data load also allows you to set a composite identifier basis, in which the full data loads are sequenced.

Sequential data load is triggered from a wrapper concurrent program. The data load request sends the identifier along with the other standard parameters to the central data load concurrent program. The following steps are performed:

  1. The last run date for each identifier is gathered. Empty identifiers are treated as a null value.

  2. If the last run has no identifier, then the last run date is set for all the identifiers that make a composite identifier.

  3. If a data set has been reset after a run for the identifiers, then the last run date would be considered to have a null value.

Enabling Sequential Full Load

The sequential full load feature is controlled by a profile option in Oracle Application Object Library. This feature has to be enabled per each data set's wrapper concurrent program definition using the steps below:

This component would be a central component similar to how the load Type Value set is defined. The upper limit on identifier length for each job is 240 characters including the separator character in the case of a composite identifier.

The profile option FND: ECC Sequential Data Load Per Data Set Language (code ECC_DATASETS_SERIAL_LANGS) has the following valid values:

For more information on using profile options, see: User Profiles and Profile Options in Oracle Application Object Library, Oracle E-Business Suite Setup Guide.

The wrapper concurrent program has the following characteristics:

For example:

PROCEDURE SUBMIT_ECC_DATA_LOAD (
             ERRBUF          OUT NOCOPY VARCHAR2 ,
             RETCODE         OUT NOCOPY VARCHAR2 ,
             p_system_name        IN VARCHAR2,
             p_dataset_key   IN VARCHAR2,
             p_load_type     IN VARCHAR2,
             p_languages     IN VARCHAR2,
             p_log_level     IN VARCHAR2,
             p_trace_enabled IN VARCHAR2,
             p_org_code      IN VARCHAR2,
             p_force_flag IN VARCHAR2,

In running the wrapper concurrent program, note the following conditions of the parameters:

-- Full Load
   fnd_file.put_line(FND_FILE.LOG,'Submitting request: PIM ECC Data Load for load type l_load_type: 1 || p_load_type) 
     IF (p_dataset_key IS NOT NULL ) THEN 
        l_request_id := fnd_request.submit_request(
               'FND',
               'ECCRUNDL',
               'ECC - Run Data Load',
               sysdate,
               true,
               p_system_name,
               lapp_short_name,
               NVL(p_dataset_key,null),
               p_load_type,
               p.languages,
               p_trace_enabled,
               p_log_level,
               p_org_code,
               null,
               null,
               null,
               null,
               null,
               null,
               null,
               null,
               null,
               null,
               ull,
               null,
               null,
               null,
               p_lastrun_identifier, 
               p_force_flag 
               ):
ELSE

The ECC Data Load Package includes