1. Implementation Guide
  2. Data Loads and Initial Batch Processing
  3. Loading Data from Files
  4. Run Nightly Batches

Run Nightly Batches

As soon as initial seeding is performed (or instead of initial seeding), you need to start nightly batch runs. If you are using the nightly batch to seed positional facts, ensure your first ZIP file upload for the batch has those full snapshots included. Once those full snapshots are loaded through seeding or the first full batch, then you can send incremental files rather than full snapshots.

Nightly batch schedules can be configured in parallel with the history load processes using a combination of the Customer Modules Management (in Retail Home) and the POM administration screens. It is not recommended to configure the nightly jobs manually in POM, as there are over 500 batch programs; choosing which to enable can be a time-intensive and error-prone activity. Customer Modules Management greatly simplifies this process and preserves dependencies and required process flows. Batch Orchestration describes the batch orchestration process and how you can configure batch schedules for nightly execution.

Once you move to nightly batches, you may also want to switch dimension interfaces from Full to Incremental loading of data. Several interfaces, such as the Product dimension, can be loaded incrementally, sending only the changed records every day instead of a full snapshot. These options use the IS_INCREMENTAL flag in the C_ODI_PARAM_VW table and can be accessed from the Control & Tactical Center. If you are unsure of which flags you want to change, refer to the Retail Insights Implementation Guide for detailed descriptions of all parameters.

Note:

At this time, incremental product and location loads are supported when using RDE for integration or when using legacy DAT files. CSV files should be provided as full snapshots.

As part of nightly batch uploads, also ensure that the parameter file RA_SRC_CURR_PARAM_G.dat is included in each ZIP package, and that it is being automatically updated with the current business date for that set of files. This file is used for business date validation so incorrect files are not processed. This file will help Oracle Support identify the current business date of a particular set of files if they need to intervene in the batch run or retrieve files from the archives for past dates. Refer to the System Parameters File section for file format details.

In summary, here are the main steps that must be completed to move from history loads to nightly batches:

  1. All files must be bundled into a supported ZIP package like RAP_DATA.zip for the nightly uploads, and this process should be automated to occur every night.

  2. Include the system parameter file RA_SRC_CURR_PARAM_G.dat in each nightly upload ZIP and automate the setting of the vdate parameter in that file (not applicable if RDE jobs are used).

  3. Sync POM schedules with the Customer Module configuration using the Sync with MDF button in the Batch Administration screen, restart the POM schedules to reflect the changes, and then review the enabled/disabled jobs to ensure the necessary data will be processed in the batch.

  4. Move the data warehouse ETL business date up to the date one day before the current nightly load (using LOAD_CURRENT_BUSINESS_DATE_ADHOC). The nightly load takes care of advancing the date from this point forward.

  5. Close and re-open the batch schedules in POM as needed to align the POM business date with the date used in the data (all POM schedules should be open for the current business date before running the nightly batch).

  6. Schedule the start time from the Scheduler Administration screen > RI schedule > Nightly tab. Enable it and set a start time. Restart your schedule again to pick up the new start time.