Using RDE for Initial Seeding (Gen 2 Architecture)

RDE nightly batch programs can be used to perform initial seeding and full snapshots of positional facts without running any ad hoc processes for data loading. This provides a way to seamlessly transition from history data loads to nightly batch loads.

Prerequisites for starting this process are:

  • You must have already initialized the RAP calendar and performed database partitioning (either by using RDE as described in the prior sections or by loading a calendar file following the Calendar and Partition Setup process).

  • If you will load any historical data for prior dates, you must do that first and come back to this section when you are ready to cut over from history loads to RDE direct integration.

  • The current business date in the data warehouse must be one day prior to the current business date being extracted from MFCS (also known as VDATE). If this was not already updated as part of loading historical data, then you must use the ad hoc process LOAD_CURRENT_BUSINESS_DATE_ADHOC to set it now.

Follow the steps below to perform this transition to nightly batches:

  1. Navigate to the System Options in POM for the AIF DATA batch schedule.

  2. Update the variables RDE_RunFactVersion and RDE_RunFactODIVersion. By default, they should have a value of I as their rightmost input parameter, which is the normal incremental batch run. Change the value to F, which will trigger a full snapshot batch run. Do not change any other values already in these options except the letter I or F at the end. The RDE_EXTRACT_FACT_INITIAL_ADHOC process also uses these parameters to extract either full or incremental datasets, if you want to use that process to extract the fact data outside of nightly batch.

  3. If you are running the extract outside the Merchandising nightly batch, then disable or skip the interschedule validation jobs (any job starting with RDE_INTERSCHED*) as well as RDE_SETUP_INCRMNTL_RESA_JOB and RDE_SETUP_INCRMNTL_DEALACT_JOB in the RDE portion of the batch flow that will prevent you from running without MFCS nightly jobs. These jobs look for status updates from MFCS nightly jobs and eventually fail if no updates are found.

  4. Schedule and run the full Merchandising and AIF nightly batch cycle and validate that all nightly data was processed as expected in your RAP solutions. If you want to validate the RDE extracts prior to running the rest of the nightly batches in RAP (for example, to confirm the full extracts worked as intended) you may place a Hold on one of the first data warehouse load jobs in the schedule, such as BATCH_START_NOTIFICATION_JOB.

There are numerous validations and checkpoints in the AIF DATA batch that aim to prevent improper use of the nightly batch cycle by failing the batch when an issue is detected. These jobs are listed below, along with reasons they might fail if this is your first time running a nightly batch cycle.

Job Reasons for Failure

RDE_BATCHFINAL_CHKBATCHSTATUSSDE_JOB

Checks the C_LOAD_DATES table for any failed RDE extract jobs. This fails if there is at least one extract job that failed. The purpose is to give you a chance to correct RDE failures before you start loading the RAP data warehouse.

ZIP_FILE_WAIT_JOB

This job is normally disabled when RDE is used; but if it is enabled, it will wait for ZIP files from Object Storage and eventually fail if none are found. If you are not providing any ZIP files, disable the job.

DAT_FILE_VALIDATE_JOB

Associated with the ZIP file jobs, and validates that all required files are present and fails if not. If you are not providing any ZIP files, disable this job.

ETL_REFRESH_JOB

This should be disabled when RDE jobs are in use; it fails if you have it enabled incorrectly. It is a check on C_LOAD_DATES statuses at the start of a non-MFCS batch cycle.

FACT_POSFACT_VALIDATOR_JOB

Checks that the DAY_DT values on all positional fact loads (such as Inventory Position) is equal to the current business date only (positional facts do not support back-posting data) and fails if any issues are found. This usually means that the business date in RAP is not correct (at this point in batch, the RAP business date should equal the MFCS VDATE that was extracted by the RDE jobs in this run).

DIM_CALENDAR_VALIDATOR_JOB

Checks that the incoming calendar is properly formed with no fatal errors in the configuration of fiscal periods. For MFCS data, this can fail if the system options for the MFCS calendar are out of sync with the actual data in the CALENDAR table.

DIM_PROD_VALIDATOR_JOB

Checks that the incoming product hierarchy and item data is properly formed and complete. MFCS data can fail several of the validation rules; review the AIF Operations Guide for details.

DIM_ORG_VALIDATOR_JOB

Checks that the incoming location hierarchy and store/warehouse data is properly formed and complete. MFCS data can fail several of the validation rules; review the AIF Operations Guide for details.