Using RDE for Initial Seeding (Gen 2 Architecture)
RDE nightly batch programs can be used to perform initial seeding and full snapshots of positional facts without running any ad hoc processes for data loading. This provides a way to seamlessly transition from history data loads to nightly batch loads.
Prerequisites for starting this process are:
-
You must have already initialized the RAP calendar and performed database partitioning (either by using RDE as described in the prior sections or by loading a calendar file following the Calendar and Partition Setup process).
-
If you will load any historical data for prior dates, you must do that first and come back to this section when you are ready to cut over from history loads to RDE direct integration.
-
The current business date in the data warehouse must be one day prior to the current business date being extracted from MFCS (also known as
VDATE
). If this was not already updated as part of loading historical data, then you must use the ad hoc processLOAD_CURRENT_BUSINESS_DATE_ADHOC
to set it now.
Follow the steps below to perform this transition to nightly batches:
-
Navigate to the System Options in POM for the AIF DATA batch schedule.
-
Update the variables RDE_RunFactVersion and RDE_RunFactODIVersion. By default, they should have a value of I as their rightmost input parameter, which is the normal incremental batch run. Change the value to F, which will trigger a full snapshot batch run. Do not change any other values already in these options except the letter
I
orF
at the end. TheRDE_EXTRACT_FACT_INITIAL_ADHOC
process also uses these parameters to extract either full or incremental datasets, if you want to use that process to extract the fact data outside of nightly batch. -
If you are running the extract outside the Merchandising nightly batch, then disable or skip the interschedule validation jobs (any job starting with
RDE_INTERSCHED*
) as well asRDE_SETUP_INCRMNTL_RESA_JOB
andRDE_SETUP_INCRMNTL_DEALACT_JOB
in the RDE portion of the batch flow that will prevent you from running without MFCS nightly jobs. These jobs look for status updates from MFCS nightly jobs and eventually fail if no updates are found. -
Schedule and run the full Merchandising and AIF nightly batch cycle and validate that all nightly data was processed as expected in your RAP solutions. If you want to validate the RDE extracts prior to running the rest of the nightly batches in RAP (for example, to confirm the full extracts worked as intended) you may place a Hold on one of the first data warehouse load jobs in the schedule, such as
BATCH_START_NOTIFICATION_JOB
.
There are numerous validations and checkpoints in the AIF DATA batch that aim to prevent improper use of the nightly batch cycle by failing the batch when an issue is detected. These jobs are listed below, along with reasons they might fail if this is your first time running a nightly batch cycle.
Job | Reasons for Failure |
---|---|
RDE_BATCHFINAL_CHKBATCHSTATUSSDE_JOB |
Checks the |
ZIP_FILE_WAIT_JOB |
This job is normally disabled when RDE is used; but if it is enabled, it will wait for ZIP files from Object Storage and eventually fail if none are found. If you are not providing any ZIP files, disable the job. |
DAT_FILE_VALIDATE_JOB |
Associated with the ZIP file jobs, and validates that all required files are present and fails if not. If you are not providing any ZIP files, disable this job. |
ETL_REFRESH_JOB |
This should be disabled when RDE jobs are in use; it fails if you have it enabled incorrectly. It
is a check on |
FACT_POSFACT_VALIDATOR_JOB |
Checks that the |
DIM_CALENDAR_VALIDATOR_JOB |
Checks that the incoming calendar is properly formed with no fatal errors in the configuration
of fiscal periods. For MFCS data, this can fail if the system options for the MFCS calendar are out of sync with the actual
data in the |
DIM_PROD_VALIDATOR_JOB |
Checks that the incoming product hierarchy and item data is properly formed and complete. MFCS data can fail several of the validation rules; review the AIF Operations Guide for details. |
DIM_ORG_VALIDATOR_JOB |
Checks that the incoming location hierarchy and store/warehouse data is properly formed and complete. MFCS data can fail several of the validation rules; review the AIF Operations Guide for details. |