Export Flow for Incremental Snapshot Types

In Incremental mode, Data Management exports only artifacts that were imported since the last snapshot. Any deleted artifacts are also removed the from the snapshot. Full setup data is always exported.

When the snapshot type is set to Incremental All, Data Management includes only new or changed historical artifacts based on the POV since the last snapshot was exported. It also includes setup artifacts (all old and new POVs) in the output file.

When the snapshot type is set to Incremental, Data Management includes only new or changed historical artifacts based on the POV since the last snapshot was exported. It also includes only setup artifacts and new POVs in the output file.

The following steps describes the export process flow for the Incremental and Incremental All snapshot types:

  1. Purges the /data/snapshots/setup folder.

  2. Exports all setup artifacts to a separate setup folder.

    A separate CSV is created for each table.

  3. Does not purge the /data/snapshots/data folder.

    The system retains the /data/snapshots/data folder "as is" because this is an incremental load.

  4. Deletes orphan POVs.

    Orphan POVs result from deleted period mappings, category mappings, locations, or data load rules.

  5. Stores snapshot metadata information in a SnapshotParams.json file in the data folder.

    The SnapshotParams.json file includes the last exported process id.

  6. Exports data and metadata of the workflow process status for a location, category, and period.

  7. Deletes the files under an /output folder for any POVs that have been deleted after the last export.

  8. Archives the setup and data folders into a ZIP file in the outbox/<filename>.zip folder.

Note:

When the snapshot type is set to INCREMENTAL ALL, all files are included in the /output folder inside the ZIP.

When the snapshot type is set to INCREMENTAL, only the incremental files exported in the current process are included in the under /output folder inside the ZIP.