Migrating Data Extract Processing from Legacy Measurement Processing to Direct Measurement Processing

When processing data extracts using Direct Measurement Processing, change data capture information is recorded in the Measurement Extract Request table instead of the General Process table. This requires that the Use Measurement Extract Request for Historical Extracts parameter on the MDM Master Configuration be set to "Yes". This applies to all types of data extracts, including:

  • Configurable Consumption Extracts
  • Measurement Data Extracts
  • Initial Measurement Data Header Extracts
  • Aggregated Measurement Data Extracts
  • DataConnect Consumption Extracts (see Oracle Utilities Dataconnect / Opower for more information about DataConnect extracts)

For customers who had been using Legacy Measurement Processing and processing data extracts (in which change data capture information is recorded in the Genera Process table), migrating to using Direct Measurement Processing will involve transitioning out of Legacy-based extract processing and into Direct Measurement Processing extract processing.

There are two approaches to this transition, one involves continuing to use the current batch controls, the other involves creating new batch controls.

Using Existing Batch Controls

To use existing batch controls, do the following:

  1. Pause all Initial Measurement Data Processing.
  2. Run each of the Historical extract processes to process all remaining records in the General Process table.
  3. Enable use of the Measurement Extract Request table by setting the Use Measurement Extract Request for Historical Extracts parameter on the MDM Master Configuration be set to "Yes".
  4. Flush the batch server.
  5. Resume Initial Measurement Data processing.

Creating New Batch Controls

To create new batch controls, do the following:

  1. Duplicate all of the Historical extract batch controls
  2. Update the Initial Measurement Data algorithms and Consumption Extract Types to reference the new batch controls.
  3. Flush the batch server.
  4. Process one final run using the original Historical extract batch controls.
  5. Enable use of the Measurement Extract Request table by setting the Use Measurement Extract Request for Historical Extracts parameter on the MDM Master Configuration be set to "Yes".
  6. Start using the new batch controls for all extract processing.