Configuring Direct Measurement Proceessing

Direct Measurement Processing is a method of processing incoming initial measurement data designed to reduce the volume of initial measurement records in the database. Rather than create and update initial measurement data records, Direct Measurement Processing creates temporary staging records (in the IMD Control Staging (D1_​IMD_​CTRL_​STAGE) table) and processes initial measurement data in memory and creates final measurements in the Measurement (D1_​MSRMT) table. With Direct Measurement Processing, records in the Initial Measurement Data (D1_​INIT_​MSRMT_​DATA) table are created only if an error is encountered during processing (including VEE processing). See Direct Measurement Processing in the Business User Guide for more information about Direct Measurement Processing.

Configuring the system for Direct Measurement Processing involves the following:

IMD Types

IMD Types are configuration records used to define how initial measurements are processed using Direct Measurement Processing. IMD Types are defined for specific measuring component types in the the "Initial Measurement Creation" processing method (“How to Create MC Related Information”) for each head end system. See Understanding IMD Types for more information about IMD Types.

Configure Extendable Lookups

Extendable lookups are used with Direct Measurement Processing for payload processing as well as with Measuring Component Filtering.

Payload Processing

To configure Payload Processing for use with Direct Measurement Processing, enable the Use IMD Type option on your SGG Payload Processing Configuration extendable lookup values. See Creating SGG Payload Processing Extendable Lookup Values for more information.

Measuring Component Filtering - Pass Through

The Head End Status to Condition Map (D1-HeadEndStatusToConditionMap) extendable lookup is used to map head end systems to head end system status code mapping extendable lookups. This allows Direct Measurement Processing to identify which status code mapping extendable lookup to use based on the head end system associated with the incoming measurement (either specified in the measurement or derived from the measuring component).

Configure Head End System Processing Methods

To configure head end systems for use with Direct Measurement Processing, update the "Initial Measurement Creation" processing method (“How to Create MC Related Information”) to specify the IMD Types to use for your measuring component types for each head end system. See Creating Head End Systems for more information.

If your implementation uses "pass through" external measuring components, the appropriate (interval or scalar) pass through IMD type should be specified for each "pass through" external measuring component type.

Batch Processing

Batch processing is used with Direct Measurement Processing for payload processing, Measuring Component Filtering, and Measuring Component Ordering.

Payload Processing

Direct Measurement Processing receives initial measurement payloads via Payload Processing batch processes based on the following templates:

  • Payload Processing Monitor Template (D1-PLPRM)
  • Payload Processing Monitor Template with Export (D1-PLPSO)

See Creating Payload Processing Batch Controls for more information.

Measuring Component Filtering - Pass Through

When using Measuring Component Filtering, the Extract Passthrough IMDs In Error (D1-EPIMD) batch process can be used to process "Waiting for Extract" records in the Initial Measurement Data (D1_​INIT_​MSRMT_​DATA) table. This batch process uses the following parameters:

  • Maximum IMDs Per File: When extracting data, a new file will be created when this limit is reached. Note: this limit is applied per thread.
  • Stream Out File Location: The file location or Object Storage bucket where initial measurement are posted for distribution to external systems. This should be a file path or the Value of the File Location Extendable Lookup value for the appropriate bucket (usage). See Referencing URIs for more information.
  • Stream Out File Name Prefix: The prefix for files created in the Stream Out File Location. The file format will be <Stream Out File Name Prefix><batch run>_​<thread>_​<thread count>_​<timestamp>{<_​file sequence>}.json.gz. If the Stream Out File Location is set then this parameter is required.

Measuring Component Ordering

When using Measuring Component Ordering, the Process IMD Control Stage Data (D1-IMDCS) batch process can be used to process "Ready to Process" records in the IMD Control Staging table. This batch process uses the following parameters:

  • Value to restrict to IMD Control Stage Status: Designates the status of records to process. When processing Measuring Component Ordering initial measurements, this should be set to "D1RD" ("Ready to Process").

  • Enable sequence handling: Designates (true or false) if initial measurements should be retrieved and ordered for a measuring component in ascending order by date time and create date time. When processing Measuring Component Ordering initial measurements, this should be set to "true".

See Measuring Component Ordering in the Business User Guide for more information.