Creating Payload Processing Batch Controls
The last step is to create payload processing monitor batch controls based on templates. You can create separate batch controls for each head end system or for specific payload types, or any combination of both. For example, you might create a separate batch control for Itron OpenWay usage payloads and device event payloads. Creating specific batch controls allows more control when setting up batch process scheduling for payload processing. For instance, you may want to run usage payload processing every hour, but device payload processing every two hours.
The base package includes the following Payload Processing template batch controls:
- Payload Processing Monitor Template (D1-PLPRM): Used with both Legacy Measurement Processing with Direct Measurement Processing
- Payload Processing Monitor Template with Export (D1-PLPSO): Used with Direct Measurement Processing and required when using Measuring Component Filtering. When used with filtering, this batch process can also send initial measurement payloads to a file system or an object storage location.
- SGG ML Payload Processing Monitor Template (D1-PLMLP): Used with Machine Learning Anomaly Scoring to retrieve measurements following machine learning processing (see Configuring Machine Learning Anomaly Scoring for more information).
To create a payload processing monitor batch control, search for the appropriate template batch control, click Duplicate, enter a unique code for the duplicate batch control and click OK.
This new batch control can be used to process payloads received from head end systems.
Payload Processing Batch Control Parameters
Payload processing batch processing is controlled by the following parameters:
|
Parameter Name |
Description |
Detailed Description |
Required? |
|---|---|---|---|
|
includeFiles |
Include Files |
The pattern for types of files to pick up during polling. The parameter supports the "glob" syntax for using wildcard characters. |
True |
|
fileLocation |
File Location |
The file location or Object Storage bucket to poll files from. This should be a file path or the Value of the File Location Extendable Lookup value for the appropriate bucket (usage or event). See Referencing URIs for more information. |
True |
|
processingConfiguration |
Processing Configuration |
The Value of the SGG Payload Processing Configuration Extendable Lookup value |
True |
|
errorLocation |
Error Location |
The Object Storage bucket where defected portions of payload and related rejection details are stored. This should be the Value of the File Location Extendable Lookup value for the appropriate “error” bucket. |
True |
|
postProcessingAction |
Post Processing Action |
Defines what happens to a file after it has been processed: delete (default), archive or rename. If "archive" is selected, an archive bucket should be provided. |
False |
|
archiveLocation |
Archive Location |
The Object Storage bucket where processed files are moved to. Required if the Post Processing Action is "archive". This should be the Value of the File Location Extendable Lookup value for the appropriate “archive” bucket. |
False |
|
processedFileExtension |
Processed File Extension |
The text to be appended to the file name after the file has been processed. Default is "PROCESSED". |
False |
|
maxRaiseSize |
Maximum Raise Size |
This parameter defines the maximum number of files that the batch program would submit for processing on each polling cycle. Default is No Maximum. |
False |
|
streamOutFileLocation |
Stream Out File Location |
The file location or Object Storage bucket where converted initial measurement are posted for distribution to external systems. This should be a file path or the Value of the File Location Extendable Lookup value for the appropriate bucket (usage). See Referencing URIs for more information. Used with D1-PLPSO. |
True |
|
streamOutFileNamePrefix |
Stream Out File Name Prefix |
The prefix for files created in the Stream Out File Location. The file format will be <Stream Out File Name Prefix><batch run>_<thread>_<thread count>_<timestamp>{<_file sequence>}.json.gz. If the Stream Out File Location is set then this parameter is required. Used with D1-PLPSO. |
True |
|
manifestOption |
Manifest Option |
This parameter specifies that a manifest file be created. See Manifest File for more information. Used with D1-PLPSO. |
False |
|
encryptExternalKey |
Encryption External Key |
The key ring used for the external key provided by the recipient of the file. If populated, the batch output file will be encrypted using a runtime generated key. The generated key will be encrypted using the active public key stored on the key ring. The recipient will use their private key to decrypt the generated key so the file may be decrypted. Only external public key rings may be used. See File Encryption and Signing Keys for more information. Used with D1-PLPSO. |
False |
|
encryptSignatureKey |
Encryption Signature Key |
The key ring for the internal key used to sign the file. If populated, the batch output file will be signed using the active private key stored on the key ring. The assumption is that the corresponding public key has been shared with the recipient. Only internal file encrypting / signing key rings may be used. See File Encryption and Signing Keys for more information. Used with D1-PLPSO. |
False |
|
DIST-THD-POOL |
Thread Pool |
Thread pool name to use if the DEFAULT thread pool is not desired. |
False |