Machine Learning Anomaly Scoring
Machine Learning Anomaly Scoring is a method of processing incoming initial measurement data designed to use Artificial Intelligence and Machine Learning neural network models to recognize "normal" versus anomalous measurement values, and only perform Validation, Editing, and Estimation (VEE) processing for measurements deemed to be anomalous. Most measurement data for most customers passes VEE processing without exceptions, but that processing requires considerable computing resources and processing time. Skipping VEE processing for "normal" data greatly reduces the compute resources and processing time needed for measurement processing.
Notes:
- Machine Learning Anomaly Scoring is used with Legacy Measurement Processing and is available with Software-as-a-Service (SaaS) cloud Implementations only. It should only be used for processing certain types of Initial Load measurements for Residential customers. It can not be used for Estimation or Manual measurements or for Commercial or Industrial customers.
- Machine Learning Anomaly Scoring is available under Limited and Controlled Availability for select Early Adopters.
- Machine Learning Anomaly Scoring relies on machine learning models that must be deployed as part of your cloud service implementation. Please contact your Oracle Customer Success Manager if you wish to use Machine Learning Anomaly Scoring.
The Machine Learning Anomaly Scoring Process
Machine learning Anomaly Scoring can be used with specific measuring components or measuring component types, and is defied for measuring components or measuring component types using the Enable ML Based Validation or Enable ML Based Validation (Fallback) flag (respectively).
Initial measurements for measuring components flagged for Machine Learning (via the Enable ML Based Validation flag) are created as usual, but are marked with an Execution Method of "Machine Learning". These measurements are transitioned to the "Waiting for ML" State, and a copy of the measurement is sent to an Object Storage location, where it is picked up by a deep learning/neural network machine learning model. The model processes the data using the incoming measurement data, sample data, and weather data to calculate estimated values for each measurement value in the payload (see Machine Learning Models, below for more details), and then returns the updated data to a different Object Storage location. A Machine Learning payload processing batch process picks up the payload and updates the initial measurement records with the machine learning measurement values as well as the Anomaly Score Threshold specified on the MDM Master Configuration. It then calculates the difference between the original Pre-VEE value and the machine learning value for each measurement value in the payload (this is known as an "Anomaly Score") and compares each to the Anomaly Score Threshold. If any of the Anomaly Scores for any of the measurement values exceeds the Anomaly Score Threshold, the initial measurement is deemed "anomalous", an exception is created, and the measurement proceeds through the normal initial measurement data lifecycle, including VEE processing and creation of final measurements. If none of the Anomaly Scores for any of the measurement values exceeds the Anomaly Score Threshold, the initial measurement moves directly to the creation of final measurements.
Machine Learning Models
The neural network machine learning model used with machine learning anomaly scoring is trained on millions of data points to recognize "normal" versus anomalous usage patterns. When the model runs on your customers' data, it processes the incoming measurement data along with weather data to calculate estimated values for each measurement value in the payload.
Machine Learning Anomaly Scoring includes 3 machine learning models used with specific types of data:
- Residential 1 hour interval data
- Residential 15 minute interval data
- Residential daily scalar data
Enabling Machine Learning Anomaly Scoring
Enabling Machine Learning Anomaly Scoring involves the following:
- Set the Enable ML Based Validation or Enable ML Based Validation (Fallback) flag to "Yes" on the specific measuring components or measuring component types (respectively) you wish to process using machine learning anomaly scoring.
- Configure an SGG ML Payload Processing Configuration extendable lookup value. See Configuring Machine Learning Anomaly Scoring in the Administrative User Guide for more information.
Machine Learning Anomaly Scoring relies on machine learning models that must be deployed as part of your cloud service implementation. Please contact your Oracle Customer Success Manager if you wish to use Machine Learning Anomaly Scoring.