Event Impact Measurement and Verification

Event Impact Measurement and Verification (M&V) for Distributed Energy Resource (DER) and Demand Response (DR) programs refers to the overall process of accurately measuring and verifying the actual energy savings or demand reductions achieved by implementing programs of these types, such as rooftop solar panels or battery storage, OR load control devices, like Smart Thermostats or Smart Water Heaters, where customers eventually end up reducing their electricity usage during peak demand periods.

The determination of energy savings achieved for an individual program event using baseline calculation methodologies allows utilities to perform the corresponding financial settlements. Refer to About Program Event Settlement Transactions for more information about how these settlements are calculated.

Event Impact Measurement and Verification (M&V) can be used to determine program-level demand reductions (either achieved or projected to be achieved) and ensure the effectiveness and reliability of these initiatives and can provide transparent data for market participants and regulators. This plays a key role in ongoing assessment and improvement in the program development cycle. At a high-level, this means evaluating the overall effects of a program (load reductions or load increases) related to a particular event or set of events, energy savings (positive or negative) and also cost effectiveness among others. The effects may be determined at the program level or at any level of granularity.

M&V calculations are performed using a type of regression analysis known as "Difference-in-Differences" (or DiD). At a high level the DiD methodology (as defined by Wikipedia), is "a statistical technique used in quantitative research that attempts to mimic an experimental research design using observational study data, by studying the differential effect of a treatment on a 'treatment group' versus a 'control group' in a natural experiment". When applied to demand response events, this analyzes the change in electricity consumption between a group of customers participating in a demand response program (the treatment group) compared to a similar group not participating (control group), both before and after the demand response event, allowing the outcome to isolate the causal effect of the program on electricity usage by accounting for any pre-existing trends in consumption patterns across the groups. This method using Treatment and Control groups is effective for impact estimation of relatively similar or uniform groups of customers, such as residential or small commercial customers , where several hundred or several thousand customers might participate in a program. The method is less effective for evaluating smaller numbers of customers or large commercial or industrial customers, because the treatment-control differences will have too much random error to be reliable.

As noted above, M&V calculations are performed for two different types of groups of customers:

  • Treatment Groups: Groups of customers enrolled in a demand response program, receiving price signals or direct load control during the event. Treatment groups are also referred to as Device Population Groups.
  • Control Groups: Groups of similar customers who are not part of the program and experience normal electricity prices during the event.

Both Device Population (or Treatment) groups and Control groups are defined using Entity Groups. See Understanding Device Population and Control Groups in the Adminitrative User Guide for more information.

As part of the DiD regression calculations, average hourly load is calculated for the following two regression periods:

  • Pre-Treatment: A single period of time, typically 3 hours immediately before and adjacent (without gaps or overlaps) to the Treatment period.
  • Treatment: A period comprising Pre-Event, Event, and Post-Event periods. Pre-Event and Post-Event periods are typically 2-3 hours each, while the Event period itself can up to 8 hours.

More specifically, a load model is constructed for each participant in both the Device Population group and the Control group using the interval meter reads for the Pre-Treatment period and the Treatment period. Subsequently a regression analysis is performed across the data to compute a ratio of the estimated Treatment period average hourly load of the Device Population group assuming no event to the Control group Treatment period average hourly load, and eventually to the Device Population group hourly (kW) Load Impact.

This ratio and other outputs from the regression analysis are captured in M&V Transactions, and additional calculations can be performed using M&V Calculation Rules.

M&V Processing and Dynamic Aggregation

M&V processing leverages functionality Dynamic Aggregation in a number of specific ways.

First, it uses Dimension Scanning to identify changes to Device Population and Control groups, and creates aggregation measuring components for each device population group. This process is run on a regular basis.

Secondly, it uses Find Constituents processing as part of the M&V calculation and regression process to identify the measuring components for both Device Population and Control groups. However, instead of aggregating measurement data for these measuring components, instead it stores the measuring components in a global temporary table called the Entity Group Constituents Temporary Table (D1_​GTT11). The algorithms that execute the regression calculations read the measuring components from this table and aggregates the data for the Device Population group and the Control group.

Both of these processes requires configuration of Data Sources, Dynamic Aggregation Measuring Component Types, Aggregation Groups, and Measuring Component Sets.