Running the Adapter
The script can be run with following command line options:
nms-DERMSAdapter [-db] [-v] [-debug][-help][-recache]
The following assumptions apply:
1. Location of DER Forecasts, DER Bids, DOE Schedules, and External Load and Gen Forecast Data
• File Based Input: Store CSV files in the directory configured by forecastDir (default: $OPERATIONS_MODELS/derms_forecasts). DOE schedule filenames must contain dermsDOEFileToken (default: _doe). External Load and Gen Forecast filenames must contain externalLoadGenFileToken (default: ExternalForecast). Both tokens are configured in PF_DERMS_ADAPTER_CONFIG; use filenames that identify only the intended file category.
Note: File selection uses the configured column counts and filename tokens. Refer to dermsColumnMap, dermsBidColumnMap, dermsDOEColumnMap, and forecastColumnMap for the corresponding CSV field mappings.
LOAD-GEN_EXTERNAL_INPUT_FILE_FORMAT is no longer used to configure the filename token. Use CSV input for DER bid and DOE schedule data; the -db forecast input tables shown below contain the traditional forecast fields, not the bid or DOE fields.
eVPP schedule input uses CSV or JSON files whose names contain the token configured by vppScheduleFileToken; the default token is _vpp_schedule. The target eVPP must already exist, have Schedule Source set to File, and have an External ID matching the snapshot.
• RDBMS Table Based Input: DERMS forecast updates are available in the following table:
• On operational service bus:
pf_derms_forecast_updates (FORECAST_DATE, RESOURCE_NAME, POWER_SOURCE_TYPE, FORECAST_VALUE)
• On enterprise service bus:
derms_forecast_updates (FORECAST_DATE, RESOURCE_NAME, POWER_SOURCE_TYPE, FORECAST_VALUE)
Note: DOE schedules and External Load and Gen Forecast data must be supplied as CSV files. The -db input option does not load these datasets.
2. Processing the Forecast Data
• File Based Input: After processing, DER forecast, DER bid, DOE schedule, and External Load and Gen Forecast CSV files are moved to the parsed_csvs subdirectory of forecastDir (default: $OPERATIONS_MODELS/derms_forecasts/parsed_csvs).
• RDBMS Table Based Input: The processed rows will be deleted from the input database table.
Each eVPP file supplies authoritative complete schedule snapshots. Each external ID is committed independently. UPDATED snapshots replace the stored schedule, increment its version, and return it to Pending approval. UNCHANGED snapshots preserve approval and duplicate rows. REJECTED snapshots leave the stored schedule unchanged and are written to the DERMS data error log with a VPP-INGEST error code. Processed eVPP files are moved to parsed_csvs. PFService is notified only after a changed snapshot commits successfully.
3. Once the data is validated, DER forecast data is written to the following table:
• On operational service:
Based on the forecast type (rule_value_2) defined by the SRS rules distGenDefault:
• 0 = Weather Affected Fuel Type; write to pf_weather_zone_forecast
• 1 = Single DER Forecast type; write to table pf_der_forecast
• 2 = Demand Response type; write to table pf_demand_reponse_forecast
Staging mechanism is used to load data from input RDBMS table to the run time forecast tables.
4. Once the data is validated, DER bid data is written to the DERMS_BIDS DB table. For duplicate items within a single input file or in the RDBMS table, the first item read will be used. In some instances, these duplicates may be flagged as errors and placed into the error log file.
Note: If running the adapter on the enterprise service bus, the information will be written to pf_derms_forecast_updates.
5. DOE Schedule Processing
Place DOE CSV files with the configured filename token in forecastDir, then run nms-DERMSAdapter without -db. Validated limits are loaded into DERMS_DOE_SCHEDULES_ST and merged into DERMS_DOE_SCHEDULES. The adapter clears the staging data after the merge. Verify the imported resource limits and schedule periods in DERMS_DOE_SCHEDULES and check the adapter error log for rejected input. Importing schedules does not itself force a DOE analysis run.
OPAL example: Run OPAL-createDERMSDOESchedules to create sample DOE CSV files using the configured filename token, then run nms-DERMSAdapter to import them. For an immediate analysis check, use Action any.DMSService force_doe all; otherwise, allow the configured DOE analysis schedule to run.
6. Once the data is validated, External Load and Gen forecast data is written to the pf_gen_load_forecast Table. The data may come from multiple external sources and these are configured in SRS rules(rule_value_1) LoadGenForecastDefault.
• rule_value_integer_1 indicates unique integerKey to the forecast source.
• rule_value_Integer_2 will define the ranking system that determine which to use in the powerflow analysis (low number has higher priority).
• rule_value_integer_5 specifies type of forecast data available at external forecast source(1=only Distgen, 2= only Load, 3= both Load and Distgen).
The OPAL configuration can be examined for examples of how this script is used. In OPAL, the script is called from a parent script (LoadOPALDermsForecasts), which is called from the OPAL‑postbuild script.
Day-Ahead Market Integration
Load DER bid CSV files and DOE schedule CSV files before preparing a day-ahead group offer. The adapter stages bids in DERMS_BIDS_ST and DOE schedules in DERMS_DOE_SCHEDULES_ST before loading DERMS_BIDS and DERMS_DOE_SCHEDULES. Ensure that the input data covers the group's market window.
Group offers are generated by Grid DERMS, rather than loaded by the adapter. DERMS_DER_GROUP_OFFER stores the aggregate proposed and network-constrained capacity limits by group, offer generation, and forecast interval. DERMS_DER_GROUP_OFFER_CURVE stores the associated cumulative capacity/price points.
Configure a project-specific integration to exchange group offers and market awards with the external market. The bid and DOE CSV formats described here do not submit offers to the market or import market awards.
Before running unit commitment for an awarded offer, populate DERMS_UC_AWARDS with the awarded target profile. Each row identifies der_group_id, offer_generation, forecast_date, and target_kw. Match the group, current offer generation, and forecast_date to the corresponding retained group offer. Provide the required interval targets across the market window and convert market quantities supplied in MW to kW (target_kw = awarded MW multiplied by 1000). Awards for a different offer generation cannot be used for the current offer. Unit commitment requires a complete matching target profile.
DERMS_UC_AWARDS stores the target quantity, not a clearing-price field. Do not place market awards in DERMS_BIDS, which stores the individual DER input bids.