Data Import Formats

The Weather Data Adapter can import the weather data to either an NMS database or to a non-NMS database from any of the following sources:

1. Flat CSV Files

2. External Database Table

3. NOAA Weather Service

Flat CSV Files

The adapter can import the data either from a single large CSV file or multiple smaller CSV files. However, they need to have a consistent format. Which means that they all should either have a header row or no header row and they all should have the columns in the same sequence. Projects should make sure that data is given in the same units as in the load profiles. The Adapter does not perform any unit conversions. However, while importing data to a NMS database, adapter can convert the data into profile keys if configured. For example, cloud coverage can be given as a string, 'Cloudy', 'Clear', 'Partially Cloudy' etc. or it can also be given as a solar potential index which varies between 0 and 1, or it can also be given in percentile. Power Flow can only process the indexes. If the data is given as a string or percentile, then adapter converts them to a profile key. In this case, project will have to set the rule_value_integer_4 to 1 in the corresponding 'distGenDefault' SRS-rule, and also have to provide the mapping between the values and the profile keys in the 'pf_weather_data_profile_map' table. If the data value is numerical then it will be mapped to the upper limit of that band. This kind of data conversion is not possible if importing data to a non-NMS database.

Input File Format

CSV files should have following four columns in any order:

• Zone: Name of the zone.

• Time: Date and time, in mm/dd/yyyy HH24:MI:SS format.

• Power Source: An integer representing temperature (-1), solar, and wind power source. This value should be consistent with the rule_value_integer_1 value of 'distGenDefault' SRS-rules.

• Value: Forecasted value.

Configuration

The adapter parameters are configured in the pf_weather_data_config database table, which should be present in the host database. The configuration table should have a row with parameter_name as 'source' and parameter_value_1 as 'CSV'. This configuration parameter tells the adapter the source of the weather data. The rest of the parameters are as follows:

• input_address: Environmental variable OPERATIONS_MODELS should be configured. This variable is set to the path to the directory that hosts the 'weather_data' directory. All input CSV files should be stored in 'weather_data' directory. The adapter creates two more directories in the weather_data directory. The first is 'log' which host the log files, and the second is 'parsed_csvs' which host all the csv files which have been processed by the adapter.

• headerPresent: This parameter should be set to 'yes' if CSV files have a header row present; otherwise, it should be set to 'no'.

• zone_column: It should be an integer specifying the column number of CSV file which contains the zone names. Column numbers start from 0 in CSVs

• datetime_column: It should be an integer specifying the column number of CSV file which contains the date and time.

• power_source_column: It should be an integer specifying the column number of CSV file which contains power_source number

• value_column: It should be an integer specifying the column number of CSV file which contains the value for the corresponding power_source.

External Database Table

The adapter can import the data from an external database table. The connection with the external database should only be made through an Oracle Wallet. Oracle Wallet should contain the connection string and password for the external database. The connection string consists of the database server name, port number and database service name. If the data is imported in an NMS database, the data conversion for solar and wind should be performed. The adapter does not perform data unit conversions while importing from a database table.

Input Table Schema

Similar to CSV format, the source database table should have the following fields:

• Zone: Name of the zone. The data type for this column should be VARCHAR2.

• Time: Date and time. The data type for this column should be VARCHAR2 and time should be given in 'YYYY-MM-DD 24hh:mm:ss' format.

• Power Source: an integer representing temperature (-1), solar, and wind power source. This value should be consistent with the rule_value_integer_1 value of 'distGenDefault' SRS-rules. The data type for this column should be NUMBER.

• Value: forecasted value. The data type for this column should be VARCHAR2.

Configuration

The adapter parameters are configured in the pf_weather_data_config database table (except sourceDBServer parameter, which is configured in CES_PARAMETERS table), which should be present in the host database. The configuration table should have a row with parameter_name as 'source' and parameter_value_1 as 'DB'. This configuration parameter tells the adapter the source of the weather data. The rest of the parameters are as follows:

• sourceDBServer: This parameter needs three values:

• parameter_value_1: Name of the server machine.

• parameter_value_2: Port number on which database service is running.

• parameter_value_3: Database service name.

• sourceDBTableName: The name of the source database table.

• zone_column: The name of the database table field that contains the zone names.

• datetime_column: The name of the database table field that contains the date and time.

• power_source_column: name of the database table field which contains power_source number.

• value_column: name of the database table field which contains the value for the corresponding power_source.

NOAA Weather Service

The adapter is capable of capturing 1 hour, 24 hour, and 7 days of hourly data from NOAA. The adapter pulls the temperature, wind speed and cloud coverage data from the Internet. NOAA provides the data in 3 hour intervals. This adapter automatically interpolates the data according to the FLM granularity provided by SRS rule 'FLM Custom Forecasting' in rule_value_integer_1.

Since the adapter is pulling data from the NOAA weather feeder, this adapter is only applicable to US customers. The adapter works by reading the configured weather zones in the local PF_WEATHER_ZONE database table. The associated latitude and longitude is then used to gather the associated weather data. Therefore, the host database must have this table for the adapter to work. The adapter parses the data and places it in the PF_WEATHER_ZONE_FORECAST database table.

Configuration

To utilize the adapter the following configuration items will need to be completed.

1. proxy: If a proxy is needed to get out of the internal network, the following rule will need to be configured with proxy details in local database table PF_WEATHER_DATA_CONFIG. If no proxy exists this rule does not need to be configured.

Example

INSERT INTO pf_weather_data_config (parameter_name,

parameter_value_1, parameter_value_2, parameter_value_3)

VALUES ('proxy', 'http', 'www-proxy.us.oracle.com', '80');

2. solar_power_source_id: It should be consistent with the corresponding 'distGenDefault' SRS-rule'.

3. wind_power_source_id: It should also be consistent with the corresponding 'distGenDefault' SRS-rule'.

4. temperature_unit: F or C. By default, NOAA gives temperature data in Fahrenheit. However, the adapter can convert the temperatures to Celsius if configured. This unit should be consistent with the unit used in the load profiles.

5. wind_speed_unit: By default, NOAA gives the wind speed in knots. However, adapter converts it to miles per hour (mph) or kilometer per hour (kph) as configured. This unit should be consistent with the unit used in the load profiles.

6. noaa_conn_reattempts: Sometimes the adapter fails to connect with NOAA or NOAA returns inconsistent data. In these cases, this option can be configured to automatically make multiple attempts to connect with the NOAA service. The maximum number of reattempts and the delay between connection attempts are configured using parameter value 1 and parameter value 2, respectively.

Load Frequency

Customers wishing to use the NMS weather adapter to pull NOAA weather service data need to configure the adapter to be invoked on a cron job. Setting up a cron job allows customers to decide the frequency and amount of data to pull at any given time. The adapter can be called for the following forecast modes.

1. Weekly - The adapter will obtain 7 days worth of weather forecast data

2. Hourly - The adapter will obtain weather data for the next 24 hours

3. Current - The adapter will obtain the current weather data if it is run without either the -hourly or -weekly options.

For example, a customer may want to update the weekly forecast daily, the hourly forecast twice a day, and gather the current weather data each hour. Another example would be to run the weekly forecast daily and just run the hourly forecast once in the morning.