Oracle DataRaker Cloud Service

User's Guide

Release 3.8.0.2.0

E73491-01

April 2016

Documentation build: 6.1.2016 13:45:47 [ODR_1464813947000]

Copyright © 2011, 2016, Oracle and/or its affiliates. All rights reserved.

This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.

The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable:

U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government.

This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.

This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle.

Typographical Conventions

The following represents the styles applied to this guide.

Convention

Meaning

Bold

Bold text indicates a user interface element. Examples include window/page titles, keywords, and interactive elements such as menus, buttons, etc.

Hyperlink

A link to another section of the document or to an external document or website.

Add this text

Indicates something the user should enter in a field. Typically rendered as constant width font with a shaded background.

varname

Constant width italic indicates a user interface choice.

constant width type
in a shaded block.

Used for code examples.

constant width

Example code in the midst of regular text.

Note: Note

A note providing a tip, suggestion, or a general information.

Important: Note

A note with additional important information.

CAUTION:

Note

A note of caution.

DataRaker Overview

DataRaker is a SaaS offering (Software as a Service, aka Cloud Services) that provides analytical insight into the most critical business areas of a utility. DataRaker unlocks smart grid data and turns it into actionable insight for electric, gas, and water utilities.

The DataRaker platform has unmatched depth and breadth of proven support for utilities’ most critical business areas. The rapidly expanding availability of smart grid data allows for powerful applications that extend far beyond meter data analytics. DataRaker provides Utilities with quick answers to most pressing questions regarding Meter to Bill, Revenue Protection, Distribution Planning, Demand Response, and Energy Efficiency.

Utilities are also faced with a rapidly evolving industry, fast growing Smart meter deployment, aging infrastructure, dispersed energy generation, and increasing expectations from their customers. Adapting to these factors with existing resources and systems likely requires significant modification to the current business model. The DataRaker application offers utilities a complete toolkit to gain insight into critical business areas and provides actionable results to address, improve, or mitigate a situation. DataRaker renders high-level snapshots of the meter install base’s health, exceptions that are available for review, and detailed information about a specific attribute (such as, a meter, transformer, or feeder). DataRaker also supports customized analytics that are built to meet distinct customer needs.

This documentation describes the features and functionality of the DataRaker system.

Getting Started

Oracle DataRaker Cloud Service is a web portal (client side) user interface (UI) with a back-end analytics engine. The UI is accessible from a customer network or through the Oracle DataRaker Cloud Service VPN (Virtual Private Network). DataRaker end user documentation (i.e., Oracle DataRaker Cloud Service User's Guide and Online Help) focuses primarily on how users interact with the UI, but the responses and actions of the back-end system are described where relevant.

Next in This Section

- Logging In To DataRaker

- Understanding Data

- DataRaker User Interface

Logging In To DataRaker

Environment Requirements

To access the DataRaker UI, you need a supported browser, network access, and DataRaker user credentials (i.e., user name and password).

Access Restrictions

Access to DataRaker is restricted to protect customer data. DataRaker will only respond to server requests coming from IP address ranges specified by customers during implementation or the utility administration. If a request comes from a valid IP address, DataRaker will load the login page otherwise the request will result in an access denied message. If you require access to the environment or access rights within the system, please see your administrator.

Logging In

When launched from a company network or Oracle VPN, the DataRaker URL will load the Oracle Access Manager Login page. To log in, you will need a DataRaker user name and password.

1. From the Login page, enter your user name in the Username field.

2. Enter your password in the Password field.

3. Click Login. Your user name and password will be authenticated by Oracle Access Manager.

Note: If you forget your password, click the Forgot Password link. Enter the required information when prompted, and a temporary password will be generated and emailed to you.

Your user name is linked to a DataRaker Group, which defines your environment and available features.

Note: Group definitions may vary by implementation. Groups are defined with the Administer Groups page, which only administrative users have access to. See Administer Groups for details.

Understanding Data

In the data model, any object that stores associated attribute data is called a Point. The point's attribute data (called Facts) depend on the object's categorization (i.e., Type). For example, an electric meter has a point type of Electric Meter; the meter's facts might include whether it is in service or out of service, readings that were returned, and its manufacturer. When a meter is in service, it has relations to other objects in the model, like its location (Service Point, Premise), the transformer that supplies it, the feeder that supplies the transformer, and the substation that supplies the feeder. All consumption data is based on information coming from the meter and is aggregated upward to its related (Parent) points.

Parent-Child Relationships

DataRaker Cloud Service allows for data points to have a hierarchical association known as a Parent-Child Relationship. As an example, a transformer can have a relationship with the meters that it supplies; therefore, the transformer is the parent and its associated meters are considered children.

Sibling Relationships

Two points that share the same parent are considered to be siblings. Examples include an electric meter and a gas meter at the same premise and transformers on the same feeder.

Parent-Child and Sibling associations are configurable (see Understanding Point Types for more information.) You can see this relationship throughout the system, but two simple illustrations are found in the Drawer Menu Related Points section and the Line Transformer example in Map Panel Options by Type.

Next in This Section

- Understanding Point Types

- Understanding Facts

- Consumption Data Type

Understanding Point Types

The default point types in DataRaker are described in the table below.

Type

Description

Parent Points

Account

Accounts are utility customer accounts in the database.

Bill Cycle, Zip Code

Bill Cycle

Bill cycles are the billing cycles that correspond to the day of the month when the bill is prepared.

N/A

Electric Meter

Electric meters are all of the electric meters in the database.

Account, Bill Cycle, Line Transformer, Premise, Service Point, Zip Code

Feeder

Contains the feeders that are associated with meters.

Substation

Gas Meter

Gas meters in the system.

Account, Bill Cycle, Premise, Service Point, Zip Code

Line Transformer

Contains the transformers that provide service connections for electric meters.

Feeder

Load Profile Class

Contains load profile classes that may be associated with a meter.

N/A

Premise

The address associated with a meter.

Account

Rate

The billing rate charged for consumption of electricity, gas, etc. The rate may vary depending on customer type, time of use, etc.

N/A

Rate Class

Rate class (or customer type) that applies to an account (e.g., residential, commercial, etc.).

N/A

Route

Contains meter reading routes.

N/A

Service Order Agreement

Contains service order agreements for accounts.

N/A

Service Point

Service connection points.

Premise

Utility or Supplier

Contains utilities or suppliers that provide service.

N/A

Zip Code

Contains zip codes in the service area where meters are located.

N/A

For example, data aggregates from an electric meter on two paths:

1. From meter to line transformer, feeder, and then substation.

2. From meter to service point, premise, account, and then either bill cycle or zip code.

Understanding Facts

Any attribute stored for a point is stored as a fact. Facts are broken into three types:

- Standing/Non Time Series (R-A-N: Relation, Attribute, Numeric Attribute): data that does not change often. Normally this data has a start date and an end date will be in the future when a change occurs (e.g., tenant move out or meter exchange).

- Time Series (M-E-L-S-T-C: Metric, Event, List, Segment, Text, Count): consistent recurring time based data such as daily register read or an event.

- Interval (IV: Interval): time series; intervals are shorter than daily (e.g., 15 minute interval reads).

Note: The mnemonic for all fact types is M-E-L-S-T-R-A-N-IV-C

Facts are auto-generated in the DataRaker metadata layer as they are loaded and defined in the relational fact table.

Note: Data types should not be confused with Point Types, which are the categories for objects in the data model.

Fact Data Type

Grouping of Data Types

Fact Example

Metric

Time series

kWh

Event

Time series

Register Gap

List

Time series

No Gas Con

Segment

Time series

Read Collection

Text

Time series

kWh Peak Date

Relation

Standing non-time series

Status

Attribute

Standing non-time series

Address

Numeric Attribute

Standing non-time series

Amps Rating

Interval Metric

Interval

kWh

Interval Text

Interval

Voltage Quality

Count

Time series

Power Out Count

Consumption Data Type

Throughout the system, you will see references to different methods of interpreting data and it is important to understand what these types of data mean. For example, DataRaker validates data daily; if the data fails validation, the system calculates a replacement value based on the validation failure type. For meters that collect register and interval data, interval data will be actual for the time-basis that matches the interval length. Actual Daily Consumption is the consumption value for the 24 hour period ending with a midnight read. In addition, there are times that data needs to be interpolated; if the midnight read time window is 60 minutes, a register read that comes in at 10:16 p.m. would be outside of the window and the daily data would be interpolated.

The following table outlines the main types of data used:

Name

Description

Derived

Derived data is calculated by the difference between the current and the previous day’s midnight read.

Note: Derived data from register reads has precedence over aggregated data from interval reads; if you have register and interval data for the same time period, register data will be available rather than interval data.

Interpolated

Interpolated data is consumption calculated from either non-sequential readings or readings that are not received at the anticipated time.

Aggregated

Aggregated data is the summation of shorter interval data for the selected time-basis.

Estimated

Estimated data is calculated based on the meter's historic usage factor and the usage factor of its peers on a day where a register reading is not received.

Estimates are only calculated for meters that are active and are limited to ninety (90) days. If a subsequent register read is received, estimated value(s) are replaced by interpolated values.

Estimated Usage (Agg)

Estimated Usage (Agg) data is estimated data that is then aggregated to a longer interval.

Best

The Best selection displays available data based on priority:

1. Derived

2. Aggregated

3. Interpolated

4. Estimated

DataRaker User Interface

DataRaker contains a number of usability features that assist users to configure, view, export, and search data. Some of the key features of this system is the structure, layout, and flexibility that I built into the user interface; for example, DataRaker features:

- Web browser user interface

- Configurable portals, zones, and user interface

- Context menus with drill-down capability

- Lists, graphs, and sortable searches

- Ad-hoc filters

- Online help with searchable index

- User display preferences (color, drop-down, graphs, and detailed information)

The following sections describe the key features and functionality of DataRaker’s user interface.

Next in This Section

- Page Elements

- Using the Drawer and Filters Menus

- Menu Bar

- Expand and Collapse Buttons

- Date Controls

Page Elements

DataRaker’s structure includes modular panes that dynamically update based on the page being viewed. The following picture is annotated to highlight the functional areas.

Note:
The above screenshot displays the user interface for a non-power user. A power user will see a similar interface, but will have additional drop-down menus within section two. See the Menu Bar description below for more information.

Name

Description

1. Heading

The Heading section is always visible as you navigate through DataRaker Cloud Service. The heading section begins with the product name. Afterward, the left side of the Heading contains the following:

- Overview button:

, navigates you to the Summary Dashboards where you can view aggregated and summarized information in graphical form. See DataRaker Overview Dashboards for details.

- DataRaker Detail button:

, provides you with different options to search, filter, and export information through an attribute; such as a meter or transformer.

- History button:

provides a drop-down list of the recent pages that was visited during this session. A Back button (less-than symbol) and Forward button (greater-than symbol) provide additional navigation options to review these pages.

The information and links on the right side of the Heading contains the following:

- User: this menu displays the logged-in user's DataRaker user name. As well, expanding the drop-down menu provides an opportunity to logout.

- Get sharable link opens a dialog that allows you to copy the URL to the current page and context.

- Help: launches the online help system.

2. Menu Bar

The Menu Bar offers navigation options to different pages throughout the system. Users commonly have the option to navigate directly to a page through a Menu bar drop-down, or through a link commonly found in the page’s data table.

Navigating to a page through a link in the data table’s results populates the row’s context and displays related information. Navigating to a page through the menu bar, however, will not. When navigating through the menu bar, you have to search or filter the information each time in order to display a specific record.

The menu bar displays different options depending on the user’s rights. Non-power users can view Explore and Export, while power users can view Explore, Manage, Administer, Monitor, and Export.

Note: Group settings and permissions determine the menus that are visible when you log in. Business class users (referred to as non-power users) have a streamlined menu that provides just the features relevant to them.
See Menus for descriptions of menu options.

3. Filters

The Filters section allows you to select the data parameters that determine what functions and data will be displayed. See Filters Menu for more details.

4. Drawer

The Drawer Menu defines high-level parameters about the environment, module, and role that will be used to analyze the data on the first tab. The second tab displays relationship information of parent, sibling, and child points that are associated to the selected data point.

5. Selected Results

The selected results field displays the ID of the object selected in main data pane.

The Links drop-down menu provides list items that allow the user to navigate to MDM (Oracle Utilities Meter Data Management) and CCB (Oracle Utilities Customer Care and Billing), as appropriate.

Note: The Links drop-down only appears when the Filters Type is meter-related (Electric Meter, Gas Meter, Water Meter, etc.)

Additional Details provides more information about the selected data point. See Additional Details for more details.

6. Main Data Pane

The main data pane displays data returned by the system. Data may be returned based on filters or search parameters and then displayed in graphs, tables, or maps, as appropriate.

The panes and options that are available in this section vary depending on the page that is displayed. See the individual page descriptions for details on how the data is presented.

Using the Drawer and Filters Menus

DataRaker provides two main sets of controls for determining what data and corresponding analytics are available to you:

- The Drawer Menu (located on the right): allows you to set global parameters associated to the environment, module, and role.

The Drawer Menu also displays relationship information of parent, sibling, and child points that are associated to the selected data point on the second tab.

See Drawer Menu for more information.

- The Filters Menu (located on the left): allows you to find data based on an object's type (e.g., meter, account, transformer) and its characteristics found by filtering.

See Filters Menu for more information.

Next in This Section

- Drawer Menu

- Filters Menu

Drawer Menu

DataRaker utilizes a combination of modules and roles to organize the data points and facts. Roles allow data and options to be configured based on job functions. The roles available to you depend on the DataRaker module that has been implemented and your assigned group(s), which controls permissions including what functions are displayed.

The following table lists the core modules with their associated roles:

Module

Role

Meter to Bill

AMI Deployment

Billing

Meter Operations

Safety

Revenue Protection

Revenue Protection

Distribution Planning and Operations

Distribution Planning

Demand Response and Energy Efficiency

Demand Response

Energy Efficiency

The Drawer menu is divided by two tabs:

- Environment Filters: defines high-level parameters about the environment, module, and role will be used to analyze the data.

- Related Points: displays related information to the point data that is highlighted in the Main Data Pane.

Environment Filters

The Environment drop-down list allows you to select from environments that are available to you.

If you only have access to one environment, this drop-down menu will be desensitized. Multiple environments may be implemented; common examples may include:

- Instances with separate production and staging environments.

- Instances for holding companies with multiple subsidiary operating companies.

The Module drop-down list provides access to core functions within the system. The base modules come pre-defined with default roles (see the above Modules And Roles table for more information). There are four core modules:

- Meter to Bill

- Revenue Protection

- Distribution Planning and Operations

- Demand Response and Energy Efficiency

The Role drop-down list is available to users who have multiple authorized roles.

Note: Changing the Role will not affect the Type selection made above the Filters Menu.

When a user is only assigned to one role, the Role drop-down list is desensitized. If a user is assigned to multiple roles, however, and they switch among their roles, the selected type in Filters menu’s Type drop-down will not be affected.

Related Points

The Related Points tab is informational only and can be used as a navigation tool. It displays links to the parent, child, and sibling points, as available, that are associated to the highlighted data in the main data panel.

See Understanding Data for more information about data types and parent-child and sibling relationships.

Filters Menu

The Filters menu allows you to find data based on object type and criteria set using one of the filtering controls. This pane, like others, can be displayed or collapsed by clicking the triangle in the title section.

The available options within the Filters pane dynamically update based on a combination of the selections made in the Drawer Menu and with the Type drop-down menu at the top of the pane.

Next in This Section

- Type Selection

- Using Filters

Type Selection

The Type drop-down list allows you to select what data you want to analyze. Relevant data types are configured for each environment. For example, a combined electric-gas utility might have gas and electric meter types, while an electric utility will only have an electric meter type.

If a user is assigned to multiple roles and they switch among their roles, the selected type in this drop-down will not be affected.

See Understanding Point Types for information on the available point types.

Using Filters

The three filter controls allow you to find data based on algorithms developed for the selected object type or using simple or advanced filtering.

- Simple Filter

- Filter by Algorithm

- Advanced Filters

Note: The order that these filters appear are configurable.

All filters fields with text entry fields allow you to perform wildcard searches to return values when partial search criteria is known.

- If you know the beginning of the value, enter it in the criteria field and either leave the rest of the field blank or append a wildcard symbol (* or %).

- If you know the end of the search criteria, enter a wildcard symbol (% or *) and the known text.

- If you know the beginning and end of the search criteria, but not what is in-between, enter the beginning text, a wildcard symbol (* or %), and the end text. For example, you know that the value begins with D1 and ends with 457E, enter D1*457E (or D1%457E) in the criteria field.

- If you know (consecutive) characters in the middle of the value, enter a wildcard symbol anywhere there is a gap.

Simple Filter

As its name suggests, the Simple Filter section provides basic search functionality that allows you to search for a specific entity or group of entities. For non-relational selections, the Simple Filter provides the Filter By drop-down menu and an attribute field.

For relational filters, such as finding meters for a transformer, you need to enter a date in the As of this Date field:

See Understanding Facts for more information on relational data time boundaries.

The following table displays the default Filter By menu options based on the Type drop-down menu selection:

Note: The search functionality is configurable and may be customized through your implementation.

Type

Description

Account

- Account Point ID

- Account ID

- Account Name

Bill Cycle

- Bill Cycle

Electric Meter

- Meter Point ID

- Meter ID

- Meter Name

- Postal Code [Relation]

- Line Transformer [Relation]

- Load Profile Class [Relation]

- Account [Relation]

- Rate [Relation]

Feeder

- Feeder Point ID

- Feeder ID

- Feeder Name

Line Transformer

- Transformer Point ID

- Transformer ID

- Transformer Name

Premise

- Premise Point ID

- Premise ID

- Premise Name

Service Point

- Service Point ID

- Service ID

- Service Name

Zip Code

- Postal Code

Filter by Algorithm

The Filter by Algorithm section provides you with the flexibility to target your query using algorithms applied against a date range. The algorithm sets the "what" criteria (e.g., slow meters, events, data quality parameters, etc.) and the date range sets the "when" criteria; so, for example, during the first quarter of last year (when) which meters reported usage spikes (what).

Note: The filter date range and the date controls in the main data pane are not related. The date controls allow you to view data for a selected point over time (regardless of the filter criteria). For example, if the algorithm returns meters reporting usage spikes in the first quarter of last year, you might look at those meters during the first of quarter of the current year.

The following table identifies the Algorithm Categories associated to each module and role combination:

Module

Role

Algorithm Categories

Meter to Bill

AMI Deployment

- AMI New Meter Health

- Customer Refusal Tracking

- Deployment Data Quality

- Deployment Tracking

- Geospatial Reporting

- Network Tracking

Billing

- Diagnostic Prioritization

- High Bill Prioritization

- Low Bill Prioritization

- Slow Consumption

- Stopped Meters

Meter Operations

- Data Quality Assurance

- Defective Meter

- Defective Socket

- Geospatial Reporting

- Geospatial Outlier

- New Meter Health

- Meter Inventory Tracking

- Monitoring Net Metering

Safety

- Gas Leaks

- Water Leaks

Revenue Protection

Revenue Protection

- Diversion

- Identifying Bypass

- Identifying Meter Swaps

- Identifying Meter Tampering

- Post-deployment Theft

Distribution Planning and Operations

Distribution Planning

- Conservation Voltage Reduction

- Connectivity Verification

- Non-technical Losses Localization

- Transformer Load Management

- Outage Management

- Overload Prevention

Demand Response and Energy Efficiency

Demand Response

- Program Customer Targeting

- Program Efficiency

Energy Efficiency

- Customer Usage Reporting

- Load Disaggregation

Advanced Filters

The Advanced Filters section allows you to create, apply, and export the results from custom made, personalized filters. In addition, you may apply customized and user-specific filters that the Data Scientists created.

The Advanced Filters section allows you to select filters based on Category and Filter Name. The category selections vary based on implementation, but they may include categories defined when a user created a custom filter or categories defined by DataRaker data scientists.

Links

- Edit: opens a dialog that allows you to modify the filter.

- Create New Custom Filter: opens a dialog that allows you to define a new filter.

See Creating and Editing Custom Filters for information on how to create and modify a custom filter.

Next in This Section

- Creating and Editing Custom Filters

Creating and Editing Custom Filters

Creating Custom Filters

You can create and save custom filters through the Create Filter dialog box.

These filters can be reused and their results can be exported. Use the following procedure to add personal filters:

1. From the Filters pane, expand the Advanced Filters section and click the Create New Custom Filter link.

2. In the Create Filter dialog box, enter a descriptive filter name in the Create Filter field.

3. Select the fact type for the filter parameter that you want to create from the drop-down list:

Fact Type

Description

Metric

Allows you to specify a value or range of values of consumption-related data.

Event

Allows you to search for specific types of events in the selected time window. Events may be actual or derived from core calculations.

List

Allows you to search for objects belonging or not belonging to a list (algorithm result).

Attribute

Allows you to search for numeric or text attributes associated with the object.

Relation

A standard relation, such as electric meter to premise, or a segment derived with an algorithm.

4. Depending on the fact type, select the appropriate choices in the next two drop-down menus. These options dynamically change based on the fact type selected.

5. Indicate the value range that you want to use in your search.

Note: If you prefer to search for a specific value, instead of a range, simply enter the same value into both Value Range fields. You may also enter MIN and/or MAX in the respective range boundaries to search for the system's default minimum and maximum value for that fact.

6. Indicate a date range for the data that you want to include in your search.

7. If you need additional criteria, click the + button at the end of the row and add the criteria. Use the logic drop-down list to select conditions (AND, NOT, and OR) for how the filters interact.

- If you have filter criteria where both have to be true, use the AND operator.

- If you have filter criteria where either can be true, use the OR operator.

- If you have filter criteria where one has to be true, but the other cannot be true, use the NOT operator.

8. Click Save.

Editing Custom Filters

The Edit link in the Advanced Filters control allows you to modify currently defined filter criteria and add new parameters. The methods for modifying and adding criteria are the same as those used when creating the filter; however, the Filter Name is not editable.

Menu Bar

The menu bar allows you to navigate by DataRaker functions.

Explore

The Explore menu appears for power and non-power users, and provides access to the Explorer page. This page allows you to visualize time series, event, and geo-spatial data for a selected object (for example, an electric meter). See Explore for more information.

Manage

The Manage menu appears for power-users and allows you to create and edit filters and datasets and run algorithms. See Manage for more information.

Administer

The Administer menu appears for power users and allows you to administer users, roles, and the system. See Administer for more information.

Monitor

The Monitor menu appears for power users and allows you to monitor system activity. See Monitor for more information.

Export

The Export menu appears for power and non-power users, and allows you to view and export data. See Export for more information about the export functions.

Expand and Collapse Buttons

Panes, through out the system, may be collapsed or expanded by clicking the triangular controls on the boundary lines. In order to toggle the panes:

- Click the Collapse Pane button, , to hide the contents of the pane.

- Click the Expand Pane button, , to view the contents of the pane.

The following section further illustrates what panes look like when expanded and collapsed:

- The screenshot below contains three zones (Filter Results, Metric, and Event). In this example, the Metric panel is expanded.

- The following screenshot contains the same three panels (Filter Results, Metric, and Event), except that the Metric panel is collapsed.

Date Controls

Date controls, located in the data panel, allow you to select date-time boundaries for the data that is displayed.

Note: The date controls set the date boundaries for displaying data in the main data pane; this is distinct from filter date ranges, which contribute to the criteria for determining the points returned by the filter. See Using Filters for more information.

Quick date selection buttons:

Date range description:

Date navigation arrows:

Date/Calendar controls:

Next in This Section

- Using Quick Date Selection Buttons

- Navigating Through Dates

- Choosing Custom Date Ranges

Using Quick Date Selection Buttons

Quick date selection buttons allow you to quickly access data within a time range. Options are available for daily (1D, 2D), weekly (1W, 2W), monthly (1 M, 2M), quarterly (1Q, 2Q), and yearly (1Y, 2Y) date selections The default range is two months (2M). When a quick date selection is chosen, the Date/Calendar control will update to show the applicable date range. The end date of the range will always be one day prior to the current day to allow for daily calculations on the AMI data that has been received throughout a day (which is often every 8 hours).

In DataRaker, a week starts on Sunday and ends on Saturday. For the weekly selections, 1W or 2W, the date range will encompass the days in the current week that began on Sunday or the days in the current week plus the previous week, respectively.

Important:

- A day starts at 00:00:01 and ends at 24:00:00

- A week starts on a Sunday and ends on a Saturday

- A month starts with the first day of the month

- Quarters start on January 1, April 1, July 1, and October 1

- Year start with January 1st.

Example

If today is September 9th, 2015:

The date ranges will be:

Control

Beginning

End

1D

September 8, 2015

September 8, 2015

5D

September 4, 2015

September 8, 2015

1W

September 7, 2015

September 8, 2015

2W

August 31, 2015

September 8, 2015

1M

September 1, 2015

September 8, 2015

2M

August 1, 2015

September 8, 2015

1Q

July 1, 2015

September 8, 2015

2Q

April 1, 2015

September 8, 2015

1Y

January 1, 2015

September 8, 2015

2Y

January 1, 2013

September 8, 2015

Navigating Through Dates

The date navigation arrows allow you to move the Calendar control date boundaries by a month. The boundaries may be changed individually or separately.

- Click the single left arrow () to move the start date back by one month.

- Click the single right arrow () to move the end date forward by one month.

- Click the double left arrow () to move the start and end dates back one month.

- Click the double right arrow () to move the start and end dates forward by one month.

Choosing Custom Date Ranges

You may select a custom start date, end date, or both. To set a custom date:

1. Click in either date field to activate the calendar control.

2. Select a month using the previous ( Prev) or next ( Next) buttons.

3. Select the day by clicking the date in the calendar.

If you choose a custom date range, the quick selection buttons will become desensitized and the date bar will show that it is in Custom Date Mode.

Explore

The Explorer page allows you to visualize data for a selected object, and can be opened by navigating Explore > Point > Data. It is also commonly the first page that you see when you log into the system.

The page data is comprised of the Filter Results data table and data panels determined by the object type.

Note: The columns that appear in the Filter Results panel are configurable. The screenshot below displays the default configuration.

The Explorer page dynamically updates with different panels and options (based on the selected data type); panels may be configured to open automatically when an object has been selected. For example, selecting Electric Meter from the Type drop-down list causes the page to add buttons for three panels (Metric, Event, and Map). When the corresponding data loads, the first point is highlighted and the Metric, Event, and Map panels open to display the meter's data.

Next in This Section

- Filter Results Data Pane

- Data Panels

Filter Results Data Pane

Once the Drawer and Filter elements have been defined, the data that is retrieved is displayed in the Filter Results pane. The Filter Results data table is configured to show data relevant to the selected Role and Type.

- Electric Meter Filter Results

- Line Transformer Filter Results

The Filter Results panel contains the following sections:

Fields

- Selected Item: displays the Asset ID of the highlighted item in the data table.
Individual points in the results table can be scrolled through by clicking the greater than (<) or less than (>) symbols on either side of Selected Item field. For example, in the Line Transformer Filter Results image, when the second row is selected, you may scroll to the first or third rows with one click.

When a new row becomes active, all of the data and panels are updated for the new selected item.

- The Links drop-down menu provides list items that allow the user to navigate to MDM (Oracle Utilities Meter Data Management) and CCB (Oracle Utilities Customer Care and Billing), as appropriate.

Note: The Links drop-down only appears when the Filters Type is meter-related (Electric Meter, Gas Meter, Water Meter, etc.)

See Page Elements for more information.

- Additional Details: displays a pop-up window showing more information, as available, about the selected asset. This information includes the following: summary, events Algorithms, relations, non attributes, and Segments. See Additional Details for more information.

- Show number of entries: This drop-down menu allows the user to select the number of entries displayed. Options include: 10, 50, 100, 500, 1000, and All. The default is to display 10 entries. Once another number of entries is selected, the data table is automatically updated.

- Search: allows you to enter search criteria to locate an individual or group of points.

Note: The search field is not limited by the Show Entries drop-down menu or the results displayed in the current Filter Results page. Criteria entered in this field will be applied to the full results set.

Data Table (Electric Meter Example)

The filter data table provides the following information for an electric meter in the product environment:

- Point ID: the points system-assigned, unique identifier

- Asset ID: the asset’s unique identifier

- Asset Status: the asset’s status (Active or Inactive)

- Rate Class: Rate class (or Commodity) that applies to an account (residential, commercial, etc.)

- Address: the street address where the data point resides or was collected

- City: the corresponding city where the data point resides or was collected

- Postal Code: Five (5) digit postal codes (for the United States) in the service area where the data point resides or was collected

Note: For the United States, Zip + 4 may also be used, if provided

Buttons

- List Page Number: if there are more results than allowed by the Show Entries drop-down menu, these buttons will provide navigation options for you to view additional pages. (Please know that these buttons may not display or may be deactivated based on the number of points returned, the number of entries being displayed at a single time, and the current results page that you are displaying; for example, if there are two pages of results available and you are looking at the first page, the First, Previous, and 1 buttons will be deactivated since you are already at the beginning of the list.)

- First: displays the first page of the results set

- Previous: displays the results page immediately before the one that you are viewing

- Page Number: displays a specific page within the results set

- Next: displays the results page immediately following the one that you are viewing

- Last: displays the final results page of the results set

- Export: Export the displayed data as a CSV file.

Next in This Section

- Additional Details

Additional Details

The Additional Details dialog box is accessed through the link in the header of the Filter Results pane. This dialog box contains multiple tabs that display further details about the selected point.

Each of the tabs in this dialog box share similar features and data table columns.

The following sections describe the Additional Information dialog box’s tabs:

Controls and Buttons

With the exception of the Summary tab, each of the tabs share a common heading row that contains the following controls:

- Show number of entries: This drop-down menu allows the user to select the number of entries displayed. Options include: 10, 50, 100, 500, 1000, and All. The default is to display 10 entries. Once another number of entries is selected, the data table is automatically updated.

- Search: allows you to enter a search criteria to filter all of the results, regardless of the number that is currently being displayed. Once the criteria is entered, you must press enter in order to apply the search.

With the exception of the Summary tab, each of the tabs share a common footer row that can contain any of the following elements:

- Previous and Next (set): If there are more results than have been selected to be viewed at one time, the Next and Previous buttons will be displayed. The buttons are disabled when there are no more sets to display using either the Previous or Next button.

- Export: Export the displayed data into a CSV file format.

- Close: Closes the dialog box.

Data Table

The data table contains the following columns:

- Fact Name: The name of the fact that used this data point to build its value.

- Start Date: The date and time that the fact’s algorithm started its process.

- End Date: The date and time that the fact’s algorithm ended its process.

- Value: The fact’s value.

Tabs

Each of the tabs within the dialog box display further details associated to the data point. The tabs include:

- Summary: The summary tab displays the same information that is displayed on the Explorer page’s Selected Asset pane.

- Events: Lists each event that used the associated data point within it’s processing.

- Algorithms: Lists each time an algorithms returned the associated asset within it’s results.

- Attributes: Lists additional information associated to this asset.

- Relations: Displays associations between points; such as a line transformer and its associated meters.

- Num (Numeric) Attributes: data that’s in numbers but not calculated.

- Segments: groups relational facts; such as, customer type (commercial, residential, and industrial).

Data Panels

The Explorer data panels present information for a selected object based on its object type.

Similar to other panels within the system, these can be expanded and collapsed using the toggle button (triangle) on the left of the panel’s title bar. As well, the panels can be reorganized by clicking and holding the panel’s title bar, and dragging and dropping it into the desired location. (Tip: you might find it easiest to reorganize the panels while their contents are collapsed.)

The following panels may be displayed depending on the Module, Role, and Type selections:

- Event Panel

- Heat Map Panel

- Metric Panel

- Map Panel

- Transformer Load Management Panel

Next in This Section

- Common Features and Controls

- Metric Panel

- Event Panel

- Transformer Load Management Panel

- Heat Map Panel

- Map Panel

Common Features and Controls

The Explorer page data panels share controls and behaviors depending on the type of data being displayed.

Next in This Section

- Buttons

- Zooming In and Navigating a Date Range

- Chart Data Hover Text

Buttons

Depending on the view and data being displayed, the panels can have the following controls:

Icon

Title

Description

Help

Opens online help for the panel.

Remove

Remove the panel from the page.

Note:
The panel may be added back to the page by clicking the appropriate add panel button:

Chart View

View the data in a chart. Use to toggle from the grid view to the chart view.

Grid View

View the selected chart data in a segmented data table. Use to toggle from the chart view to the grid view.

Export Data

Export the displayed data. If in chart view, the chart will be exported as a PNG file; if in grid view, the data will be exported as a CSV file.

Zoom

Resets the panel to show the entire date range. See Zooming In and Navigating a Date Range for related information.

Zooming In and Navigating a Date Range

Time series data is initially displayed for the date range selected in the page calendar controls. The zoom bar, which is at the bottom of the panel, allows you to focus on a shorter date range.

To zoom in on a date range in a chart:

1. Click the starting date boundary.

2. Drag to select the zoom date boundaries.

3. The panel will update to only show data for the selected date range.

Note: In addition to the zoom bar, you can select a date range in the consumption chart to zoom, but not the register chart.

Scroll Through Dates

With the initial zoomed view, notice that hovering your mouse over a date (in this case, 15 February 2015) displays summary information.

To view other data using this range, drag the selection bar forwards or backwards. Notice the date has moved.

Resize the Zoom Range

The zoom range may be expanded or contracted using the mouse. When you hover over the starting or ending date boundaries, the pointer will change and you will be able to resize the range.

Reset the Zoom

To reset the chart to the page date range, click the Zoom button ().

Chart Data Hover Text

The data values represented on the charts may be seen by hovering over the data point to display hover text.

For example, hovering over an individual bar in a bar chart will display the date and time recorded, the type of data, and the value:

Hovering over the beginning or end of a static line, such as a statistic, benchmark, or boundary, displays the date and time, the type of data, and the value. The Metric Panel Benchmark is an example of a static line:

Metric Panel

The Metric panel provides consumption and register data for the selected type.

Note: The following information discusses the default configuration and features of the Metric Panel. This panel’s options can be further refined as needed.

Chart View

The consumption chart (top) displays daily data; the register data (middle) displays the daily register value.

Grid View

The grid view displays the data behind the chart view in a table.

To see the data, click the grid view icon ():

Column Heading

Description

Fact Name

The fact name; for example, Aggregated daily kWh from interval data.

Date

The date for the fact.

Value

Provides the Value for the fact listed in a row.

Note: Weather data will be included in the grid view if it is selected from the Weather drop-down list.

Each column can also be sorted in ascending or descending order.

To switch back to the chart view, click the chart view icon ().

Next in This Section

- Metric Panel Data by Type

- Options

Metric Panel Data by Type

Metric panel data is dependent on the selected type.

Type Selected

Data Available

Electric Meter

Gas Meter

Consumption data for the selected meter. Each vertical bar represents consumption; the bar color corresponds to the data type.

Premise

N/A

Feeder

Line Transformer

Feeder and Line Transformer display aggregated daily consumption in kWh.

Note: Data is only available when meter to line transformer and line transformer to feeder relationships are available in the connectivity model.

Bill Cycle

Zip-Code

Rate

Commodity

Meter data is aggregated to the selected type.

Options

The Metric panel allows you to modify the conditions under which the chart is rendered. You may change the following parameters (from left to right):

Option

Possible Values

Default Value

Time-Basis

Monthly, Weekly, Daily, Hourly, 30 Minute, 15 Minute, Actual

Daily

Units of Measure

(Electricity) kWh, Volt; (Gas) CCF; (Water) CF

Depends on the Type. For example, kWh is the default for Electric Meter.

Chart Type

All, Best, Double, Single, Stacked

Best

Data Type 1, Data Type 2

Aggregated, Estimated, Derived, Interpolated

Note: Data Type 1 drop-down appears when the Chart Type is Single, and Data Type 1 and 2 appear when the Chart Type is Double.

Data Type 1: Estimated

Data Type 2: Interpolated

Benchmark

LP Class, Transformer

Weather

Average Daily Temp F, Max Daily Temp F, Min Daily Temp F, Precipitation rate.

Statistics

Average, Maximum, Minimum, Median, 10th Percentile, 90th Percentile, One Std Deviation, Two Std Deviation

Last Year

Checkbox selected or unselected

Unselected

Next in This Section

- Time Basis

- Units of Measure

- Chart Type

- Benchmark

- Weather

- Statistics

- Last Year

Time Basis

The time basis drop-down list allows you to view consumption over different data read intervals. If you select an interval shorter than the minimum time between reads, the chart will not display data. For longer intervals, you will see aggregated data. The Actual option provides register read values.

Example: Meter Data Provided Daily

Selection

Consumption Chart Behavior

15 Minutes

No data will be displayed since 15 minute data does not exist.

Hourly

No data will be displayed since hourly data does not exist.

Daily

Chart will display data for each day, which will be the difference between that day's register read value and the register read value from the previous day.

Weekly

Chart will display data for each week, which will be aggregated from the daily data during that week.

Monthly

Chart will display data for each month, which will be aggregated from the daily data during that month.

Actual

Chart will show the register read value for each day.

Units of Measure

The units of measure drop-down list allows you to select the type of consumption data to view. The units of measure options are dependent on data source (i.e., electric, gas, or water meter) and object type.

- Electricity: kWh, Volt

- Gas: CCF

- Water: CF

Consumption data options for electricity, water, and gas will typically include Usage Factor, which is a ratio of the selected object's consumption to the average consumption of objects in its rate class. Usage factor is calculated daily, and, therefore, varies on a daily basis. For example, a meter with a usage factor of 1 is consuming exactly the same amount as the class average while a meter with a usage factor of 0.6 is consuming forty percent less than the class average. Usage factor is useful in separating the behavior of a particular meter from the overall behavior of its rate class.

Note: Additional units of measure may be available, depending on implementation.

Chart Type

The chart type drop-down list allows you to change how data in the consumption chart is plotted. Options include:

- Best

- Single

- Double

- Stacked

- All

Best

This chart displays the highest priority data available. See Consumption Data Type for more information.

Single

This chart displays a bar chart that allows the user to select one data type that is associated to the asset; such as estimated consumption.

Double

This chart displays a bar chart that allows the user to select two data types that is associated to the asset; for example, you can display derived and estimated consumption.

Stacked

This chart displays a bar chart style that displays all of the related values on top of each other; for example, a meter's load as part of a transformer's total load for each day.

All

This chart displays the different chart options overlaid upon each other.

Benchmark

The Benchmark drop-down list allows you to compare a meter against meters sharing an attribute or relation. For example, it might be comparing against all meters in the same rate class or connected to the same transformer. The benchmark is displayed as the 10% and 90% value range.

Weather

The Weather drop-down list allows you to view weather data over time. The options available are dependent on what weather data exists in the database. The selected weather parameter is plotted against the second y-axis. For example, if temperature, humidity, or dew point were available, they would be plotted against degrees, percentage, or degrees, respectively.

Statistics

The Statistics drop-down list allows you to view statistics for the displayed data. The statistic values are rendered as lines that span over the time window (as selected with the date control).

Options include:

Statistics Selection

Description

Example

Minimum

The minimum time-basis consumption value during the time window.

Maximum

The maximum time-basis consumption value during the time window.

Median

The middle value of the data in the selected time window. All numbers are sorted and the value in the middle is the medium.

For example, if you have seven numbers {10, 11, 13, 15, 16, 23, 26}, the medium is 15. If the median has to be found from an even number of values, the median is the average of the middle 2 values.

Average

Average is the arithmetic mean, which is the sum of all values divided by the number of values.

10th Percentile

The bottom 10 percent of values in a population.

90th Percentile

The top 90 percent of values in a population.

One Std Deviation

+/- one standard deviation from the average, which contains 68% of all values in the population.

Two Std Deviations

+/- two standard deviations from the average, which contains 95% of all values in the population.

Last Year

The Last Year selection allows you to compare consumption in a selected time period to consumption in the previous year (if the data exists). The last year data is shown as an area highlighted behind the bar chart.

Event Panel

The Event panel displays event flags for the selected object. Events are categorized as one of the following:

- Meter Events received from data collection.

- Account Events, such as move-in/move-out, customer call to call center, etc., that are provided by a customer information system (CIS),

- Derived Events that are identified by DataRaker core processes.

- Lists that are the results of an analytic calculation or test.

Derived Event Definitions

Flag Name

Flag Description

Association

Derived Usage Spike

An event to indicate that the usage calculated by using register reads exceeds a threshold value set as spike.

EM, GM, WM

Estimated Usage

An event to indicate that usage was calculated by estimation. An estimated usage is calculated when today's read is missing and no future read exists.

EM, GM, WM

Estimated Usage Spike

An event to indicate that the estimated usage exceeds a threshold value set as a spike.

EM, GM, WM

Interpolated Usage Spike

Flag created when interpolated usage exceeds 10% of the max dial count.

EM, GM, WM

Meter Lock Status

Shows whether the meter is locked or unlocked.

EM, GM, WM

Negative Aggregated Usage

An event to indicate that the usage calculated by aggregating hourly interval is less than 0 (negative).

EM, GM, WM

Negative Derived Usage

An event to indicate that the usage calculated by using register reads is less than 0 (negative).

EM, GM, WM

Negative Estimated Usage

An event to indicate that the usage calculated by estimation is less than 0 (negative).

EM, GM, WM

Negative Interpolated Usage

An event to indicate that the usage calculated by interpolation is less than 0 (negative).

EM, GM, WM

Negative Register

An event to indicate that the register read obtained is less than 0 (negative).

EM, GM, WM

NIC Power Down

An event to indicate that a NIC (network interface card) is powered down and the meter is not communicating with the network.

EM

Register Gap

An event to indicate that the register missed a read during the day.

EM, GM, WM

Register Roll-over

An event to indicate that the register has rolled over beyond the max read that it can register (based on dial count).

EM, GM, WM

Register Roll-over Spike

An event to indicate that the register has rolled over beyond the max read that it can register (based on dial count) also the roll over value is significantly higher that it exceeds a threshold value set as spike.

EM, GM, WM

Reverse Register

An event to indicate that the register read obtained is less than previous register thus indicating a reverse in register.

EM, GM, WM

Spike Register

An event to indicate that the register read obtained exceeds a threshold value set as spike.

EM, GM, WM

Static Register

An event to indicate that the register is not reporting any read and it static (compared to previous register values).

EM, GM, WM

Status Active

Show the status of the meter (active, inactive)

EM, GM, WM

Usage Aggregation Spike

An event to indicate that the usage derived by aggregating hourly interval exceeds a threshold value set as spike.

EM, GM, WM

Zero Register

An event to indicate that the register value obtained is 0.

EM, GM, WM

Zero Consumption

An event to indicate that there was no consumption.

EM, GM, WM

Derived Event Definitions for Transformers

The following derived events are available in the Distribution Planning and Operations module. They can be viewed in when the Line Transformer is selected in the Type Filter.

Flag Name

Flag Description

Association

Overload Flags

- Overload High %

- Temp Adjusted Overload High %

- Overload Low %

- Temp Adjusted Overload Low %

Determines whether a transformer is overloaded based on the value being held in Interval Overload Amount.

There are two thresholds because associated to this flag to identify low and high overloading.

- Low: Typically, configured as over 100% of the transformer’s load rating (this threshold can be changed base on business needs).

- High: Typically, configured as 165% of the transformer’s load rating (this threshold can be changed base on business needs).

TX

Line Transformer Power Out

When power was disrupted at the transformer level.

TX

Chart View

The panel displays all available flags (top chart) and total flags observed (bottom chart) over time.

Note: Flags are indicated by diamonds.

The panel may also display event-like data elements such as a state or lock status. A state flag could indicate, for example, whether a selected object is active or inactive. A locked status could indicate if the object was locked (at the meter or pole). The image below displays a closer view of the Event Key:

Note:

1. Locked statuses include cut (C), locked (L), locked at pole (P), unknown (N/A), and unlocked (U). Lock information can be useful in identifying theft.

2. Status relates to the selected type. For example, if Meter is selected, the flag will display whether the meter is active or inactive; if Account is selected, the flag will display whether the account is active or inactive. Status is always available for meters, but may not be available for other types.

Grid View

To see the data behind the chart, click the grid view icon ().

Column Heading

Description

Fact Name

The fact name; for example, Installation Date.

Date

The date and time that the event occurred.

Value

Provides the Value for the fact listed in a row.

Each column can also be sorted in ascending or descending order.

To switch back to the chart view, click the chart view icon ().

Next in This Section

- Options

Options

The Events portal contains two drop-down lists that allow you to choose the time window and types of events to display.

Time-Basis

The time-basis drop-down list allows you select what data increments to display. The options are Daily, Hourly, and Actual.

Event Category

The Event Category drop-down list allows you to choose which type of event to visualize from the following options:

- DataRaker Events: events found by DataRaker processes (for example, daily register gap flag).

- Meter Events: events reported by or associated with a meter (for example, Endpoint Power Restore).

- Account Events: events (provided by a CIS) that are associated with an account (for example, account opening, account closing, temporary disconnect, etc.).

- List Events: events found by algorithms.

Transformer Load Management Panel

The Transformer Load Management panel displays the load that is passed through the transformer at a given point in time.

Chart View

The Transformer Load Management chart displays two data charts and a zoom chart at the bottom. The upper chart displays the transformer data and the middle chart displays the data for the child meters based on the selected Fact Category. The transformer data includes the transformer's rating in kVA using a default power factor of 1; by default, the rating lines are set at 100% and 165%, but these values are configurable.

The child meter legends are links that will load the individual meter data.

Grid View

To see the data behind the chart, click the grid view icon ().

Note: Each table column can be sorted in ascending or descending order.

- Fact Category: Voltage

Column Heading

Description

Average Voltage

Transformer voltage value at the row's date and time.

100% Rating

The kVA value for the transformer rating.

165% Rating

The kVA value when the transformer would hit 165% of its load capacity.

Date

The date and time that the data point was collected.

Contributing Meters

The number of meters that supplied information to the aggregated data.

Note: the Contributing Meters value may be different than the Connected Meters, which represents the number of meters that are associated to the transformer and could, potentially, contribute information.

Remaining Columns

The voltage data for each of the transformer's child meters.

Example Corresponding Chart View

- Fact Category: kWh Aggregated Data

Column Heading

Description

kWh Aggregated

The aggregated data collected from the transformer’s child meters.

100% Rating

The kVA value for the transformer rating.

165% Rating

The kVA value when the transformer would hit 165% of its load capacity.

Date

The date and time that the data point was collected.

Contributing Meters

The number of meters that supplied information to the aggregated data.

Remaining Columns

The consumption collected from the transformer’s child meters. Each meter is displayed with its consumption.

Example: Corresponding Chart View

Hover information provides the date, the value, and the number of contributing meters:

- Fact Category: kWh Aggregated by Phase

Note: Phase information is only available in the Distribution Planning and Operations module.

Column Heading

Description

Aggregated Phase A

The aggregated A phase data collected from the transformer’s child meters, if applicable.

Aggregated Phase B

The aggregated B phase data collected from the transformer’s child meters, if applicable.

Aggregated Phase C

The aggregated C phase data collected from the transformer’s child meters, if applicable.

100% Rating

The kVA value for the transformer rating.

165% Rating

The kVA value when the transformer would hit 165% of its load capacity.

Date

The date and time that the data point was collected.

Contributing Meters

The number of meters that supplied information to the aggregated data.

Remaining Columns

The consumption collected from the transformer’s child meters. Each meter is displayed with its consumption.

Example: Corresponding Chart View

Values for the individual phases may be seen by hovering over a point in the chart. The hover information provides the date, the phase, the value, and the number of contributing meters:

- To switch back to the chart view, click the chart view icon ().

Next in This Section

- Options

Options

The Transformer Load Management panel allows you to modify the conditions that help to render the chart. You may change the following parameters (from left to right):

Option

Possible Values

Default Value

Time-Basis

Hourly

Hourly

Fact Category

Volt; kWh Aggregated, kWh Aggregated by Phase

Volt

Weather

Fahrenheit Historical NHD, Fahrenheit Observation NOD

Heat Map Panel

Note: The Heat Map Panel is available through the Distribution Planning and Operations module when displaying transformer data.

The Heat Map panel displays the transformer’s load over time. The vertical axis (y-axis) represents the hours during a day, and the horizontal axis (x-axis) represents calendar days. Load readings are charted throughout this graph using the following color sequence:

- Green: not overloaded, or 0–100% of capacity.

- Yellow: overloaded, or 101–165% of capacity.

- Red: greatly overloaded, or greater than 165% of capacity.

Note: The Heat Map Panel’s boundary conditions can be configured.

Chart View

Grid View

To see the data behind the chart, click the grid view icon ().

Column Heading

Description

Fact Name

Displays the name given to the metric fact when initiated - typically, Usage.

Date

The date and time that the fact was generated.

Value

The fact’s derived value for the associated instance.

Each column can also be sorted in ascending or descending order.

To switch back to the chart view, click the chart view icon ().

Next in This Section

- Options

Options

The Heat Map panel displays the transformer’s load over time. You may change the following parameters (from left to right):

Fact Category Options

Fact Name Options

kWh Phase

Usage (Phase Selection)

kWh Avg

Average Usage

Map Panel

The Map panel provides a geospatial view of an object's location. When an object is selected, opening the map panel will center the object in the map. Click the object location to see object information, which may be the object name or detailed information.

Map panels display objects having a discrete location, such as a meter, as a single point on the map, which is indicated by a pin; for types that have an area or territory, the map displays the location as a shaded polygon.

Location-Associated Types

Only object types with location attributes have Map panels.

- Bill Cycle

- Feeder

- Meters and Service Points

- Rates

- Routes

- Transformers

- Zip Codes

Buttons

- Neighboring Electric Meter: Displays nearby meters highlighted with blue flags.

- Help Launches online help in a separate window.

- Switch to Street/Switch to Satellite Toggle:

- Grid View Displays attribute information about the selected asset.

In the base product environment, the Map Panel grid view data table contains the following columns:

Name

Description

Fact Name

Fact Name of the row data fact. For example, County is the fact name of an Address fact category.

Date

The Date value may be a default date for facts in the system (i.e., year 2001) or may be a date related to the fact. For example, if a meter is installed after the default date, the installation date fact will be updated.

Value

Provides the Value for the fact listed in a row.

Note: The grid view data table columns are configurable, allowing customers to choose what data to display.

- Close Closes the panel.

Maps Information Pop-up

When you click a selected point, a pop-up will appear with information about the point. The information that appears on the pop-up depends on configuration for each type.

Next in This Section

- Map Panel Options by Type

Map Panel Options by Type

Map panels behave similarly from type to type, but there are differences due to the nature of the data. The following illustrates the differences.

Meter

Map panels for meters open with the meter centered in the map. The following screen captures uses an electric meter map panel to illustrate the functionality:

Select Neighboring Electric Meter to see neighboring meters.

Line Transformer

Line Transformers display the area that they supply in a polygon.

Select Points to view the child meters connected to the transformer.

Select Neighboring Line Transformer to view the adjacent transformer polygon.

Select Line Transformer Phase Info to view adjacent transformer polygons color-coded by phase.

Export

The Fact Data Export page allows you to search for, view, and export objects that share the same fact. The fact of interest is selected in the Filters pane and the data to be returned can be refined using options in the main data panel.

Fields and Buttons

- Limit: drop-down list to select the number of records that will be returned by the query.

- Format: allows you to select whether to see the data in a data table or export the data as an XML, JSON, or CSV file.

- Offset: allows you to set the first point to return. For example, if your query is limited to 100 and you set the offset to 3, you will get a data table that starts at what would have been the fourth row.

- Verbose: if selected, will return additional data columns.

- Get Data: click to apply the selections and query the database.

Filters

The filters pane offers additional methods to target your results. The filter sections include:

- Metric: time series data that is collected at regular intervals.

- Event: occurrences that do not happen at regular intervals and are identified as outliers.

- List: objects that are returned through a calculation.

- Attribute: additional information that is provided for an entity.

- Relation: shows how an object is related to other objects in the database.

See Fact Data Export Filtersfor more information.

Next in This Section

- Fact Data Export Filters

- Export by Fact Options

- Export by Fact Data Table

Fact Data Export Filters

The Filters section for the Fact Data Export page provides controls that allow you to select data attributes (facts) stored against data types. Different data types may have different facts and the availability of facts is dependent on what data is available in the database.

The Filters pane contains controls for five fact categories:

Category

Description

Metric

Metric facts are time series data that is collected at regular intervals. For example, consumption data that is collected by the meter daily, hourly, and monthly. Commonly viewed metric facts are daily consumption data and weather data.

Event

Event facts are for occurrences that do not happen on regular intervals, but are identified when an outlier is identified. Power Outage, Spike Flag, and Inverse Rotation are examples. Events can come from the data collection system (meter events) or can be generated as part of the daily core calculation runs (derived events).

List

A list contains the objects returned as the results of a calculation or test created in DataRaker Cloud Service. For example, if an algorithm (calculation) looking for low consumption is executed, it would generate a list of all Point IDs that show low consumption for the selected time period.

Attribute

The Attribute fact category contains additional information that is provided for a specific type and that does not change frequently (for example, an address that belongs to a meter or a meter dial). DataRaker distinguishes between numeric and text attributes. Dial count is an example of a numeric attribute and an address is an example of a text attribute.

Relation

Relation shows how the selected object is related to other objects in the database. For example, the relationship between a meter and the transformer that it is linked to. The relation fact also includes the data source (for example, whether the data was received from another system or calculated).

Controls and Indicators

Description

The fact category title bar with expand button. Click to open the fact category.

Note: When you click to open a fact category, the fact category that is already open will close.

The fact category title bar with open indicator, which does not have an action. To close a fact category, you must open a different fact category.

Radio buttons allow you to select from the different fact sub-categories.

Slider allows you to choose the time-basis for the fact. The shortest valid time-basis is one for which data is available. For example, if you have meters that return data every 15 minutes, you may select a time-basis of 15 minutes or greater; however, if you select a shorter time-basis, a message will inform you that no data is available and allow you to reset the slider to the latest time-basis selection.

For facts that do not have a time-basis, the slider can be set to n/a.

Export by Fact Options

Next in This Section

- Limit

- Offset

- Format

- Verbose

Limit

The Limit option allows you to set the upper limit for the number of entries you want returned. The default limit is 1,000; selecting an option larger than 1000 will cause the Format to change to an export format that will download the data to file and remove the data table from the page.

Note: High limits can negatively impact performance.

Offset

With an offset, the data set returned from a query will have the number of rows based on the limit and will start at the first point ID that meets the criteria. The data set that is returned may be shifted by entering a positive number in the Offset field to create a data set that starts with the row corresponding to the value of the offset plus one.

Example

If you have a limit of 7, the returned data set without an offset, will contain the first through seventh rows:

Index

Point ID

1

12203850

2

12203860

3

12203870

4

12203880

5

12203890

6

12203900

7

12203910

If you add an offset of 5, the returned data set will start with the sixth row (offset plus one) and end with the twelfth row:

Index

Point ID

1

12203900

2

12203910

3

12203920

4

12203930

5

12203940

6

12203950

7

12203960

Format

The Data Table format displays returned data in a table on the page. If you wish to export the data, select XML, JSON, or CSV and click Get Data; the data will be downloaded as a *.xml or *.json or *.csv file, respectively.

Verbose

When the Verbose option is selected additional columns of information are provided. The different settings provide the following options:

Default Setting

Verbose Setting

Description

Point ID

Value

The value that we searched for. So this value should lie between the start and the end value that where selected.

Date

The date the value was recorded, for example the date for which the estimation was calculated for.

Point ID

Unique Identifier which is stored in the database

Point Type Code

Is the Type that has been selected such as Account, Electric Meter. It reflects what was selected in the Global Filter under Type.

Point Lookup

The Point Lookup is the Point Name where empty spaces have been removed and replaced by hyphen if multiple empty spaces are available they are replaced by only one hyphen, and all lower case letters will be replaced by capital letters. So, for example, Point Name “Meter Electric 457” will appear as METER-ELECTRIC-457. In this way data is getting standardized and can be more easily found and searched for.

Point Name

The point name is either the asset number as it is stored in the Utility database or it is a concatenation of information. This information is unique by instance implemented. So in our example to the left we have a concatenation of Premise ID-Commodity Type (E for electric) plus the Meter ID. We could also just have the meter ID here. How this is configured depends on the individual customer configuration. This is a unique identifier.

Point Tag

The Point Tag is the asset number as used by the utility. So Point Lookup and Point Name are often the same, and depends on the individual configuration and is decided with the customer. A tag allows additional information to be stored for a point and serves as an alternate way to lookup a point. Unlike point lookup or point name, point tag does not have to be unique and a query by point tag can result in multiple matches.

Each column can also be sorted in ascending or descending order.

Export by Fact Data Table

The Export by Fact data table contains the following standard and verbose columns:

Default Setting

Verbose Setting

Description

Point ID

Point ID

The Point ID is a unique identifier.

--

Point Type Code

Point Type Codes are DataRaker codes for types. The Point Type Codes will vary by implementation.

--

Point Lookup

The Point Lookup is constructed from the Point Name by replacing spaces with a hyphen and lower case letters with capital letters. For example, if

Point Name: Meter Electric 457
then

Point Lookup: METER-ELECTRIC-457

--

Point Name

The Point Name is a unique identifier. It is either the asset number (provided by the utility) or it is created by concatenating information about the point. The Point Name will vary by implementation.

--

Point Tag

The Point Tag is an asset number by the utility. The Point Tag allows additional information to be stored for a point. Unlike Point Lookup or Point Name, Point Tag does not have to be unique so a search by Point Tag can result in multiple matches.

Value

Value

Provides the Value for the fact listed in a row.

Effective Date

Effective Date

The date the value was recorded, for example the date for which the estimation was calculated for.

Each column can also be sorted in ascending or descending order.

Next in This Section

- Exporting a Data Table

Exporting a Data Table

The default Format is Data Table, which provides the data in a table on the page. To export the data to a file, select XML, JSON, or CSV and click Get Data. The data will be downloaded as a, XML, JSON, or CSV file, respectively.

Note: JSON is a file extension and an acronym for JavaScript Object Notation, which is an open standard used to store data that is human readable and easy for a script to parse.

The export file will contain the information as shown in the data table. If Verbose is selected, the export will contain the verbose columns.

DataRaker Overview Dashboards

The DataRaker dashboards are available through the Overview link. Dashboard pages (or tabs) provide aggregated and summarized information from algorithms implemented for a selected Role (see Drawer Menu for more information on Roles). Dashboards are currently available for the Distribution Planning and Operations, Meter to Bill, and Revenue Protection modules.

Next in This Section

- Understanding the Dashboard User Interface

- Dashboards By Module
The following sections describe the dashboards that are available for the Meter to Bill, Revenue Protection, and Distribution Planning and Operations modules.

Understanding the Dashboard User Interface

The dashboards user interface provides a common structure of tabbed page views [1] having a basic layout with a Filter pane in the left column [2], data presentation on the right [3], and a hyperlink that opens the DataRaker Explore page [4]:

Note: The data displayed varies by the selected module and role; see the Drawer Menu for a list of modules and roles and the Environment Filters section for information on module and role selection.

Next in This Section

- Filters

- Tabs

Filters

The filter pane for each dashboard page allows you to select a Role, based on licensed module(s), that will determine the type of data that will be displayed. Other filter criteria vary depending on which tabbed view is selected. When you access the dashboard, the role will be populated with the role selected in DataRaker (see the Environment Filters for Role selection information).

Tabs

The following dashboard tabs are available:

- Summary Statistics

- Categories Overview

- Category

- Algorithm

- Trend

Note: The Distribution Planning and Operations module currently only provides data for the Summary Statistics tab.

Dashboards By Module

The following sections describe the dashboards that are available for the Meter to Bill, Revenue Protection, and Distribution Planning and Operations modules.

- Meter to Bill and Revenue Protection Dashboards

- Distribution Planning and Operations Dashboard

Next in This Section

- Meter to Bill and Revenue Protection Dashboards

- Distribution Planning and Operations Dashboard

Meter to Bill and Revenue Protection Dashboards

The Meter to Bill and Revenue Protection modules share a common set of dashboard pages and charts for displaying meter data. The data that is displayed varies by the algorithms assigned to the module and role.

Next in This Section

- Summary Statistics Tab

- Categories Overview Tab

- Category Tab

- Algorithm Tab

- Trends Tab

Summary Statistics Tab

The Meter to Bill and Revenue Protection module's Summary Statistics tab provides meter information based on the data returned from the module algorithms.

The page has the following components:

1. Title: Module Name (a) - Role (b).

Note: The role is determined by the Filter Role selection. Roles correspond to modules so choosing a role automatically determines the module.

2. Filter: the filter section allows you to select criteria for data display.

- Module: a required component of the data filter criteria. The module will be populated by the module selected in DataRaker, but may be changed to another module that you have access to.

- Role: a required component of the data filter criteria. The role will be populated by the role selected in DataRaker, but may be changed to another role that you have access to.

- Year: a required component of the data filter criteria. Select the applicable year from the drop-down list.

- Month: a required component of the data filter criteria. Select the applicable month from the drop-down list.

- Month Range: a required component of the data filter criteria. Select the range from the options: 1, 3, 6, 12.

- Commodity: a required component of the data filter criteria. Select from the commodities available in the drop-down list: Electric, Gas, or Water.

Note: The commodities available may vary.

3. Charts: displays charts applicable to the selected module and role:

- Meter by Data Collection System

- Meter by Status

- Meter by Customer Type

- Meter by Rate Type

- Meter Outages

Meter by Data Collection System

The Meter by Data Collection System chart provides information on data collection method; for example, by an AMI system for sub-daily information, AMR system for less frequent data collection (daily or longer), or manually (monthly, bimonthly, etc.). The graph displays how many meters have been read (Meter Count y-axis) over time by meter data collection method (Meter Type).

Meter by Status

The Meter by Status chart provides information about meter consumption based on the meter's status (i.e., active, inactive, or disconnected). Active meters are expected to have consumption while inactive or disconnected meters should not. This information can be viewed by total meter count, percentage of the total installed meter population, or as change from one month to the next.

Meter by Customer Type

The Meter by Customer Type chart provides information by customer type (e.g., Residential, Commercial, Industrial, or Agricultural). The available display options are Meter Count, Meter %, and Consumption.

Meter by Rate Type

The Meter by Rate Type chart displays meter data based on their assigned rate class. The available display options are Meter Count, Meter %, and Consumption.

Meter Outages

The Meter Outages chart provides information regarding Power Outages, Isolated Outages, and Transformer Outages that might occur on a daily basis. This section applies to electric utilities only. The definitions for the different outages are as follows:

- Isolated Outage: Outages detected on individual meters, but not on other meters connected to the same transformer.

- Transformer Outage: All meters downstream of the transformer are out.

Note: Substation outages will also appear as transformer outages. If more detailed information regarding outages and distribution network is required, the Distribution Planning and Operations module will provide further details.

- Power Outage: All outages experienced at the meter level regardless of outage source.

Categories Overview Tab

The Categories Overview tab provides summary information about the data found for the categories associated with a specific role and module.

The page has the following components:

1. Filter: the filter section allows you to select criteria for data display.

- Year: a required component of the data filter criteria. Select the applicable year from the drop-down list.

- Commodity: a required component of the data filter criteria. Select from the commodities available in the drop-down list: Electric, Gas, or Water.

Note: The commodities available may vary.

- Meter Status

2. Total Exception for [Module Name] - [Role]: the number of exceptions found presented in a pie chart broken down by category. Clicking a slice of the pie chart updates the selection in the Categories chart.

3. Categories: bar chart showing the count for the category over time.

4. Show Category Detail link: navigates to the Category Tab.

Category Tab

The Category tab provides information about meters found by algorithms.

The page has the following components:

1. Filter: the filter section allows you to select criteria for data display.

2. Table: the table lists the algorithms and each row provides counts for meters found by the algorithm by month over the year selected in the filter's Year drop-down list. The month that had the highest meter count for the algorithm is highlighted in red; months with the least exceptions over the year are highlighted in green.

Algorithm sub-types, if applicable, may be viewed by clicking the triangle control at the beginning of the row:

Click the algorithm name to have the charts display data for the algorithm.

The table has the following controls:

Button

Description

Resets the table view to show the first row. If the table is showing the first row, the button will be desensitized ().

Shows the next set of table rows. If the table is showing the last set of rows, the button will be desensitized ().

Shows the previous set of table rows. If the table is showing the first set of rows, the button will be desensitized ().

Shows all table rows. If the table is showing all rows, the button will be desensitized ().

3. Charts : show data based on either the algorithm selected in the table or through the individual drop-down lists.

1. Customer Type Chart: shows the counts returned by the algorithm based on the customer type (Commercial, Industrial, Residential).

2. Data Collection System Chart: shows the counts returned by the algorithm based on the data collection system (AMR, AMI).

3. DataRaker Meter Events Chart: shows the counts for events (Derived Usage Spike, Register Gap, Rollover Register, Spike Register, Static Register). See Derived Event Definitions for details on event definitions.

Algorithm Tab

The Algorithm tab provides the results found by an algorithm for a time range selected in the Filter Date selection.

The page has the following components:

1. Filter: the filter section allows you to select criteria for data display.

2. Table:the table lists the meters found by the selected algorithm(s).

Column

Description

Meter ID

Provides the Meter ID with a hyperlink that navigates to the DataRaker Detail for the meter. See Explore for details on the information provided in the DataRaker Detail view.

Service Point ID

The ID assigned to the service point.

Algorithm Start Date

The beginning boundary used in the algorithm that returned the meter.

Address

The address where the meter is located.

Postal Code

The postal code for the meter location.

Meter Status

The meter's status (Active, Inactive, Disconnected).

Algorithm Hit Count

The number of times the meter met the algorithm criteria during the selected time range.

The table may be sorted by any of the columns by hovering over the column heading to display the Sort Ascending and Sort Descending controls.

The table has the following controls:

Button

Description

Resets the table view to show the first row. If the table is showing the first row, the button will be desensitized ().

Shows the next set of table rows. If the table is showing the last set of rows, the button will be desensitized ().

Shows the previous set of table rows. If the table is showing the first set of rows, the button will be desensitized ().

Shows all table rows. If the table is showing all rows, the button will be desensitized ().

3. Map: shows a map displaying locations and frequency of algorithm hits. The map's legend can be shown or hidden by clicking the expand button:

Trends Tab

The Trends tab displays the results of the algorithms over time. The information is displayed as a heat map that shows counts by color; higher counts have darker, more intense colors.

The page has the following components:

1. Filter: the filter section allows you to select criteria for data display.

2. Daily Meter Counts: shows a heat map displaying frequency of algorithm hits. Click a square representing a day to display the data for the day in the Meters Identified for Selected Day pane.

3. Meters Identified for Selected Day: lists the meters that the algorithm found on the day selected in the Daily Meter Counts heat map and a map with hit count frequency.

Note: Click the day's heat map square to update the Meters Identified for Selected Day data.

.

Column

Description

Meter ID

Provides the Meter ID with a hyperlink that navigates to the DataRaker Detail for the meter. See Explore for details on the information provided in the DataRaker Detail view.

Service Point ID

The ID assigned to the service point.

Address

The address where the meter is located.

Algorithm Hit Count

The number of times the meter met the algorithm criteria during the selected time range.

The table may be sorted by any of the columns by hovering over the column heading to display the Sort Ascending and Sort Descending controls.

The table has the following controls:

Button

Description

Resets the table view to show the first row. If the table is showing the first row, the button will be desensitized ().

Shows the next set of table rows. If the table is showing the last set of rows, the button will be desensitized ().

Shows the previous set of table rows. If the table is showing the first set of rows, the button will be desensitized ().

Shows all table rows. If the table is showing all rows, the button will be desensitized ().

The map's legend can be shown or hidden by clicking the expand button ().

Distribution Planning and Operations Dashboard

The Distribution Planning and Operations dashboard provides panels on the Summary Statistics page that display information related to distribution network transformer and feeder performance and health.

Next in This Section

- Distribution Planning and Operations Summary Statistics Tab

Distribution Planning and Operations Summary Statistics Tab

The Distribution Planning and Operations Summary Statistics tab provides overview data and key performance indicators (KPIs) for distribution network transformers and feeders.

Note: The data for the examples given are for illustration only and do not represent real-world values.

The page has the following components:

1. Title: Module (a) - Role (b)

Note: The role is determined by the Filter Role selection. Roles correspond to modules so choosing a role automatically determines the module.

2. Filter: the filter section allows you to select criteria for data display.

- Module: a required component of the data filter criteria. The module will be populated by the module selected in DataRaker, but may be changed to another module that you have access to.

- Role: a required component of the data filter criteria. The role will be populated by the role selected in DataRaker, but may be changed to another role that you have access to.

- Year: a required component of the data filter criteria. Select the applicable year from the drop-down list.

- Month: a required component of the data filter criteria. Select the applicable month from the drop-down list.

- Month Range: a required component of the data filter criteria. Select the range from the options: 1, 3, 6, 12.

- Commodity: the only option is Electric.

3. Charts: displays the following charts:

- Asset Health Index

- Outages of Transformer

- Age of Transformer Population

- Outages of Feeder

- Feeder Load

- Age of Failed Transformers

Asset Health Index

The Asset Health Index report provides information about transformers characteristics (e.g., the transformer age) and calculated performance indicators. The data is presented in a table that may be sorted by column.

Name

Description

Transformer ID

The transformer identifier.

Transformer Type

Type of transformer (underground, pad mounted, etc.).

Age

Age of transformer in years. The age cell is color coded according to configurable boundary conditions based on the expected lifespan of the transformer:

Color

Description

Green

Age is less than the life expectancy.

Orange

Age is equal to or up to a configurable number of years greater than the life expectancy.

Red

Age is greater than the configured high limit for life expectancy.

Loss of Life

Calculated value for loss of life (in hours).

Note: Loss of life is calculated in DataRaker from data related to a number of factors including environmental conditions (heat) and overload conditions.

Estimated Remaining Life

Transformer life expectancy minus age (in years).

# of Daily Overload Events High

The number of daily overload events with a "high" threshold boundary value that is determined by the customer implementation requirements.

# of Daily Overload Events Low

The number of daily overload events with a "low" threshold boundary value that is determined by the customer implementation requirements.

# of Transformer Level Outages

Number of outages due to a transformer failure.

# of Phases

The number of phases supplied by the transformer.

Type of Phases

Lists the phases supported by the transformer (e.g., ABC, AB, C).

Count of Meters

The count of connected meters.

Outages of Transformer

The Outages of Transformer is a bar chart that displays the count of transformers versus the number of outages experienced. When you hover over a bar, a hover text box is displayed showing the number of outages and count of transformers.

Age of Transformer Population

The Age of Transformer Population chart provides the distribution of transformers or connected meters by the age of the transformer. The y-axis shows the count of transformers by default.

Select Count of Meters from the Y-Axis Selection drop-down list to show the meter count. For example, the charts below show that for age equal to 16, there are 38304 transformers with 289586 connected meters:

Outages of Feeder

The Outages of Feeder is a bar chart that displays the count of feeders versus the number of outages. When you hover over a bar, a hover text box is displayed showing the number of outages and count of transformers.

Feeder Load

The Feeder Load chart provides feeder load versus time segmented by customer type (i.e., commercial, industrial, or residential).

All feeder load is displayed by default, but you can see the load for a specific feeder by choosing the feeder id from the drop-down list.

Age of Failed Transformers

The Age of Failed Transformers is a bar chart that displays the count of transformers that failed versus age in months. When you hover over a bar, a hover text box is displayed showing the age when a transformer failed and the number of transformers that failed at that age.

The Select a view drop-down list allows you to see the transformer population by manufacturer or rating (kVA):

Manage

The Manage menu provides access to Operations and Analytics features.

- Manage Operations provides options to initiate and track functions (such as Run, Task, and Transactions) and their corresponding logs.

- Manage Analytics provides features that allow you to find and analyze data.

Next in This Section

- Manage Operations

- Manage Analytics

Manage Operations

The Manage Operations menu offers functionality to initiate and track system procedures. Within this functional area, some of the key features, which correspond to sub-menu items, are:

- Task:procedures that perform analytics or system administration.

- Run: a container that can bundle and organize tasks to be executed in a specific order. Tasks can be defined to be executed sequentially or in parallel.

- Transaction: the system creates transactions to track events. Examples include noting a user’s session and processing a scenario. Each task that executes within a run results in a transaction.

Within the Manage Operations menu group, an administrator can create tasks and bundle them into runs. Tasks can be initiated manually and runs can be automated into scheduled executions. Once the run, or task, is executed, the system creates a log entry that highlights key information from the resulting transaction.

Similar to other functional areas, you can navigate to each of the Manage Operations features from the sub-menus. The best practice, however, is to perform an initial search through the Activity Dashboard or Manage Runs page, and then use the links in the search results to find related information. Navigating through these links brings context so you do not need to search for and find the desired event again.

Next in This Section

- Activity Dashboard

- Manage Runs

- Manage Run Logs

- Manage Tasks

- Manage Task Logs

- File Log

- Manage Transactions

- Transaction Log

- Batch Log

- Fact Audit

- Run/Task Operations

Activity Dashboard

The Activity Dashboard provides information about processes (known as tasks) that have been executed by the system. When a task is executed, it creates a transaction, which is added to the Activity Dashboard data table. From the Activity Dashboard, you may view the task that created the transaction, the run that executed the task, the status of the transaction, and transaction status.

The Activity Dashboard is commonly used as an initial query to see overview information and the status of processes that have run or are running. From this page, you can see the current status (color coded) and navigate to further pages to see more information about the associated transaction, task, and run.

Fields

The top section of the Activity Dashboard page allows you to search for tasks matching criteria in the fields:

- Number of Previous Runs: sets the number of historical runs to display as well as include when calculating an average time for the run.

- Delay Margin (%): allows you to set a tolerance to designate an on-time run (labeled in green). The average time is calculated against the historical runs that the Number of Previous Runs specifies. The delay margin percentage is then added to the average in order to create the margin.

- Refresh Rate: the frequency (in seconds) to automatically update the results.

- Status: allows you to search for transactions that are a specific status.

Buttons

- Get Data: initiates a search for data matching criteria set in the fields.

Search Results

Highlighted and color-coded rows in the search results show the status, or timeliness, of the execution.

- Green: On-Time, within the average time plus delay margin.

- Yellow: Slow, later than the average time plus delay margin.

- Red: Failed; this is commonly due to an error in the run.

Note: These row colors do not indicate completion – completion is indicated by an entry within the Actual End Time column. Once complete, the Actual End Time column displays the date and time that the activity completed; therefore, no entry in the Actual End Time column indicates that the activity is currently running.

Data Table

- Transaction ID: the transaction’s system-assigned, unique identifier.

- Transaction Name: the system-assigned name that combines the task lookup name with key information from the task. The transaction name is formed by concatenating the task lookup with the task start date, task end date, and the task run mode; for example, CORE_FACT_AUDIT from 08/19/2015 to 08/19/2015 (STD).

- Node: indicates the server node that executed the transaction.

- Actual Start Time: the date and time when the transaction was executed.

- Estimated End Time: the date and time when the transaction was expected to end.

- Actual End Time: the date and time when the transaction actually ended.

- Estimated Duration: the amount of time that the transaction was expected to take

- Actual Duration: the amount of time that the transaction actually took.

- Data Start Date: the earliest date and time of the data that was selected for the transaction.

- Data End Date: the latest date and time of the data that was selected for the transaction.

- Links:

- View: opens the View Activity Dashboard dialog box, which displays the same data as provided in the data table row.

- Transaction: navigates to the Manage Transactions page and displays the current context to provide further information.

- Task: navigates to the Manage Tasks page and displays the current context to provide further information.

- Run: navigates to the Manage Runs page and displays the current context to provide further information.

Note: If the task was executed manually outside of a run, no run data will exist.

Manage Runs

A run is a way to combine and organize a group of tasks into a process flow in order to perform a function. Runs contain tasks for the system to perform and can be run manually or automatically.

Manual runs are launched through the dialog boxes in the Manage Runs data table (see Executing Runs for more information). Automated runs are scheduled as a background process on the application server.

Run Fields

The top section of the Manage Runs page allows you to find existing runs matching criteria in the fields. (See Adding Runs for more information about the specific search fields.)

- Run ID: the run’s system-assigned, unique identifier.

- Run Type Code: drop-down list to search by the run type. Valid values are STD (standard) and DYNAMIC, but the system currently disregards this setting and all runs are processed as STD.

- Run Name: the run "friendly" name.

- Run Lookup: a variation on the run name, which does not include spaces and is uppercase. (This field is case sensitive.)

- Status: drop-down list to search by the run status (Active, Inactive).

Data Table

- ID: the run’s system-generated, unique identifier.

- Run Type Code: the type code that was assigned to the run when it was added. (See Adding Runs for more information.)

- Run Name: the run’s name as designated when the run was added.

- Run Lookup: a variation on the run name, which does not include spaces and is uppercase.

- Run Graph: provides a link to open a graphical view of the run, which is a flowchart that shows the tasks that make up the run in the order they are executed. (See Editing Runs with the Run XML Editor and Creating Edges for more information on run graphs.)

- Status: the run’s status (Active/Inactive).

- Create Time: the date and time when the run was created.

- Update Time: the date and time when the run was updated.

- Links:

- View: opens the View Runs dialog box, which displays the same data as provided in the data table row. In addition, it includes whether the run is locked or not. This dialog box also provides links that allow you to edit or execute the run. (See Viewing Run Informationfor more details.

- Edit: opens the Manage Runs dialog box, which provides the same fields as the View Runs dialog box. In addition, it allows you to edit the Locked Flag and Status. (See Managing Run Information for more information.)

- XML: displays the Run Definition dialog box, which shows the configuration XML for the run. (See Viewing and Editing the Run Definition XML for more information.)

- Run Log: navigates to the Manage Run Logs page and displays the current context to provide further information.

- XML Editor: displays the Run XML Editor. (See Editing Runs with the Run XML Editor for more information.)

Next in This Section

- Adding Runs

- Viewing Run Information

- Managing Run Information

- Editing Run XML

- Executing Runs

Adding Runs

To add a run, click Add Run on the Manage Runs page. The Create Runs dialog box opens.

On the Manage Runs page, click Add Run.

1. In the Create Runs dialog box, do the following:

1. From the Run Type Code drop-down list, select STD.

2. In the Name field, enter a descriptive name.

3. In the Run Lookup field, enter a value for run lookup, which can be any unique, alphanumeric value without spaces. A common practice is to use the run name with all capital letters and without spaces

4. (Optional) In the Run Description field, enter a short description of the run’s function.

5. From the Status drop-down list, select whether the run is Active or Inactive.

2. Click Save. The Create Runs dialog box will close and the new run will be added to the Manage Runs data table

Note: click Cancel to close the dialog box without saving.

Viewing Run Information

The View link in the Manage Runs data table row launches the View Runs dialog box. This dialog box displays run information and allows you to view the run definition XML, edit the run information, and execute the run.

Fields

- Run ID: the run’s system-generated, unique identifier.

- Run Type Code: the type code that was assigned to the run when it was added. (See Adding Runs for more information.)

- Run Name: the run’s secondary name, or nickname, that was given to the run when it was added.

- Run Lookup: a variation of the run name, which does not include spaces and is uppercase.

- Run Desc: a short explanation of the run’s purpose or function.

- Locked Flag: indicates whether changes to the run are allowed to be made (Unlocked) or not (Locked).

- Status: defines whether the run can be executed (Active) or not (Inactive).

- Create Time: the date and time that the run was added.

- Update Time: the date and time of the most recent change.

Buttons and Links

- XML link: opens the Run Definition dialog box. (See Viewing the Viewing and Editing the Run Definition XML for more information.)

- Execute Now link: opens the Execute Runs Details dialog box. (See Executing Runs for more information.)

- Edit button: opens the Manage Runs dialog box. (See Managing Run Information for more information.)

- Cancel button: closes the dialog box.

Managing Run Information

The Manage Runs dialog box opens when you click the Edit link on the run data table row or the Edit button on the View Runs dialog box.

Fields

- Run ID: the run’s system-generated, unique identifier.

- Run Type Code: the type code that was assigned to the run when it was added. (See Adding Runs for more information.)

- Run Name: the run’s name as designated when the run was added.

- Run Lookup: the run’s secondary name, or nickname, that was given to the run when it was added.

- Run Desc: a short explanation of the run’s purpose or function.

- Locked Flag: drop-down list that allows you to define whether changes to the run are allowed (Unlocked) or not allowed (Locked).

- Status: drop-down list that allows you to define whether the run can be executed (Active) or not (Inactive). Inactivating a run is a way to prevent it from executing without removing the run from the system.

- Create Time: the date and time that the run was added.

- Update Time: the date and time of the most recent change.

Buttons and Links

- XML link: opens the Run Definition dialog box. (See Viewing and Editing the Run Definition XML for more information.)

- Execute Now link: executes the run. (See Executing Runs for more information.)

- Save button: allows you to save any changes that have been made; the button is disabled until a field has been modified.

- Cancel button: closes the dialog box.

Editing Run XML

The XML that defines a run may be edited as text or by using the Run XML Editor. Both options allow you to create, view, and edit a run. The text option gives you the additional feature to copy an existing run definition into a new run. The following sections describe how to work with the Run Definition XML as text or with the Run XML Editor.

Next in This Section

- Viewing and Editing the Run Definition XML

- Editing Runs with the Run XML Editor

- Duplicating the Run Definition XML

Viewing and Editing the Run Definition XML

XML is typically created and edited with the Run XML Editor, but the Run Definition dialog box allows you to quickly modify XML parameters or define a run using XML from a similar run. From the Run Definition dialog box, you can view or edit the raw XML code that specifies the run parameters. (see Duplicating the Run Definition XML for information).

Editing Runs with the Run XML Editor

The Runs XML Editor provides a user interface to define and organize procedures to be run. The procedures, called nodes, are connected with edges that define the flow for the run. For example, in the following illustration, five nodes (Start, D1, T1, U1, and T2) are connected in a process flow using six edges (E1-E6).

Note: Nodes and Edges are described in the Creating Nodes and Creating Edges sections.

Buttons:

- Save: saves the run in its current state. If the run is structurally invalid, it will show a warning indicating that the run was saved, but it will fail execution. The Runs XML Editor will close when you click Save.

- Validate: validates the run to make sure it is syntactically valid and shows validation errors, if any exist.

- Preview: allows you to view an image of the run graph based on the nodes and connectivity defined in the edges.

- Close: closes the Runs XML Editor without saving changes.

Next in This Section

- Graph Components

Graph Components

The Runs XML Editor uses graph terminology to describe the structure of the run. The run configuration is defined by Graph Type, which contains the Graph Header, Time Windows, Nodes, and Edges.

To define the run, complete the tasks outlined in the following sections.

Next in This Section

- Creating the Graph Header

- Creating Time Windows

- Creating Nodes

- Creating Edges

Creating the Graph Header

Select Graph Header in the Runs XML Editor tree-view.

Provide the data in the following three elements:

- Expected Duration Mins:The anticipated time, in minutes, that the run should take.

- Graph Name: The graph name is a user friendly name for the Run Graph.

- Graph Desc (Optional): The graph description allows you to add information describing the run.

- Graph Processing Basis: graph processing basis comes into play when a task is run repeatedly for an extended period of time. For example, if a run with a processing basis of daily is executed over a week or a month, it will be run once for every day of the week or month, respectively.

Creating Time Windows

In order to create time windows using the Runs XML Editor, do the following:

1. Select Time Windows in the Runs XML Editor tree-view.

2. Click Create New Time Windows.

3. Select the newly created time window in the tree view.

Note: You may edit the name to something meaningful.

4. Select the Type and choose from:

- Static Time Window

- Offset Time Window

Static Time Windows

Static time windows have fixed start and end dates that are defined in the filter. When you select Static Time Window from the Type drop-down list, you may edit the parameters:

- Name: populates with the name from the tree-view; you may change the name in the tree-view or in the static time window Name field.

- Description: optional field that allows you to enter a description for the time window.

- Start Date: the start date for the filter criteria.

- End Date: the end date for the filter criteria.

Offset Time Windows

Offset time windows allow you to create a time window that is offset from the start and/or end dates, which are input when the run is executed. An Offset Time Window has three unique sections:

1. Start Offset:
The Start Offset start date may be offset using one of two methods: simple offset or directional snapping.

- Simple offset offsets the start date by a selected time basis (Unit; e.g., day or month) and how many (Valueof) of those units .

- Snapping sets whether the start offset will expand backward in time or forward (snap direction).

2. End Offset: The End Offset is similar to the Start Offset with the exception that the resulting date must be later than the Start Offset date.

3. Time Basis: The Time Basis field allows you to interpret your data over a different time frame (for example, daily data over a month).

Creating Nodes

Nodes perform a task. The tasks may be an analytic operation, such as aggregating data, or it may be checking a condition to see if it is true or not true.

1. Select Nodes in the Runs XML Editor tree-view.

2. Click Create New Nodes.

3. Select the new node in the tree view and then select a node type from the drop-down list.

4. The node types are:

- Start Node

The start node is a "dummy" node that denotes the starting point of a run. The start node contains the following fields:

- Name: the name given to the node.

- Desc (optional): description of the node.

- Decision Node

A decision node allows the run operation to be branched by evaluating a condition. The condition is created in the Expression field by adding custom code that will dictate branching of the run logic. A decision node should always return a Boolean value (True/False). Based on the output of the decision node, children node will either be skipped or processed.

A decision node has three fields:

- Name: the name given to the node.

- Desc (optional): description of the node.

- Expression: contains free form code that determines whether the decision criteria have been met.

- Union Node

The union node allows the run execution to continue even if there is a failure condition in a previous node. Because the union joins the result set of two nodes, the failing node would not contribute to the outcome. The union node includes the following fields:

- Name: the name given to the node.

- Desc (optional): description of the node.

- Task Node

Executes the task defined within the node for the time period that the time window attached to the node results in. The task node contains the following fields:

- Name: the name given to the node.

- Desc (optional): description of the node.

- Task Lookup: assigns a task to the run using the task lookup value. You need to know the appropriate task lookup value to complete this field. (See Tasks for more information.)

- Dataset Lookup (optional): assigns a dataset to the task to be used when executing the task. (See Manage Datasets Page for details)

- Filter Lookup (optional): assigns a filter to the task to be used when executing the task. (See Manage Filters Page for details.)

- Calc Lookup (optional): assigns a calc to the task to be used when executing the task. (See Manage Calcs Page for details.)

- Scenario Lookup (optional): assigns a scenario to the task to be used when executing the task. (See Manage Scenarios Page for details.)

- Env Name: the name of the task’s applicable environment.

- Task Run Mode: the run mode for the assigned task

- Error Threshold: allows an error threshold to be set, which, if exceeded, causes the run execution to halt.

Creating Edges

Edges, like the arrows in a flowchart, connect two nodes together to create a sequence for node execution.

1. Select Edges in the Runs XML Editor tree-view.

2. Click Create New Edges.

3. Select the new edge in the tree view and then select the edge type in the Type drop-down list.

There are two edge types:

1. Edge: an edge that connects two non-decision nodes.

2. Decision Edge: an edge that connects a decision node to another node. If Yes is selected, the edge will be traversed when the decision is true; if Yes is not selected, the edge will be traversed when the decision is false.

4. Select the starting node and the ending node from the Source Name and Target Name fields, respectively.

In the following example, an edge connects the START node to the ETL_TEST node; the ETL_TEST node is connected to the MEM Derivation task node via a Yes Decision Edge and to the Core_C_Union node with a non-Yes Decision Edge; the MEM Derivation task node is connected to the Core_C_Union node, which allows the run to continue even if the MEM Derivation task fails; finally, the Core_C_Union node is connected to the Daily Core C task node.

The Run Graph would then look like the following:

Duplicating the Run Definition XML

Instead of creating a run from the beginning, it might be more efficient to duplicate an existing run and customize it to your needs. In order to copy a run, do the following:

1. In the Manage Runs page, search for and select the run that you wish to use as the starting point.

2. Click the XML link to open the Run Definition dialog box.

3. Copy the Definition XML and close the dialog box.

4. On the Manage Runs page, click Add Run to open the Create Runs dialog box.

5. In the Create Runs dialog box, enter the information in the following fields:

1. From the Run Type Code drop-down list, select STD.

2. In the Name field, enter a descriptive name.

3. In the Run Lookup field, enter a value for run lookup, which can be any unique, alphanumeric value without spaces. A common practice is to use the run name with all capital letters and without spaces

4. (Optional) In the Run Description field, enter a short description of the run’s function.

5. From the Status drop-down list, select whether the run is Active or Inactive.

6. In the Run Definition pane, paste the definition XML from the copied run.

7. Click Save. The Create Runs dialog box will close and the new run will be added to the Manage Runs data table

Executing Runs

Runs can be executed manually or on schedule. The Execute Runs Details dialog box can be accessed from the Execute Now link found in the Manage Runs' View Runs or Edit Runs dialog boxes.

1. On the Manage Runs page, click either the View or Edit link to display the corresponding dialog box.

2. From the dialog box, click the Execute Now link.

3. In the Execute Runs Details dialog box, do the following:

1. Verify the name field (display only). This is the name assigned to the run when it was added. (See Adding Runs for more information.)

2. In the Start Date field, enter the starting date for run execution.

3. In the End Date field, enter the date when the run should end. This will be the same as the start date for single instances of a manual run.

4. In the Priority drop-down, select the run’s priority on the node. Base values include:

- Highest

- Default

- Lowest

5. (Optional) In the Failure Email field, enter an email address to send a message to if the run ends unexpectedly.

6. Verify the lookup field (display only). This is the run’s lookup name as designated when the run was added.

7. In the Type drop-down, select the MANUAL execution method.

8. In the Run Mode drop-down, select the mode. Base values are:

- STD (Standard)

- Dynamic – not currently being used.

9. (Optional) in the Success Email field, enter an email address to send a message to if the task completes successfully.

4. Click Ok.

Manage Run Logs

Once a Run ends, the system automatically generates a log that describes the run’s process. The Manage Run Logs page provides these details about the run’s processing information and its corresponding tasks.

Fields

The top section of the Manage Run Logs page allows you to search for run logs matching criteria in the fields:

- Run ID: the run’s system-assigned, unique identifier.

- Run Log ID: the run log’s system-assigned, unique identifier.

- Refresh Rate: the frequency, in seconds, that the results should be automatically updated.

Buttons

- Get Logs: initiates a search for logs matching criteria set in the fields.

Data Table

- Run Log ID: the run log’s system-generated, unique identifier.

- Server Process: the server node that executed the run.

- Run Start Time: the date and time that the run started.

- Run End Time: the date and time that the run ended.

- Links:

- View: opens the View Run Logs dialog box, which displays the Run’s processing information such as start and end time, status code, and run progress.

- Run Summary: opens the Run Summary dialog box, which displays processing information about each task that was executed within the run. (See Viewing Run Summary Information for more information).

- Task Log: navigates to the Manage Task Logs page and displays the current context to provide further information.

Next in This Section

- Viewing Run Logs Information

- Viewing Run Summary Information

Viewing Run Logs Information

The View Run Logs dialog box is accessed from the View link, which is found in the Manage Run Logs page’s data table. This feature offers additional information to what is displayed in the Manage Run Log’s data table.

Fields

- Run Log ID: the run log’s system-generated, unique identifier.

- Run ID: the run’s system-generated, unique identifier.

- Execution Type: identifies the method that the run was initiated.

- Success Email: the address to send a notice when the run completes.

- Failure Email: the address to send a notice in case the run fails.

- Server Process: the server node that executed the run.

- Run Mode: the run’s processing category.

- Data Start Time: the earliest date and time of the data that was included in the run.

- Data End Time: the latest date and time of the data that was included in the run.

- Run Start Time: the date and time that the run started.

- Run End Time: the date and time that the run ended.

- Status Code: indicates if the run is In Progress, ran Successfully, or had a Fatal error.

- Run Progress: indicates the run’s current status.

- Run Priority: indicates the run’s priority on the node.

- Create Time: the date and time that the run was created.

- Create Transaction ID: the transaction ID when the run was created.

- Update Time: the date and time that the run was most recently updated.

Buttons

- Cancel button: closes the dialog box.

Viewing Run Summary Information

The Run Summary link in the Manage Run Logs data table launches the view Run Summary dialog box and supplies additional information.

Columns

- Task Lookup: the task’s secondary description, or nickname, that was given to the task when it was added.

- Data Start Date: the earliest date and time of the data that was included in the task

- Data End Date: the latest date and time of the data that was included in the task.

- Elapsed Time: the total amount of time that it took for the task to execute.

- Task Start Time: the date and time when the task initiated.

- Task End Time: the date and time when the task completed (either succeeded or failed).

- Transaction Name: the system-assigned name of the transaction associated to this task.

Buttons

- Ok button: closes the dialog box

Manage Tasks

Tasks are procedures that perform analytics or system administration. For example, analytic tasks might include aggregating data and system administrative tasks might include importing and exporting data. Tasks can be executed manually (ad hoc) or included within a run’s flow.

The Manage Tasks page gives you the ability to view information associated to current and historic tasks. In addition, you can create and execute new tasks. As an example, this page could be used as a starting point when you know the tasks names, while you are building a run, and if you want to see all the tasks of a specific type that are currently active. It is also common that you may reference this page through the Activity Dashboard and the Manage Runs page.

Fields

The top section of the Manage Tasks page allows you to search for tasks that match criteria in the fields.

- Task ID: the task’s system-assigned, unique identifier

- Task Type Code: allows you to search for tasks that are a specific task type code.

- Task Name: the name that was given to the task when it was added.

- Task Lookup: the task’s secondary description, or nickname, that was assigned when the task was added.

- Status: allows you to search for tasks that are a specific status.

Buttons

- Get Tasks: allows you to search for tasks matching the criteria set in the top section’s fields.

- Add Task: opens the Create Tasks dialog box where you may build a new task. (See Adding Tasks for details).

Data Table

- ID: the task’s system-assigned, unique identifier.

- Task Type Code: the task type that was assigned to the run when it was created.

- Task Name: the task’s name as designated when the task was added.

- Task Lookup: the task’s secondary description, or nickname, that was assigned when the task was added.

- Status: the task’s status (Active/Inactive).

- Create Time: the date and time when the task was created.

- Update Time: the date and time when the task was updated.

- Links:

- File Log: navigates to the Manage File Logs page and displays the current context to provide further information. (Note: this link only appears when a log exists.)

- XML Editor: displays the Tasks XML Editor. This link only appears for ETL tasks since they have XML embedded in the YAML. (See Editing ETL Task XML with the Tasks XML Editor for more information.)

- View: opens the View Task dialog box, which displays the same data as provided in the data table row as well as task description and locked flag information.

- Edit: opens the Manage Tasks dialog box, which provides the same fields as the View Tasks dialog box, and allows you to edit the Locked Flag and Status.

- YAML: opens the Task Definition dialog box, which allows you to view and edit the associated YAML. (See Editing Task YAML for more information.)

- Task Log navigates to the Task Log page and displays the current context to provide further information.

Next in This Section

- Adding Tasks

- Viewing Task Information

- Managing Task Information

- Viewing Task Definition YAML

- Editing Task YAML

- Editing ETL Task XML with the Tasks XML Editor

- Executing Tasks

Adding Tasks

to add a task, do the following:

1. On the Manage Tasks page, click Add Task.

2. In the Create Tasks dialog box, enter the following:

1. In the Task Type Code field, enter the appropriate type:

- Dataset Export (DSEXP)

- ETL on core schema (ETL)

- Fact audit (FACT_AUDIT)

- Geocode data (GEO)

- Point Aggregation (PAGG)

- Relation aggregation or derivation (RAGG)

- RAN maintenance (RANMAINT)

- Scenario to run calc (SCENARIO)

- Time Aggregation (TAGG)

3. In the Task Name field, enter a descriptive name.

4. In the Task Lookup field, enter a value for task lookup. The task lookup value can be any unique alphanumeric value without spaces. A common practice is to use the task name without spaces in uppercase.

5. In the Task Desc field, enter a short description of the task function.

6. From the Status drop-down list, select whether the task is Active or Inactive. Only active tasks may be executed

7. In the Task Definition field, enter the template YAML for the task type. (See Editing Task YAML for more information.)

Viewing Task Information

The View Tasks dialog box is accessed from the View link, which is found in the Manage Tasks page’s data table. This feature offers additional information to what is displayed in the Manage Tasks data table.

Fields and Links

- ID: the task’s system assigned, unique identifier.

- Task Type Code: the type code type that was assigned to the task when it was added. (See Adding Tasks for more information.)

- Task Name: the name that was assigned to the task when it was added.

- Task Lookup: the task’s secondary description, or nickname, that was assigned when the task was added.

- Task Desc: additional information that describes the task’s function.

- Locked Flag: the task’s lock flag value (Locked/Unlocked).

- Status: the task’s status (Active/Inactive).

- Create Time: the date and time when the task was created.

- Update Time: the date and time when the task was updated.

- Task Definition link: contains the YAML link, which opens the Task Definition dialog box. See Viewing Task Definition YAML for more information.

Buttons and Links

- Edit button: opens the Manage Tasks dialog box. See Managing Task Information for more information.)

- Cancel button: closes the dialog box.

- Execute Now link: opens the Execute Tasks Details dialog box. (See Executing Tasks for more information.)

Managing Task Information

The Manage Tasks dialog box opens when you click the Edit link on the task data table row or the Edit button on the View Tasks dialog box.

Fields and Links

- ID: the task’s system assigned, unique identifier.

- Task Type Code: the type code type that was assigned to the task when it was added. (See Adding Tasks for more information.)

- Task Name: the name that was assigned to the task when it was added.

- Task Lookup: the task’s secondary description, or nickname, that was assigned when the task was added.

- Task Desc: additional information that describes the task’s function.

- Locked Flag: drop-down list that allows you to set the task's lock flag (Locked/Unlocked).

- Status: drop-down list that allows you to set the task's status (Active/Inactive).

- Create Time: the date and time when the task was created.

- Update Time: the date and time when the task was updated.

- Task Definition link:contains the YAML link, which opens the Task Definition dialog box. See Viewing Task Definition YAML for more information.

Buttons and Links

- Execute Now link:opens the Execute Tasks Details dialog box. (See Executing Tasks for more information.)

- Save button: allows you to save any changes that have been made; the button is disabled until a field has been modified.

- Cancel button: closes the dialog box.

Viewing Task Definition YAML

The Task Definition dialog box displays the YAML code that defines the task. From this dialog box, you may view the parameters defined in the YAML or edit the content by clicking Edit. See Editing Task YAML for details.

Editing Task YAML

The YAML that defines a task is edited from the Task Definition dialog box, which is accessed by clicking the YAML link in either the Task Data Table or the View Tasks dialog box. Once the Task Definition dialog box is opened, click Edit to make the Task Definition YAML editable.

Each task type has unique requirements that are specified in its YAML. You define your task by updating generic YAML with task parameters. All tasks begin with:

---
task:
# <name or descriptor> task definition

# -------------------------------------------------------
name: task_<task name>
module_name: dr.task.<task-code> #.py is always assumed by python
task_method_name: startTask
post_task_method_name: postTask

Next in This Section

- YAML Task Templates

YAML Task Templates

With the exception of the ETL task, all tasks require that you add and modify template YAML to the Task Definition. The templates provide comments (beginning with #) that provide usage information. For example, if the requirement is to add a file format, the usage instructions would be:

# CSV, JSON, XML

DSEXP: Dataset Export

Exports dataset values in the format specified.

---
task:
# Dataset export task definition
# -------------------------------------------------------
name: task_dsexp
module_name: dr.task.dsExp #.py is always assumed by python
task_method_name: startTask
post_task_method_name: postTask

debug_flag: False
header_flag: False
delimiter: "|"
quoting:  none
footer_flag: False
export_format: CSV # Formats: CSV, JSON, XML
point_format: name # Formats: ID, lookup, name, verbose
target_name: XXX
target_date_format: ".%Y%m%d."
target_extension: txt
target_directory: /XXX/XXX
target: XXX
#blacklistedFields: ['point', 'startDate']

ETL

The ETL task contains YAML with embedded XML. Some XML data may only be added in the Task Description, while others may be added and modified with the Tasks XML Editor.

---
task:
# etl task definition
# -------------------------------------------------------
name: etl
module_name: dr.task.etl #.py is always assumed by python
task_method_name: startTask
post_task_method_name: postTask

XML:
<etl xmlns="http://dataraker.net/etl.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://dataraker.net/etl.xsd">
<source desc="XXX" name="mr" xsi:type="source_type_file"
file_archive_flag="true" file_compress_flag="false">
<source_name></source_name>
<conn_name></conn_name>
<file_path>/XXX/XXX/</file_path>
<archive_path>/XXX/XXX/archive</archive_path>
<file_mask>XXX.XXX.[%Y.%m.%d].*.XXX</file_mask>
</source>
<formats>
<format xsi:type="format_type_fixed_width"
name="XXXX_fixed_width" desc="XXX" line_terminator="\n"
min_row_length="154" max_row_length="1000">
<format_name></format_name>
<format_options></format_options>
<fields xsi:type="field_type" length="XXX" start_pos="1"
name="PTC" data_type="string" date_format=""></fields>
<fields xsi:type="field_type" length="XXX" start_pos="1"
name="PARENT_PTC" data_type="string" date_format=""></fields>
<fields xsi:type="field_type" length="XXX" start_pos="1"
name="VAL" data_type="string" date_format=""></fields>
</format>
</formats>
<etl_options>
<option name="read_batch_size" value="10000"/>
<option name="point_batch_size" value="1000"/>
<option name="target_batch_size" value="10000"/>
<option name="log_all_errors" value="true"/>
<option name="debug_table" value="false"/>
<option name="debug_file" value="false"/>
<option name="auto_trim" value="true"/>
<option name="max_error_count" value="0"/>
</etl_options>
<global_variables></global_variables>
<variables>
<variable is_active="true" name="point_lookup">
<value>XXXX</value>
</variable>
</variables>
<points>
<point duplicates="last" is_active="true" name="PTC"
xsi:type="point_lookup_type_ins">
<point_type_code></point_type_code>
<point_lookup></point_lookup>
<point_name></point_name>
<point_tag></point_tag>
<point_utc_offset></point_utc_offset>
<point_desc></point_desc>
</point>
<point duplicates="last" is_active="true" name="PARENT_PTC"
xsi:type="point_lookup_type_ins">
<point_type_code></point_type_code>
<point_lookup></point_lookup>
<point_name></point_name>
<point_tag></point_tag>
<point_utc_offset></point_utc_offset>
<point_desc></point_desc>
</point>
</points>
<variables_pt>
</variables_pt>
<targets>
<target xsi:type="target_type_metric" name="XXX"
duplicates="max" is_active="true">
<fact_attr>
<attr name="fact_lookup" value="XXX"/>
<attr name="fact_name" value="XXX"/>
<attr name="fact_category" value="XXX"/>
<attr name="fact_sequence" value="0"/>
<attr name="time_basis" value="XXX"/>
<attr name="agg_method" value="n/a"/>
<attr name="infer_delete_flag" value="false"/>
</fact_attr>
<point>P_PTC</point>
<start_time>XXX</start_time>
<filters>
<filter>P_PTC</filter>
</filters>
<value>F_VAL</value>
</target>
<target xsi:type="target_type_relation" name="XXX"
duplicates="last" is_active="true">
<fact_attr>
<attr name="fact_lookup" value="XXX"/>
<attr name="fact_name" value="XXX"/>
<attr name="fact_category" value="XXX"/>
<attr name="fact_sequence" value="0"/>
<attr name="time_basis" value="n/a"/>
<attr name="agg_method" value="n/a"/>
<attr name="infer_delete_flag" value="false"/>
</fact_attr>
<point>P_PTC</point>
<start_time>XXX</start_time>
<filters>
<filter>P_PTC</filter>
</filters>
<parent_point>P_PARENT_PTC</parent_point>
</target>
</targets>
</etl>

Once the data has been entered in the XML elements, you may use the Tasks XML Editor to modify your settings. See Editing ETL Task XML with the Tasks XML Editor for information on modifying the XML.

FACT_AUDIT: Fact Audit

The fact audit task allows you to perform a fact audit of generated or loaded data (METRIC, INT_METRIC, and/or RELATION fact types). The fact audit looks for trends in data.

---
task:
# fact audit task definition
# -------------------------------------------------------
name: task_factaudit
module_name: dr.task.factAudit #.py is always assumed by python
task_method_name: startTask
post_task_method_name: postTask

targets: [
# [ 'METRIC', '*'],
# [ 'INT_METRIC', '*'],
# [ 'RELATION', '*'],

]

# subscriber_list: [] # Default behavior. Will execute all subscribers as set in FactAudit
# worker code
subscriber_list: ['FASDBAudit'] # Optionally override the subscribers you want to run for.
# If defined subscriber is invalid its execution is skipped

# The targets criteria may be limited by substituting a point type code for the '*' wildcard,
# which will audit facts for all points types. For example, if you want metric facts
# for an electric meter, the target would be:
targets: [
[ 'METRIC', 'EM'],
]

FOPS: File Operations

File operations rename files.

---
task:
# ETL trigger for CSS Demand Reads
# -------------------------------------------------------
name: task_file_ops
module_name: dr.task.fileOps #.py is always assumed by python
task_method_name: startTask
file_ops:
- fops_rename:
is_active: True
conn_type: SFTP
conn_name: ssh_sftp_int
content_keyword_inspect:
file_path: /XXX/XXX
source_file_pattern: XXX.XXX.[%Y.%m.%d].*.txt
filename_keyword_exclude: xxx
target_file_pattern: XXX.XXX.[%Y.%m.%d].*.txt
target_task_env: XXX
target_task_lookup: YYYY
target_task_type_code: ETL

GEO: Geospatial

---
task:
# GeoSpatial task definition
# -------------------------------------------------------
name: task_geospatial
module_name: dr.task.geoSpatial #.py is always assumed by python
task_method_name: startTask
post_task_method_name: postTask

debug_flag: False
filter_limit: 1000
proxy: http://geocoding.dataraker.net/geocoder
#relations: [['PTC','PARENT_PTC']]

PAGG: Point Aggregation

The point aggregation task aggregates data from child to parent types such as from a meter to a line transformer.

Inputs:

- PTC: point type code for child.

- Mode: fact type (INT, INT_METRIC, RELATION). See Fact Types.

- factLookupList: the facts within the mode that you wish to aggregate.

- targetPTCList: the point type that you wish to aggregate to.

- targetMetricList: the metrics that you want returned.

---
task:
# pagg task definition
# -------------------------------------------------------
name: task_pagg
module_name: dr.task.pagg #.py is always assumed by python
task_method_name: startTask
post_task_method_name: postTask
min_batch_count: 30

# List of definitions in the following format:
# [[ PointTypeCode, Mode, [factLookupList], [targetPTCList], [targetMetricList] ], ...]
# Example:
#    targets: [
#       ['EM', 'INT_METRIC', ['HOURLY_KWH'], ['PRMS'], ['SUM']],
#       ['EM', 'COUNT', ['DAILY_POWER_OUTAGE_FLAG'], ['PRMS'], []],
#       ['EM', 'METRIC', ['DAILY_KWH_CSM'], [], ['AVG','SUM','P10','P90']]
#    ]
targets: [
#['PTC', 'METRIC', ['FACT_LOOKUP'] ,['PARENT_PTC'], ['CNT','SUM']]
]

If the target metric list is empty, all metrics will be returned.

RAGG: Relation Aggregation

The relation aggregation connects all parent point types to their child point types. There is no configuration needed.

---
task:
# ragg task definition
# -------------------------------------------------------
name: task_ragg
module_name: dr.task.ragg #.py is always assumed by python
task_method_name: startTask
post_task_method_name: postTask

output_to_file_flag: False # sample output only for testing purpose
source_ptc_list: ['*'] # this results in standard behavior

RANMAINT: Relation and Numeric Attribute Maintenance

The relation and numeric attribute maintenance task caches attribute data to enhance system performance. This task is performed after daily core processing. The cache is valid for 24 hours.

---
task:
# RAN Maint task definition
# -------------------------------------------------------
name: task_ranmaint
module_name: dr.task.cacheheatup #.py is always assumed by python
task_method_name: startTask
post_task_method_name: postTask
source_ptc_list: ['*']

SCENARIO: Scenario

The scenario task executes a scenario; see Scenarios for information.

---
task:
# Scenario task definition
# -------------------------------------------------------
name: task_scenario
module_name: dr.task.scenario #.py is always assumed by python
task_method_name: startTask
post_task_method_name: postTask
debug_flag: False
variables:
DUMMY: true

TAGG: Time Aggregation

The time aggregation task allows you to aggregate data over time.

Inputs

- Point Type Code: what point type to use in the time aggregation.

- Mode: the data fact type from:

- METRIC (daily)

- INT_METRIC (sub-daily)

- EVENT (head-end system events)

- Time Aggregation Basis: what time frame to aggregate to.

- Day Types: the type of days to use in the aggregation:

- WEEKDAY: only weekdays (Monday through Friday)

- WEEKEND: only weekend days (Saturday through Sunday)

- ALL_DAYS: all days (Sunday through Saturday)

- Fact Types: which facts to include in the aggregation.

- Target Metrics: what metrics (e.g., average, count, etc.) to return.

---
task:
# tagg task definition
# -------------------------------------------------------
name: task_tagg
module_name: dr.task.tagg #.py is always assumed by python
task_method_name: startTask
post_task_method_name: postTask
# List of definitions in the following format:
# [[ (PointTypeCode, Mode, TAggBasis, [DayTypes], [factLookupList], [targetMetricList])] ..]
# If DayTypes is not specified it defaults to all the day types supported:
# [ 'WEEKDAY' , 'WEEKEND', 'ALL_DAYS' ]
# If targetMetricList is not specified it defaults to all the target
# metrics supported: [ 'CNT','SUM','AVG','SSQ','MAX','MXD']
# Example:
#    targets: [
#       [ ACCNT, METRIC, MONTHLY, [], [SUM_DAILY_KWH_CSM], [] ],
#       [ EM, INT_METRIC, DAILY, [], [HOURLY_KWH], [] ],
#       [ EM, EVENT, MONTHLY, [], [DAILY_MAGNETIC_FLAG], [] ]
#    ]
#targets: [ [ ACCNT, METRIC, MONTHLY, [], [SUM_DAILY_KWH_CSM], [] ] ]

targets: [
#[ PTC, INT_METRIC, DAILY, ['ALL_DAYS'], ['FACT_LOOKUP'], ['SUM','CNT']]
]

Editing ETL Task XML with the Tasks XML Editor

ETL XML may be edited with the Tasks XML Editor. All elements shown in the template YAML's XML element correspond with information found in the XML Editor; however, it is best to fill in as much detail as possible with the Task Definition because some fields may be edited, but not added with the XML Editor.

Executing Tasks

Tasks can be executed manually or scheduled to run periodically by creating a run (see Managing Run Information for details). The Execute Tasks Details dialog box can be accessed from the Execute Now link found in the Manage Tasks’ View Tasks or Edit Tasks dialog boxes.

1. On the Manage Tasks page, click either the View or Edit link to display the corresponding dialog box.

2. From the dialog box, click the Execute Now link.

3. In the Execute Tasks Details dialog box, do the following:

1. Verify the name field (display only). This is the name assigned to the task when it was added. (See Adding Tasks for more information.)

2. In the Start Date field, enter the date that the task should be executed.

3. In the End Date field, enter the date that the task should be stopped if it is still running.

4. In the Priority drop-down, select the task’s priority on the node. Base values include:

- Highest

- Default

- Lowest

5. (Optional) In the Failure Email field, enter an email address to send a message to if the task ends unexpectedly.

6. Verify the lookup field (display only). This is the task’s lookup name as designated when the task was added.

7. In the Type drop-down, select the MANUAL execution method.

8. In the Run Mode drop-down, select the type of task. Base values are:

- STD (Standard)

- Dynamic – not currently being used.

9. (Optional) in the Success Email field, enter an email address to send a message to if the task completes successfully.

4. Click OK.

Manage Task Logs

Once the Task ends, the system automatically generates logs that describe the task’s process. The Manage Task Logs provides these details about the task’s processing information.

Fields

The top section of the Manage Task Logs page allows you to search for task logs matching criteria in the fields:

- Task ID: the task’s system-assigned, unique identifier.

- Task Log ID: the task log’s system-assigned, unique identifier.

- Refresh Rate: the frequency, in seconds, that the results should be automatically updated.

- Run Log ID: the run log’s system-assigned, unique identifier. By entering this criteria, you will search for task logs that were executed within a specific run.

Buttons

- Get Logs: initiates a search for logs matching criteria set in the fields.

Data Table

- Task Log ID: the task log’s system-assigned, unique identifier.

- Server Process: the server node that executed the task.

- Task Start Time: the date and time that the task started.

- Task End Time: the date and time that the task ended.

- Transaction: the associated transaction’s system-assigned, unique identifier

- Links:

- Explore: opens the Explore Calc dialog box. (See Explore Task Results for more information.)

- View: opens the View Task Logs dialog box, which displays processing information such as the execution type, addresses to email if the task was successful or failed, and the task’s status code. (See View Task Logs for more information).

- Transaction: Navigates to the Manage Transactions page and displays the current context to provide further information.

Next in This Section

- View Task Logs

View Task Logs

The View link in the Manage Task Logs’ data table launches the View Task Logs dialog box . This dialog box displays some of the same run log information as well as some additional information:

Fields

- Task Log ID: the task log’s system-assigned, unique identifier.

- Run Log ID: the run log’s system-assigned, unique identifier.

- Task ID: the task’s system-assigned, unique identifier.

- Execution Type: the method that the task was initiated.

- Success Email: the address to send a notice when the run completes.

- Failure Email: the address to send a notice in case the run fails.

- Server Process: the server node that executed the task.

- Run Mode: the run’s processing category.

- Data Start Time: the earliest date and time of the data that was included in the run.

- Data End Time: the latest date and time of the data that was included in the run.

- Task Start Time: the date and time that the task started.

- Task End Time: the date and time that the task ended.

- Status Code: indicates if the run is In Progress, ran Successfully, or had a Fatal error.

- Task Progress: the task’s current status (percentage of completion).

- Task Priority: indicates the task’s priority on the node.

- Scenario ID: the associated scenario’s ID.

- Filter ID: the associated filter’s ID.

- Filter Seq ID: the filter’s system-generated, unique identifier as listed on the filter sequence table

Buttons

- Cancel button: closes the dialog box.

File Log

Whenever a file is uploaded into the system, an associated log is generated. This page allows you to search for and view related information.

File Log Fields

The top section of the Manage File Logs page allows you to search for file logs matching criteria in the fields:

- Task ID: the file’s system-assigned, unique identifier.

Buttons

- Get Logs: initiates a search for logs matching criteria set in the field.

Data Table

- File Log ID: the file’s system-generated, unique identifier.

- Task ID: the associated task’s system-generated, unique identifier.

- File Name: the name that was given to the file prior to loading.

- File Timestamp: the date that the file was created.

- File Date: the date and time that was included inside the file. (Note: time is optional because since not all files arrive with that information.)

- Links:

- View: opens the View File Logs dialog box, which displays the same data as provided in the data table row. In addition, it includes the file size, file line count, and create transaction ID.

Next in This Section

- View File Logs

View File Logs

Fields

The View link in the Manage File Logs’ data table launches the View File Logs dialog box. This dialog box displays some of the same file log information as well as some additional information:

- File Log ID: the file log’s system-generated, unique identifier.

- Task ID: the associated task’s system-generated, unique identifier.

- File Name: the name that was given to the file prior to loading.

- File Size: the file’s size in bytes.

- File Line Count: the number of lines that the file contains (based on the number of line breaks within the file).

- File Timestamp: the date and time that the file was last modified.

- File Date: the date and time that was included inside the file. (Note: time is optional because since not all files arrive with that information.)

- Target File Name: the name that the system assigns the file once it’s moved to the internal file system.

- Create Time: the date and time that the file log entry was created (this should coincide with the time that the file was loaded into the system).

- Create Transaction ID: the transaction’s ID that was responsible for creating the file log entry when the file was loaded.

Buttons

- Cancel button: closes the dialog box.

Manage Transactions

Transactions are created to document system events; for example, a transaction is created when you log in, or whenever a task is executed. Transactions are typically used to troubleshoot failed processes, see how many entities were included within a task, and review system performance.

This page allows you to search for and view information related to transactions. This page is commonly accessed through links from other pages (such as, the Activity Dashboard, Manage Runs, and Manage Tasks) because it is necessary to know the system-generated transaction identifier or name in order to find a specific transaction.

Fields

The top section of the Manage Transactions page allows you to search for transactions matching criteria in the fields:

- Transaction ID: the transaction’s system-generated, unique identifier.

- Task Log ID: the task log’s system-generated, unique identifier.

- Transaction Type Code: drop-down list to search by the transaction type.

- Transaction Name: the system-assigned name to the transaction.

- Refresh Rate: the frequency, in seconds, that the results should be automatically updated.

Buttons

- Get Transactions: allows you to search for transactions matching the criteria set in the top section’s fields.

Data Table

- Transaction ID: the transaction’s system-generated, unique identifier.

- Task Log ID: the task log’s system-generated, unique identifier.

- Transaction Type Code: the transaction’s system-assigned type.

- Transaction Name: the system-assigned transaction name.

- Server Process: the server node that executed the task.

- Transaction Value Count: displays the number of values that were used to complete the transaction.

- Links:

- View: opens the View Transactions dialog box, which displays the same data as provided in the data table row. In addition, it includes transaction description, status code, transaction progress, transaction start time, transaction end time, and create time.

- Transaction Log navigates to the Transaction Log page so that you can see further information.

- Batch Log navigates to the Batch Log page so that you can see further information.

Transaction Log

Whenever transactions are created, they generate an associated log within the system. This page allows you to search for and view related information.

Fields

The top section Manage Transaction Logs page allows you to search for logs matching criteria added to the fields:

- Transaction ID: the transaction’s system-generated, unique identifier.

Buttons

- Get Logs: initiates a search for logs matching criteria set in the field.

Data Table

- Transaction ID: the transaction’s system-generated, unique identifier.

- Message Type: the log’s system assigned type.

- Log Description: additional information that describes why the log entry was created.

- Create Time: the date and time that the log was generated.

Batch Log

Whenever background processes execute, they generate an associated log within the system. This page allows you to search for and view related information.

Fields

The top section of the Manage Batch Logs page allows you to search for logs matching criteria added to the fields:

- Batch ID: the batch’s system-assigned, unique identifier.

- Transaction ID: the transaction’s system-assigned, unique identifier.

- Refresh Rate: the frequency, in seconds, that the results should be automatically updated.

Buttons

- Get Logs: allows you to search for batch logs matching the criteria set in the top section’s fields.

Data Table

- Batch ID: the batch’s system-assigned, unique identifier.

- Transaction ID: the transaction’s system-assigned, unique identifier.

- Batch Type Code: the associated batch code that was assigned when the batch job was added.

- Status Code: the batch process’ current status.

- Server Process: the server node that executed the batch job.

- Batch Value Count: the number of values loaded by the batch job.

- Create Time: the date and time that the batch log was generated.

- Links:

- View: opens the View Batch Logs dialog box, which contains the same information as the data table. In addition, it includes batch description, start time, end time, parent batch ID, and update time.

Next in This Section

- View Batch Logs

View Batch Logs

The View Batch Logs dialog box is accessed from the View link, which is found in the Manage Batch Log page’s data table. This feature offers additional information to what is displayed in the Manage Batch Log’s data table.

Fields

- Batch ID: the batch log’s system-generated, unique identifier.

- Transaction ID: the associated transaction’s system-generated, unique identifier.

- Batch Type Code: the batch type that was assigned to the batch when it was added.

- Batch Description: a short description that explains the batch job’s function.

- Start Time: the date and time that the batch job was executed.

- End Time: the date and time that the batch job stopped.

- Status Code: indicates if the batch job is In Progress, ran Successfully, or had a Fatal error.

- Server Process: indicates the batch’s current status.

- Batch Value Count: the number of entities that were processed.

- Parent Batch ID: the parent batch’s system-generated, unique identifier, if applicable.

- Create Time: the date and time that the batch was added.

- Update Time: the date and time that the batch was most recently updated.

Buttons

- Cancel button: closes the dialog box.

Fact Audit

Facts can be imported into the system or generated through an analytics process. A fact is a single characteristic or event that can be used to look for occurrences of that characteristic; for example, a fact can be defined as a non-reporting meter event. You can then use the fact in a filter to search for all the instances of non-reporting meters (see Fact Data Export Filters for more information).

Facts are typically updated daily through core processes. The Manage Fact Audits page assists in troubleshooting fact or finding more information about the results from the core processes. You can see the number of data points (Point Count) that are available as well as the number of instances that are associated to the fact (Value Count).

This page allows you to view fact audit data that was selected in the Filters Menu. The Manage Fact Audits page is accessible from the Manage Operations menu and from the Administer Facts page.

Fields

- The top section of the Manage Fact Audits page allows you to search for fact audits matching criteria in the fields:

- Fact ID: the fact’s system-generated, unique identifier.

- Fact Type Code: allows you to search for facts that are a specific fact type. (See Understanding Facts for more information.)

- Time Basis: allows you to search for facts that are based on a specific period of time. (See Time Basis for more information.)

- Fact Category: allows you to search for facts that are a specific fact category.

- Fact Name: the name that was given to the fact when it was added.

- Fact Lookup: the secondary description, or nickname, that was given to the fact when it was added. (This field is case sensitive.)

- Fact Source: the data or calculation that produced the fact.

- Role Type Code: allows you to search for facts associated to a specific role type. (See Drawer Menu for more information about the base roles that come with the system.)

- Aggregation Method: the mathematical function used to calculate the fact.

- Parent PTC: the point type code associated to a parent object.

Buttons

- Get Fact Audits: allows you to search for facts matching the criteria set in the top sections fields.

Data Table

- Fact ID: the fact’s system-assigned, unique identifier.

- Point Type Code: the point type code that was used within the fact.

- Fact Type Code: the fact type, if any, that was assigned to the fact when it was added.

- Role Type Code: the associated role type, if any, that was assigned to the fact when it was added.

- Fact Name: the name that was give to the fact when it was added.

- Fact Lookup: the secondary description, or nickname, that was given to the fact when it was added. (This field is case sensitive.)

- Audit Date: the date and time that the audit was run.

- Point Count: the number of potential data points identified to derive this fact.

- Value Count: the number of actual data points used to derive this fact.

- Parent Point Count: the number of parent data points that were used when deriving this fact.

- Links:

- View: opens the View Fact Audits dialog box, which contains the same information as the data table. In addition, it displays point type code and create transaction ID

Next in This Section

- View Fact Audits Information

View Fact Audits Information

Fields

The View Fact Audits dialog box opens when you click the View link on the Manage Fact Audits data table row.

- Fact ID: the fact’s system-assigned, unique identifier.

- Point Type Code: the point type code that was used within the fact.

- Fact Type Code: the fact type, if any, that was assigned to the fact when it was added. (See Understanding Facts for more information.)

- Fact Category: a grouping of like facts such as consumption facts.

- Time Basis: the period that consumption is viewed across data read intervals. (See Time Basis for more information).

- Role Type Code: the associated role type, if any, that was assigned to the fact when it was added. (See Drawer Menu for more information about the base roles that come with the system.)

- Fact Name: the name that was give to the fact when it was added.

- Fact Lookup: the secondary description, or nickname, that was given to the fact when it was added.

- Audit Date: the date and time that the audit was run

- Point Count: the number of potential data points identified to derive this fact.

- Value Count: the number of actual data points used to derive this fact.

- Parent Point Count: the number of parent data points that were used when deriving this fact.

- Create Time: the date and time that the batch was added.

- Create Transaction ID: the associated transaction’s system-defined, unique identifier when the fact was created.

Buttons

- Cancel button: closes the dialog box.

Run/Task Operations

The Run/Task Operations page provides an overview of the system performance data for runs and tasks. The page table rows link back to data from the Manage Runs and Manage Tasks pages.

Each of the data tables provides performance charts with hover text for the various points along the transaction curve:

If you select Task, the page will update with a Task type selection drop-down list:

Note: See Adding Tasks for the list of task types.

You may select a row in the Task Logs table to drill down on the data:

If you select Summary, the Details tables will be replaced with Summary tables:

Manage Analytics

The DataRaker analytics functions provide the means to find and manipulate data. Each function corresponds to a Manage Analytics sub-menu:

- Filters : Filters reduce the scope of points to be evaluated when searching for and analyzing data.

- Datasets : Datasets provide criteria for gathering and aggregating data. For example, given a population of meters (defined by a filter), what is the average daily consumption in kWh?

- Calcs : Calcs define logical and mathematical operations to perform on data defined by a dataset.

- Scenarios : Scenarios are a collection of calcs to be executed sequentially on a population found using a filter. Scenario results are based on the calcs and filter associated with the scenario. A different scenario that uses the same calcs, but a different filter (population), would produce different results.

For example, to find meters with low consumption, which may indicate that a meter is malfunctioning, you could analyze data over a short time frame (e.g., daily) or a longer time frame (e.g., monthly) to look for patterns. Either way, you would need the following components:

- Filters: the daily or monthly versions share one filter that retrieves all meters that are active, fit a particular configuration, and do not have a seasonal usage restriction.

- Dataset: two datasets are required. Each dataset pulls consumption related information; however, one will be on a monthly processing basis and the other will be on a daily processing basis.

- Calc: two calcs are required for the two different time basis (daily versus monthly). Either calc would evaluate data from the corresponding dataset (daily or monthly) and determine if the consumption profile fits that of a “breaking/slowing” meter.

- Scenario: one scenario may be created that will evaluate both calcs against the data found using the filter.

Next in This Section

- Manage Analytics Common Features

- Filters

- Datasets

- Calcs

- Scenarios

Manage Analytics Common Features

Next in This Section

- Understanding Time Basis in Analytics

- Understanding DataRaker XML Editors

Understanding Time Basis in Analytics

In DataRaker, time basis relates to how often data is received and how much time does the data represent. For example, electric meter register reads and consumption data each provide metric data in kWh; however, they have different time bases because register reads are received at various times of the day and have variable timestamps (ACTUAL time basis) and daily consumption represents a calculation of a day’s worth of data and will have a timestamp of “midnight-to-midnight” (DAILY time basis). Commonly used time basis are Yearly, Monthly, Weekly, Daily, and Actual, which is often aggregated from shorter intervals.

Understanding DataRaker XML Editors

Filters, Datasets, and Calcs are defined as XML that is stored in the metadata database. DataRaker XML Editors provide a user interface for creating these definitions.

The XML Editors have the same basic structure:

1. The tree-view (on the left) provides a hierarchical view of the XML element.

2. The definitions (on the right) display the content of the selected element, which may be overview data or fields for defining the selection.

When you click on a tree-view folder (), summary information is displayed for the elements within the folder. The Summary pane also provides a Create New button (for example, Create New Time Windows) to create a new element of the selected type.

The individual elements in the tree-view provide fields for defining the node. For example, selecting the START node in the tree-view allows you to define its settings.

Note: All fields with a red asterisk () are required to have values.

Filters

Filters reduce the scope of points to be evaluated when searching for and analyzing data. Once a filter has been defined, it may be selected from the an Advanced Filter option and applied to scenarios (see Scenarios for information).

Next in This Section

- Manage Filters Page

- Adding Filters

- Viewing Filter Information

- Managing Filter Information

- Defining and Editing XML with the Filter XML Editor

- Cloning a Filter

- Copying a Filter

- Viewing the Filter Definition XML

- Executing Filters

Manage Filters Page

The Manage Filters page provides fields and controls for filtering existing filters and a data table that lists existing filters. The Manage Filters data table displays filters based on the selected Type and that either match the selected Role or have no associated role.

Navigate to the Manage Filters page by selecting Filters from the Analytics section of the Manage menu.

Filter Fields

The top section of the Manage Filters page allows you to find existing filters that match criteria added to the fields:

- Filter ID: the system generated numeric identifier.

- Filter Category: a grouping of like filters.

- Filter Name: the filter's "friendly" name, which may contain spaces.

- Filter Lookup: the lookup value is an alternate name for the filter without spaces in uppercase text.

- Status: whether the filter is active or inactive. If it is active, it will be available to use in advanced searches and to assign to scenarios.

- Current Version: whether the filter is the current version in use.

Action Buttons

- Get Filters: initiates a search for filters matching criteria set in the fields.

Note: Alternately, you may press Enter when you finish adding content to the fields and click Submit when prompted to Reset or Submit.

- Add: opens the Create Filters dialog box that allows you to define a new filter. See Adding Filters for details.

Data Table

The data table contains information about the filters:

- ID: the system generated numeric identifier

- Sequence ID: a system generated number that is assigned based on whether the filter is original (Sequence ID = 1) or a cloned version (Sequence ID = 2 or higher). See Cloning a Filter for information.

- Filter Category: a grouping of like filters.

- Name: the filter's "friendly" name, which may contain spaces.

- Filter Lookup: the lookup value is an alternate name for the filter without spaces in uppercase text.

- Role: the role, if any, associated with the filter. Filters without an assigned role will have an n/a value.

- Status: whether the filter is active or inactive. If it is active, it will be available to use in advanced searches and to assign to scenarios.

- Current Version: whether the filter is the current version in use.

- Start Time: the date when the filter became active.

- End Time: the end date for the filter. For active filters, the end time is set to an arbitrary end date used in the system; when a new version of the filter becomes active, the end time is updated to equal the start time of the new version. See Cloning a Filter for information.

- Read Only: whether the filter is read-only or may be edited.

- Links:

- View: opens the View Filters dialog box, which displays the same filter data as provided in the data table row as well as buttons and links that allow you to edit or configure the filter. See Viewing Filter Information for more information.

- Edit: opens the Manage Filters dialog box, which displays an editable view of the filter information shown in the View Filters dialog. SeeManaging Filter Information for more information.

Note: It is possible to directly access and edit the filter criteria (XML) from the Manage Filters dialog; however, the primary intent of the Edit features is to enter or updated information about the filter.

- XML: displays the Filter Definition dialog box, which shows the configuration XML for the filter. See Viewing the Filter Definition XML for more information.

- XML Editor: opens the filter definition XML in the Filters XML Editor. See Defining and Editing XML with the Filter XML Editor for information.

- Search field: allows you to search for a value in the table.

Click the expand button () at the beginning of a row to reveal a list view of the row values and information on the associated point type code; click the minimize button () to hide the extra information. The information contains the data found in the table plus:

- Point Type Code: point type for the filter.

- Filter Description: a description of the filter purpose.

- Lock Status: whether the filter is locked to editing.

See Viewing Filter Information for information on field information.

The data table may be exported by clicking the export data icon ().

Adding Filters

When creating a filter, you must know three of the following four data attributes:

- Point ID: the object you are looking for.

- Fact: the fact you are looking for.

- Time: the time frame for your analysis.

- Value: the data value, which typically is associated with a fact.

Example

If you are looking for the consumption value for an electric meter on a given day, you will need to provide the meter identifier, the consumption fact category, and the time, which (in this case) is the date.

Seeking daily kWh derived on 2/2/2014 for EM point_name = EM-123

- Time: 02-Feb-2014

- Fact: fact_lookup = DAILY_KWH_USAGE_DERIVED

- Point: point_name = EM-123

1. On the Manage Filters page, click Add Filter. The Create Filters dialog box will open.

2. Enter the following:

1. From the Filter Category drop-down list, select an appropriate category.

2. In the Name field, enter a descriptive name. For example, EM Zero Consumption.

3. In the Filter Lookup field, enter a value for filter lookup. The filter lookup value can be any unique alphanumeric value with uppercase text and without spaces. A common practice is to use the filter name without spaces. For example, EM_ZERO_CONSUMPTION.

4. The Filter Role drop-down list allows you to specify if the filter is applicable to all roles or to the role selected in the Drawer Menu Role field. If you wish it to be applicable to all roles, leave the selection blank; if you wish the filter to be role specific, select the role (lookup value) from the drop-down list.

5. In the Filter Description field, enter a short description of the filter function. For example, This filter finds electric meters with zero consumption.

6. From the Status drop-down list, select whether the filter is Active or Inactive.

Note: Filters cannot be deleted through the user interface, but may be set to inactive.

7. Leave the Filter Definition blank.

Note: The Filter Definition XML is typically defined with the Filter XML Editor; however, if you wish to start with the Filter Definition XML from another filter, you may copy it and paste it in the Filter Definition field (see Viewing the Filter Definition XML for details).

3. Click Save. The Create Filters dialog box will close.

Note: Click Cancel to close the dialog box without saving the filter; a dialog will open asking you to confirm that you want to close the Create Filters dialog.

Viewing Filter Information

The View link, in the filter data table row, launches the View Filters dialog box, which displays filter properties and allows you to view the filter definition XML, edit the filter properties, create a new version of the filter, create a copy of the filter, or execute the filter.

Fields

- ID: the system generated numeric identifier.

- Sequence ID: a system generated number that is assigned based on whether the filter is original (Sequence ID = 1) or a cloned version (Sequence ID = 2 or higher).

- Point Type Code: the code for the object type.

- Filter Category: a grouping of like filters.

- Name: the filter's "friendly" name, which may contain spaces.

- Filter Lookup: the lookup value is an alternate name for the filter without spaces in uppercase text.

- Filter Description: an optional description of the filter.

- Status: whether the filter is active or inactive. If it is active, it will be available to use in advanced searches and to assign to scenarios.

- Lock Status: whether the filter is locked to editing; only locked filters may be executed.

- Start Time: the date when the filter became active..

- End Time: the end date for the filter. For active filters, the end time is set to an arbitrary end date used in the system; when a new version of the filter becomes active, the end time is updated to equal the start time of the new version. See Cloning a Filter for information..

- Read Only: whether the filter is read-only (not editable).

- Filter Definition: contains the XML link, which opens the Filter Definition dialog box. See Viewing the Filter Definition XML for more information.

Buttons and Links

- Edit button: opens the Manage Filters dialog box. See Managing Filter Information for more information.

- Clone button: creates a new version of the filter. The clone's Sequence ID is the original filter's Sequence ID plus one; all other filter information is the same. Cloning allows you to keep the initial filter active while working on modifications to the filter parameters. See Cloning a Filter for details..

- Copy button: creates a copy of the filter having a unique ID. See Copying a Filter for details.

- Set Current button: makes the filter the active executable version.

- Cancel button: closes the dialog box.

- Execute Now link: executes the filter. See Executing Filters for more information.

Managing Filter Information

The Manage Filters dialog box opens when you click the Edit link on the filter data table row or the Edit button on the View Filters dialog box.

Note: The Manage Filters dialog is an editable version of the View Filters dialog.

From this dialog box, you may edit the filter information See Filters for field definitions.

Fields

- ID: the system generated numeric identifier.

- Sequence ID: the filter's version number.

- Point Type Code: the code for the object type.

- Filter Category: allows you to select a filter category, which is a grouping of like filters.

- Name: allows you to edit the filter's name

- Filter Lookup: allows you to edit the filter's lookup value.

- Filter Description: allows you to edit the optional description of the filter.

- Status: allows you to set whether the filter is active or inactive. If it is active, it will be available to use in advanced searches and to assign to scenarios.

- Lock Status: whether the filter is locked to editing.

- Start Time: the date when the filter became active.

- End Time: the end date for the filter. For active filters, the end time is set to an arbitrary end date used in the system; when a new version of the filter becomes active, the end time is updated to equal the start time of the new version. See Cloning a Filter for information.

- Read Only: whether the filter is read-only (not editable).

- Filter Definition: contains the XML link, which opens the Filter Definition dialog box. See Viewing the Filter Definition XML for more information.

Buttons and Links

- Save button: saves any changes to the filter information; the button is only active when a change has been made in one or more of the editable fields.

- Clone button: creates a new version of the filter. The clone's Sequence ID is the original filter's Sequence ID plus one; all other filter information is the same. Cloning allows you to keep the initial filter active while working on modifications to the filter parameters. See Cloning a Filter for details..

- Copy button: creates a copy of the filter having a unique ID. See Copying a Filter for details.

- Set Current button: makes the filter the active executable version.

- Cancel button: closes the dialog box.

- Execute Now link: executes the filter. See Executing Filters for more information.

Defining and Editing XML with the Filter XML Editor

The Filter XML Editor provides a user interface for creating and editing the filter definition XML. To create or edit a filter definition, click the XML Editor link in the filter's data table row. The Filters XML Editor dialog box will open allowing you to define the filter parameters.

The filter tree-view options are Time Windows and Filter Components.

Buttons

- Save&Close: saves the filter in its current state and closes the editor.

- Validate&Save: validates and saves the filter syntax and displays any errors.

- Close: closes the Filters XML Editor without saving changes.

Next in This Section

- Time Windows

- Filter Components

Time Windows

When you select Time Windows in the tree-view, the editor will update with the Create New Time Window button, which allows you to define time windows for the filter.

When you click the Create New Time Windows button, the tree-view will add a branch for the time window; the name will be in an editable state so you may enter a name directly in the branch or leave the system defined name.

Note: The system defined time window name is time_window_n, where n is a sequential number indicating the time window's order of creation.

Once the time window is created, select the time window name in the tree-view.

Select a time window Type from the drop-down list:

- Static Time Window: a time window with a defined start and end date. See Static Time Windows for information.

- Offset Time Window: a time window containing a time offset. See Offset Time Windows for information.

Next in This Section

- Static Time Windows

- Offset Time Windows

Static Time Windows

Static time windows have fixed start and end dates that are defined in the filter. When you select Static Time Window from the Type drop-down list, you may edit the parameters:

- Name: populates with the name from the tree-view; you may change the name in the tree-view or in the static time window Name field.

- Description: optional field that allows you to enter a description for the time window.

- Start Date: the start date for the filter criteria.

- End Date: the end date for the filter criteria.

Offset Time Windows

The offset time window allows you to create a time window that is offset from the start and/or end dates, which are input when the filter is executed.

An Offset Time Window has three unique sections:

1. Start Offset

2. End Offset

3. Time Basis

Note: When you execute the filter, you select the start and end dates. For reference purposes, these will be known as Data Start Date and Data End Date.

Start Offset

The Start Offset start date may be offset using one of two methods: simple offset or directional snapping.

- Simple offset offsets the start date by a selected time basis (Unit; e.g., day or month) and how many (Valueof) of those units .

- Snapping sets whether the start offset will expand backward in time or forward (snap direction).

End Offset

The End Offset is similar to the Start Offset with the exception that the resulting date must be later than the Start Offset date.

Time Basis

The Time Basis field allows you to interpret your data over a different time frame. For example, you could have a filter that looks for daily data, but have DataRaker Cloud Service evaluate data over the entire month.

Filter Components

Filter components describe the criteria (facts) on which data is filtered.

To add filter components, click the Filter Components label in the tree-view; the Filters XML Editor will display the filter components view that provides a button to create new filter components and a table that lists existing components.

To add a component, click Create New Filter Components. The tree-view will add a branch for the component and the Filters XML Editor will update with component options. You may create a custom component label or leave the system defined component name, which is filter_component_n, where n is a sequential number indicating the component's order of creation.

The available component fields are dependent on the fact type (Type) selection. The following table describes the possible options.

Field Name

Description

Type

The fact type drop-down lists all available fact types. See Fact Data Export Filters for information.

Time Window Name

Drop-down list populated with the names of the defined time windows.

Time Basis

Describes the time basis for the chosen fact. This is distinct from the time basis for the time window and describes the granularity of the data which is used to filter results.

Fact Category

Drop-down list is automatically populated with the fact categories belonging to the selected fact type.

Fact Name

Lists possible fact names within the selected fact category.

Parent Point ID

In a Fact Relation type component, you choose specific parent points based on the fact category and fact name.

The Visual selector tab allows you to search for potential points. Click Add Point ... to open a search dialog:

Select the search by type (Name, ID, Lookup, or Tag), enter the appropriate criteria, and click Find. The results will be displayed in a table; select the row containing the point of interest and click Add.

If you know the point information, you may manually enter it. Click the Manual entry tab; a form for adding Point IDs or Lookup values will be displayed.

Enter values separated by commas.

Value A/B

Metric or attribute data require beginning and ending values:

- In the case of qualitative data (attribute), Value A and Value B will represent the leading and ending characters.

- In the case of quantitative data (metric), Value A and Value B will represent the bounding numerals: greater than or equal to Value A, but less than Value B.

Note: Values A and B may be bounded by the minimum and maximum values available by entering the terms min and max in the Value A and Value B fields, respectively.

- Dates are entered as DD-MMM-YYYY (e.g., 01-JAN-2015).

Point ID

For a Point type component, like the Parent Point ID, you may choose the point to be used in the filter visually by searching (Visual selector) or manually by entering the ID in the field (Manual entry).

Set Operation

Drop-down list with operators to use to filter data. Operators include:

- Intersection (A and B)

- Union (A or B)

- Difference (A not B)

Cloning a Filter

The clone function, which is found in the View Filters and Manage Filters dialogs, creates a new version of the filter; the Sequence ID will be incremented by 1. The clone is unlocked when created, which allows it to be edited while the original is locked and ready to be executed (see Executing Filters); once the new version is ready to be put in production, the previous version's End Time is set to the new version's Start Time.

The cloning function has the following workflow:

1. In the View Filters or Manage Filters dialog, click Clone.

2. Edit the clone (see Defining and Editing XML with the Filter XML Editor).

3. When the clone is ready to be deployed, open its Manage Filters dialog and click Set Current, which will set the original filter's Locked Status to Unlocked (see Managing Filter Information) and the clone's Locked Status to Locked.

Copying a Filter

The copy function, which is found in the View Filters and Manage Filters dialogs, creates a new filter with a unique ID. Copying allows you to start a new filter from an existing one. The copy has a Sequence ID of 1 since it is the first version of a new filter.

Viewing the Filter Definition XML

From the Filter Definition dialog box, you can view or edit the filter definition XML. This allows you to quickly modify a parameter rather than opening the Filters XML Editor .

Filter Definition XML

<filter xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns="http://dataraker.net/filter.xsd" pointTypeCode="SSN_EM">
<time_windows>
<tw xsi:type="time_window_static_type" name="No Consumption">
<start_date>2014-01-01</start_date>
<end_date>2014-03-31</end_date>
</tw>
</time_windows>
<filter_components>
<fact_metric name="Consumption" tw_name="No Consumption"
xsi:type="findPointInt_source_type" factLookup="DAILY_KWH_USAGE_DERIVED"
operator="between" valueA="-10" valueB="1"></fact_metric>
</filter_components>
<set_operations>
<filter_component name="Consumption"></filter_component>
</set_operations>
</filter>

Copying Filter XML for a New Filter

The Filter XML can be copied and used as a starting point for a new filter. You can do this by adding a filter and pasting the copied XML into the filter definition field.

The filter can then be edited manually or in the Filters XML Editor .

Executing Filters

To run a filter, click View or Edit in the filter row to open the View Filters dialog box or the Manage Filters dialog box, respectively. Click Execute Now. The Execute Filters Details dialog box will open:

Enter the data start and end dates and a limit for the number of results to be returned. Click Ok to execute the filter or Cancel to close the dialog box without executing the filter. When you execute the filter, the results will be returned in a data table.

Datasets

Datasets provide criteria for gathering and aggregating data. The methods defined by datasets are used in analytical calculations (Calcs); when a Calc is executed, either manually or in a Scenario, it uses a population defined by a filter and applies a dataset to gather the data from the population.

Note: Datasets use procedures called Get Methods to pull data from the population.

Dataset Fields

A dataset consists of one or more dataset field objects. Each dataset field object can be configured independently to access data from different time periods. Where time series data is queried, datasets allow dynamic aggregations to be done at runtime. The type of aggregation is decided by the method chosen and attached to the dataset field. Each dataset field generates metrics based on the type of method chosen.

Dataset Fields are columns within a dataset record having the following basic characteristics.

- Enabled Flag: Allows fields to be disabled and their execution skipped. (Default: Enabled)

- Name: A user defined named for the field. Every reference to the dataset in a calc that it is bound to is done using the field name.

- Time Window Name: A reference to the time window configured in the time windows section that gets attached to a field. Each field has its own time window. See Adding Time Windows for information.

- Level: Each field is associated with a level, which is a point type code. Levels allow a dataset to go up the relationship hierarchy and pull data from parent points and make that available in the same dataset. Datasets require a level named core, which must be set to the dataset's point type code. Additional levels may be added to get data from parent points. There is no limit on the number of levels, but performance may suffer if too many levels are added. See also Adding Levels.

- Get Method: A get method operates on the data for the time window attached to the field and aggregates it into several metrics. See Get Methods for information.

Processing Basis

Processing Basis controls the level of aggregation as well as how the data set should evaluate the time period for which it will be executed. For example, if the processing basis is set to daily and the dataset is run for 3 days, it will apply the daily basis and run the dataset once per day. Additionally, all data will be aggregated up to the daily level so if the dataset is used to return interval level consumption, the dataset output will be aggregated up to the daily level.

Note: Processing mode is set in the Options section of the Dataset XML Editor, which corresponds to the options element in the dataset XML definition:

<options>
<option name="processing_basis" value="DAILY" ></option>
</options>

Valid processing basis are:

- INT_1: one second

- INT_60: one minute

- INT_300: five minutes

- INT_900: fifteen minutes

- INT_1800: thirty minutes

- INT_3600: one hour

- DAILY:

- WEEKLY:

- MONTHLY:

- YEARLY:

- CUSTOM:

- QUARTERLY:

Example

- Start Date: January 1

- End Date: January 31

- Processing Basis: Daily

The above example would result in dataset being executed 31 times in a loop with each slice being the day that the dataset is executed.

Resultant time slices: (1/1,1/1), (1/2,1/2), …, (1/31,1/31).

Next in This Section

- Manage Datasets Page

- Adding Datasets

- Viewing Dataset Information

- Managing Dataset Information

- Defining and Editing XML with the Datasets XML Editor

- Cloning a Dataset

- Copying a Dataset

- Viewing the Dataset Definition XML

- Executing Datasets

Manage Datasets Page

The Manage Datasets page provides fields and controls for filtering existing datasets and a data table that lists existing datasets. The Manage Datasets data table displays datasets based on the Type selected in the Filters pane.

Navigate to the Manage Datasets page by selecting Datasets from the Analytics section of the Manage menu. The datasets data table will load with the datasets that match the Type and that either match the selected Role or have no associated role.

Datasets Fields

The top section of the Manage Datasets page allows you to search for datasets matching criteria added to the fields:

- Dataset ID: the system generated dataset ID.

- Dataset Category: a category that a dataset is assigned to.

- Processing Basis: the processing basis for the dataset (e.g., Daily, Monthly, Weekly).

- Dataset Name: the dataset's "friendly" name.

- Dataset Lookup: a database lookup value; usually the dataset name without spaces in all caps.

- Status: the dataset status (Active/Inactive).

- Current Version: whether the dataset is the first or in production version.

Note: When datasets are cloned and the clone is made active, the clone becomes the current version.

Buttons

- Get Datasets: allows you to initiate the search for datasets matching the criteria set in the fields.

Note: Alternatively, you may press Enter when you finish adding content to a field and then click the Submit link when the system displays a message that the search criteria has changed.

- Add: opens the Create Datasets dialog box from which you may create a new dataset. See Adding Datasets for details.

Data Table

The data table contains information about the datasets. The table contains the following columns:

- ID: the system generated dataset ID.

- Sequence ID: the dataset's sequence ID, which is its version number.

- Point Type Code: the code representing the object type (e.g., EM for electric meter).

- Dataset Category: a category that a dataset is assigned to.

- Name: the dataset name.

- Processing Basis: specifies the time interval data to be analyzed (e.g., Daily, Monthly, Weekly).

- Status: the dataset status (Active/Inactive).

- Current Version: whether the dataset is the first or in production version.

- Start Time: the date/time when the dataset became active.

- End Time: the end date for the dataset. For active datasets, the end time is set to an arbitrary end date used in the system; when a new version of the dataset becomes active, the end time is updated to equal the start time of the new version. See Cloning a Dataset for information.

- Read Only: whether the dataset is read-only or may be edited.

- Links:

- View: opens the View Datasets dialog box, which displays the same data as provided in the data table row as well as buttons and links that allow you to edit or configure the dataset. See for more information.

- Edit: opens the Manage Datasets dialog box, which displays an editable view of the dataset information. See for more information

- XML: displays the Dataset Definition dialog box, which shows the configuration XML for the dataset. See Viewing the Dataset Definition XML for more information.

- XML Editor: opens the Datasets XML Editor. See Defining and Editing XML with the Datasets XML Editor for information.

Note: For enabling certain advanced options, editing the XML manually is required.

Click the expand button () at the beginning of a row to reveal the data in the data table plus the following additional data:

- Dataset Role: an optional setting for a role that is applicable to the dataset.

- Dataset Description: an optional setting that describes the dataset purpose.

- Lock Status: whether the dataset is locked to editing.

Click the minimize button () to hide the extra information.

Note: The data table may be exported by clicking the export data icon () above the table's Search field.

Adding Datasets

To add a dataset, click Add Dataset on the Manage Datasets page. The Create Datasets dialog box will open.

1. On the Manage Datasets page, click Add Dataset. The Create Datasets dialog box will open.

2. Enter the following:

1. [Optional] select the Dataset Role if the dataset should be applicable only to the Role selected in the Drawer Menu.

2. In the Name field, enter a descriptive name. For example, Hourly Validation.

3. In the Dataset Lookup field, enter a value for dataset lookup. The dataset lookup value can be any unique alphanumeric value with uppercase text and without spaces. A common practice is to use the dataset name without spaces. For example, HOURLY_VALIDATION.

4. In the Processing Basis field, enter the data processing basis (e.g., Daily, Monthly, Weekly).

5. In the Dataset Description field, enter a short description of the dataset function. For example, This dataset performs validation on hourly data.

6. From the Status drop-down list, select whether the filter is Active or Inactive.

Note: Datasets cannot be deleted through the user interface, but may be set to inactive.

7. Leave the Dataset Definition blank.

Note: The Dataset Definition XML is typically defined with the Dataset XML Editor; however, if you wish to start with the Dataset Definition XML from another dataset, you may copy it and paste it in the Dataset Definition field (see Viewing the Dataset Definition XML for details).

3. Click Save. The Create Datasets dialog box will close.

Note: Click Cancel to close the dialog box without saving the dataset; a dialog will open asking you to confirm that you want to close the Create Datasets dialog.

Viewing Dataset Information

The View link in a dataset data table row launches the View Dataset dialog box, which displays the dataset information and allows you to view the dataset definition, edit the dataset information, create a new version of the dataset (clone), copy the dataset, or execute the dataset.

Fields

- ID: the system generated dataset ID.

- Sequence ID: the dataset's sequence ID, which is its version number.

- Dataset Role: an optional setting for a role that is applicable to the dataset.

- Point Type Code: the code representing the object type (e.g., EM for electric meter).

- Category: a category that a dataset is assigned to.

- Name: the dataset name.

- Dataset Lookup: a database lookup value; usually the dataset name without spaces in all caps.

- Processing Basis: specifies the time interval data to be analyzed (e.g., Daily, Monthly, Weekly).

- Dataset Description: an optional setting that describes the dataset purpose.

- Status: the dataset status (Active/Inactive).

- Lock Status: whether the dataset is locked to editing.

- Start Time: the date/time when the dataset became active.

- End Time: the end date for the dataset. For active datasets, the end time is set to an arbitrary end date used in the system; when a new version of the dataset becomes active, the end time is updated to equal the start time of the new version. See Cloning a Dataset for information.

- Read Only: whether the dataset is read-only or may be edited.

- Dataset Definition: contains the XML link, which opens the Dataset Definition dialog box. See Viewing the Dataset Definition XML for more information.

Buttons and Links

- Edit button: opens the Manage Datasets dialog box. See Managing Dataset Information for more information.

- Clone button: creates a new version of the dataset. The clone's Sequence ID is the original dataset's Sequence ID plus one; all other dataset information is the same. Cloning allows you to keep the initial dataset active while working on modifications to the dataset parameters. See Cloning a Dataset for details.

- Copy button: creates a copy of the dataset having a unique ID. See Copying a Dataset for details.

- Set Current button: makes the dataset the active executable version.

- Cancel button: closes the dialog box.

- Execute Now link: executes the dataset. See Executing Datasets for more information.

Managing Dataset Information

The Manage Datasets dialog allows you to edit the variable information fields. To access the dialog, click the Edit link on the dataset data table row or the Edit button on the View Datasets dialog box.

Fields

- ID: the system generated dataset ID.

- Sequence ID: the dataset's sequence ID, which is its version number.

- Dataset Role: an optional setting for a role that is applicable to the dataset.

- Point Type Code: the code representing the object type (e.g., EM for electric meter).

- Category: a category that a dataset is assigned to.

- Name: the dataset name.

- Dataset Lookup: a database lookup value; usually the dataset name without spaces in all caps.

- Processing Basis: specifies the time interval data to be analyzed (e.g., Daily, Monthly, Weekly).

- Dataset Description: an optional setting that describes the dataset purpose.

- Status: the dataset status (Active/Inactive).

- Lock Status: whether the dataset is locked to editing.

- Start Time: the date/time when the dataset became active.

- End Time: the end date for the dataset. For active datasets, the end time is set to an arbitrary end date used in the system; when a new version of the dataset becomes active, the end time is updated to equal the start time of the new version. See Cloning a Dataset for information.

- Read Only: whether the dataset is read-only or may be edited.

- Dataset Definition: contains the XML link, which opens the Dataset Definition dialog box. See Viewing the Dataset Definition XML for more information.

Buttons and Links

- Save button: saves any changes to the dataset information; the button is only active when a change has been made in one or more of the editable fields.

- Clone button: creates a new version of the dataset. The clone's Sequence ID is the original dataset's Sequence ID plus one; all other dataset information is the same. Cloning allows you to keep the initial dataset active while working on modifications to the dataset parameters. See Cloning a Dataset for details.

- Copy button: creates a copy of the dataset having a unique ID. See Copying a Dataset for details.

- Set Current button: makes the dataset the active executable version.

- Cancel button: closes the dialog box.

- Execute Now link: executes the dataset. See Executing Datasets for more information.

Defining and Editing XML with the Datasets XML Editor

To configure a dataset, click the XML Editor link in the dataset's data table row. The Datasets XML Editor dialog box will open allowing you to define the dataset parameters. The Datasets XML Editor tree view allows you to define the dataset by Options, Time Windows, Levels, and Fields.

Buttons

- Save & Close: saves the dataset in its current state and closes the Datasets XML Editor.

- Validate & Save: validates the dataset syntax and displays any errors.

- Close: closes the Datasets XML Editor without saving changes.

Next in This Section

- Adding Options

- Adding Time Windows

- Adding Levels

- Adding Fields

Adding Options

When you select the Options heading in the tree-view, the editor will update with the Options Summary table, which lists existing options, and the Create New Options button.

Processing Basis

The processing_basis is automatically populated with the selections chosen when creating the dataset.

Processing Mode

Processing mode is pre-defined as FULL, which is the only option currently available in the XML Editor. It allows dataset execution to be time based such that given a time period, the dataset execution will take place for all time slices of the time period being run.

Note: A processing mode of DATADRIVEN may be used via backend methods, but that is beyond the scope of this documentation.

Adding Time Windows

Time Windows function much the same in datasets as in filters; however, in datasets, you have to note the dataset processing basis, which tells the dataset how to interpret the days you execute the data for. In the example above, the time basis is DAILY, so it will slice/interpret the execution dates on a daily basis. However, if you were to use WEEKLY, it will interpret the execution dates as the entire week and the data start date and end date will not be on the days that you selected to execute, but rather the week starting on Sunday and ending on Saturday.

Adding Levels

Note: Core is data on the most fundamental level to be analyzed and saved. If core is a meter, everything above it are considered levels. If you start an analysis for premise, then premise becomes the core and everything above it are levels.

To create a new level, click Create New Levels and complete the information the dialog box:

The Point Type Code is the system code for a point type.

Note: The codes may be found using the Administer Facts page.

Select Cache for improved performance. Data stored in a cache is data that is computed earlier and can be used later in a calculation, which otherwise would require it to be recalculated. The performance is only improved in cases where the points in a batch associated with a distinct parent points are > 10.

Adding Fields

Fields allow you to select the type of fact to pull and how to pull it. To add a field, do the following:

1. Select Fields in the tree-view and click Create New Fields.

2. Edit the name of the new field, if desired, and select it to enable editing the field criteria. Fields have the following components:

1. Enabled Flag: Allows fields to be disabled and their execution skipped. (Default: Enabled)

2. Name: The field name.

3. Time Window Name: drop-down list that allows you to reference one of the time window configured in the Adding Time Windows section. Each field has its own time window.

4. Level: drop-down list that allows you to reference a level configured in the Adding Levels section.

5. Get Method: drop-down list allows you to select a get method to operate on the data. See Get Methods for details.

Next in This Section

- Get Methods

Get Methods

Each Get Method accepts a list of inputs and generates a list of output or metrics.

Every data point in DataRaker can be uniquely identified by a 4 member tuple:

(point,fact,time,value)

In the context of a dataset, the point is provided by the filter attached to the dataset, the fact comes from the list of inputs to the Get Method, time is provided by the time window attached to the field and passed on to the Get Method, and value is the raw fact value(s) that would be aggregated into a metric. Metrics then are the outputs of a Get Method.

Your choice of Get Method depends on what type of fact you want to pull and will define how the fact will be aggregated in your dataset output.

There are two major groupings of Get Methods: Basic Gets and Multi Gets.

- Basic Gets allow you to pull data for one selected fact.

- Multi Gets pull data based from a fact category. The Multi Gets will return the “best” fact from the category. Facts are grouped together by fact_categories and, within a fact category, each fact has a fact_sequence, which sets the priority of the fact; the best fact will always be the one with the lowest fact_sequence value (i.e., 1 is the highest fact sequence). For example, with regard to an electric meter, Premise has a Fact Sequence of 5 and Account has a Fact Sequence of 6.

Note: You may find the fact sequence priority for facts using the Administer Facts page.

Non-Time Series Get Methods

Get Method

Applicable fact_type (Input)

Primary Output

dsGetRelation

Relation

Parent point object with all point table columns available as metrics.

dsGetMultiRelation

dsGetNumAttr

Numeric Attribute

Numeric attribute value (float)

dsGetMultiNumAttr

dsGetAttribute

Attribute

Text attribute value (string)

dsGetMultiAttribute

Time Series Get Methods

Get Method

Applicable fact_type (Input)

Primary Output

Pre-Aggregation

Midnight Handling

dsGetBasic

Metric, Event, List, Interval, Count (MELIC)

Count, Sum, Average, Standard Deviation, Min, Max, Nth Max, Nth Max Date, Nth First Value, Nth Last Value, Nth First Date, Nth Last Date, Nth Cumulative Value

No. If multiple values are available for the same day (as in multiple registers for ACTUAL time basis) all values get included.

Yes, hours forward and hours backward allow looking within a configurable threshold of buffer around midnight; actual register timestamp can be retrieved using ACTUAL mode.

dsGetMultiBasic

Metric, Event, List, Interval, Count (MELIC)

Same as dsGetBasic.

Pre-aggregation support to choose a single value when multiple values exist.

Same as dsGetBasic.

dsGetMedian

Metric, Event

Value

dsGetMultiMedian

Metric, Event

Value

dsGetSegment

Segment

Segment

dsGetQuantile

Metric, Event

dsGetValue

Metric, Event, List, Interval, Count (MELIC)

Value; first value without aggregation.

N/A

Same as dsGetBasic.

dsGetMultiValue

Metric, Event, List, Interval, Count (MELIC)

Value, Date Found, Fact Lookup

Pre-aggregation or pre-selection allows a single value to be chosen when multiple exist.

Same as dsGetBasic.

Next in This Section

- Using Get Methods

Using Get Methods

Within the category daily kWh, for example, consumption may be derived, aggregated, interpolated, etc. In order to account for all possible consumption types, you must use a dsGetBasic for each of the facts or use a dsGetMultiBasic on the entire fact_category and it will evaluate which of the facts has the highest fact_sequence priority.

Note: You may find the fact sequence priority for facts using the Administer Facts page.

Despite evaluating several facts in a dsGetMulti_, it will always return just one value (the best fact). If you want to return all of the facts, you will need to use dsGetValue and input a fact_category. Each dsGet_is equipped with its basic required fields; however, they all have additional panels that you can configure by clicking Add Inputs.

dsGetAttribute

Get attribute facts. Use the drop-down list to select the desired Fact Lookup. Aggregation mode is an additional field that allows you to select whether you want the first relation or the last relation found for that fact (if there happens to be more than one value within your selected time window).

dsGetNumAttr

Get numeric attribute facts. Similar to dsGetAttribute; choose the fact lookup and aggregation mode, if applicable.

dsGetRelation

Get the relation facts. Similar to dsGetAttribute; choose the fact lookup and aggregation mode, if applicable.

dsGetBasic

Use dsGetBasic to evaluate more than one fact_type.

Note: You must specify the fact_type_code before choosing a fact_lookup.

For time series gets, there are inputs that help identify which fact’s time series data would be utilized. In dsGetBasic, fact type code (FTC) and Fact Lookup (FL) help uniquely identify a single fact. The third arrow labeled as behavioral input controls different behaviors as described below.

Note: Behavior changing inputs are optional.

- Fact Filter: Fact filters allow you to exclude certain time periods from being included (or excluded) in the aggregation. Fact filter allows multiple facts to be specified using the following notation:

FactTypeCode:FactLookup,FactTypeCode,FactLookup.

When multiple fact type codes are specified the implicit exclusion or inclusion operation is OR.

dsGetBasic also has many different additional inputs.

Fact Filter and Fact Filter Mode
The Fact filter and Fact filter mode input options are used in combination to selectively remove points from being evaluated. dsGetBasic has its mandatory primary fact input that you specified first; however fact filtering gives you the ability to exclude certain values of the mandatory fact.
For example, if your primary fact is kWh consumption, and you choose fact filter mode to exclude and the fact filter to be register validation gaps, then you will be excluding all kWh consumption values for days on which there was a gap.

Hours to search backwards/forwards:

For the most part, the system implies a midnight time-stamp for most data and the dataset will search for midnight timestamps only. However, for facts such as register reads, their reads may not necessarily come in at exactly midnight, so with these functions you can expand the search to “hours before/after midnight”.

Midnight timestamp mode:
Midnight timestamp mode allows reads that are within a specified threshold (typically +/- 1 hour of midnight) and treat it as midnight. For example, a read that comes in at 11:53 PM is considered to be a midnight read as is a read that came in at 12:53 AM. In both cases, no interpolation is done and the register value is accepted without any adjustment. However, if a read comes in at 10:55 PM or at 1:05 AM, it will not be considered a midnight read since it would be outside of the +/- 1 hour threshold; reads outside of the threshold will result in data interpolation to derive daily consumption.

Note: The threshold +/- n hour value is configurable.

Maximum ‘nth metric to compute’:

This feature is to accommodate for facts that may not necessarily occur every day or on a regular basis. So in these situations, you can create a time window that is a wide enough net to grab a large number of occurrences/events of these facts. However, if your goal is to say, only aggregate the first 5 occurrences and ignore the rest, then you can specify that in this section.

dsGetValue

dsGetValue is used to get a value with no aggregation. Multiple facts or a fact category can be specified to get a value for multiple facts in one call. You can use dsGetValue to specify a fact_category or a specific fact_lookup. However, regardless of the time window, this will only return the first value of each fact (no aggregations).

Cloning a Dataset

The clone function, which is found in the View Datasets and Manage Datasets dialogs, creates a new version of the dataset; the Sequence ID will be incremented by 1. The clone is unlocked, which allows it to be edited while the original is locked and ready to be executed (see Executing Datasets); once the new version is ready to be put in production, the previous version's End Time is set to the new version's Start Time.

The cloning function has the following workflow:

1. In the View Datasets or Manage Datasets dialog, click Clone.

2. Edit the clone (see Defining and Editing XML with the Datasets XML Editor).

3. When the clone is ready to be deployed, open its Manage Datasets dialog and click Set Current, which will set the original dataset's Locked Status to Unlocked (see Managing Dataset Information) and the clone's Locked Status to Locked.

Copying a Dataset

The copy function, which is found in the View Datasets and Manage Datasets dialogs, creates a new dataset with a unique ID. Copying allows you to start a new dataset from an existing one. The copy has a Sequence ID of 1 since it is the first version of a new dataset.

Viewing the Dataset Definition XML

From the Dataset Definition dialog box, you can view or edit the XML that is produced by the Dataset XML Editor. This allows you to quickly modify a parameter without needing to drill down in the editor.

Executing Datasets

When done with the dataset, click View or Edit and then click Execute Now. The Execute Datasets Details dialog box will open:

Similar to filters, you will be able to select the execute Start Date and the execute End Date, which are used as Data Start Date and Data End Date in the time windows. You must select a filter to execute your dataset upon; the filter will decide what points to grab data for. Click Ok to execute the dataset or Cancel to close the dialog box without executing the dataset.

Calcs

Calcs define logical and mathematical operations to perform on data defined by a dataset. Calcs take the form of a directed acyclic graph (DAG) that consist of nodes, which provide instructions for individual processes within the calc, and edges, which provides the order of execution for the nodes.

Next in This Section

- Manage Calcs Page

- Adding Calcs

- Viewing Calc Information

- Managing Calc Information

- Defining and Editing XML with the Calcs XML Editor

- Cloning a Calc

- Copying a Calc

- Viewing the Calc Definition XML

- Viewing a Calc Graph

- Executing Calcs

Manage Calcs Page

Navigate to the Manage Calcs page by selecting Calcs from the Analytics section of the Manage menu. The Manage Calcs page allows you to add a new calc, view existing calcs, and edit calcs with the Calc XML Editor.

Calcs Fields

The top section of the Manage Calcs page allows you to search for calcs matching criteria in the fields.

- Calc Category: allows you to find calcs that belong to the selected Calc Category.

- Processing Basis: allows you to find calcs when you know the calc's processing basis (e.g., WEEKLY).

- Calc Name: allows you to find a calc based on its name.

- Calc Lookup: allows you to find a calc based on its lookup value.

- Status: allows you to search for active or inactive calcs.

- Current Version: whether the calc is the first or in production version.

Note: When calcs are cloned and the clone is made active, the clone becomes the current version.

Buttons

- Get Calcs: allows you to initiate the search for calcs matching the criteria set in the fields.

Note: Alternately, you may press Enter when you finish adding content to a field and then click the Submit link when the system displays a message that the search parameters have changed.

- Add: opens the Create Calcs dialog box from which you may build a new calc. See Adding Calcs for details.

Data Table

The data table contains information about the calcs.

- ID: the system generated calc ID

- Sequence ID: the calc's sequence ID.

- Calc Category: a category that a calc is assigned to.

- Name: the calc name.

- Processing Basis: the calc's processing basis.

- Calc Graph: provides a link to open a graphical view of the calc. See Viewing a Calc Graph.

- Dataset Name: the name of the dataset used in the calc.

- Status: the calc's status (Active/Inactive).

- Current Version: whether the calc is the first or in production version.

- Start Time: the date/time when the calc became active.

- End Time: the end date for the calc. For active calcs, the end time is set to an arbitrary end date used in the system; when a new version of the calc becomes active, the end time is updated to equal the start time of the new version. See Cloning a Calc for information.

- Read Only: whether the calc is read-only or may be edited.

- Links:

- View: opens the View Calcs dialog box, which displays the same data as provided in the data table row as well as buttons and links that allow you to work with the calc. See Viewing Calc Information for more information.

- Edit: opens the Manage Calcs dialog box, which provides the same fields as the View Calcs dialog box, but allows you to edit Category, Name, Time Basis, Calc Description, Dataset Name, Status, and Lock Status. See Managing Calc Information for more information.

- XML: displays the Calcs Definition dialog box, which shows the configuration XML for the calc. See Viewing the Calc Definition XML for more information.

- XML Editor: displays the Calcs XML Editor. SeeDefining and Editing XML with the Calcs XML Editor for more information.

Click the expand button () at the beginning of a row to reveal the data in the data table plus the following additional data:

- Calc Role: an optional setting for a role that is applicable to the calc.

- Calc Lookup: the alphanumeric lookup value, which is typically the name in all caps without spaces.

- Calc Description: an optional setting that describes the calc purpose.

- Lock Status: whether the calc is locked to editing.

Click the minimize button () to hide the extra information.

Note: The data table may be exported by clicking the export data icon () above the table's Search field.

Adding Calcs

To add a calc, click Add Calc on the Manage Calcs page. The Create Calcs dialog box will open.

Fields

A calc is primarily configured with the following attributes:

- Calc Role: A calc can be created with the role selected in the Drawer Menu or with an n/a (empty role).

- Category: A category allows calcs to be grouped together for quick viewing from the UI. There is no special significance of the category field from a processing standpoint.

- Name: A user friendly name describing the calc.

- Calc Lookup: A standardized formatted name that is used by calc engines.

- Processing Basis: When running a calc for an extended time period this property controls how the bigger time frame gets sliced into smaller time slices. The calc processing basis should match with the dataset processing basis that it is bound to. Valid time basis examples are: Hourly, Daily, Weekly, Monthly.

- Calc Description: Description of the calc with comments.

- Dataset Name: The drop-down list shows all datasets whose processing basis matches that of the calc. (Note that every calc is bound to a dataset and that a dataset provides all input data to a calc.)

- Status: Enable or disable a calc from showing up in the calc list. In addition, an inactive calc (Status = Inactive) cannot be launched from the system's backend.

- Calc Definition: The XML that is generated by building a calc. An existing calc definition may be copied and pasted to this field as a starting point for a new calc.

Buttons

- Save: saves the calc.

- Cancel: closes the dialog box without saving the calc.

Create a Calc

To create a calc:

1. On the Manage Calcs page, click Add Calc.

2. In the Create Calcs dialog box, do the following:

1. From the Calc Role drop-down list, select an appropriate role.

2. From the Category drop-down list, select an appropriate category.

3. In the Name field, enter a descriptive name.

4. In the Calc Lookup field, enter a value for calc lookup. The calc lookup value can be any unique alphanumeric value without spaces. A common practice is to use the calc name without spaces.

5. From the Processing Basis drop-down list, select a processing basis for the calc. For example, Weekly.

6. In the Calc Description field, enter a short description of the calc function.

7. From the Status drop-down list, select whether the calc is Active or Inactive.

3. Click Save. The Create Calcs dialog box will close.

When you click Save , a row is added in drm.,calc table. The calc definition, however, is stored in a different table (calc_seq); the sequence table allows multiple versions of the calc to be stored, with one version being the locked calc that will get executed when a calc is launched.

Viewing Calc Information

The View link in a calc data table row launches the View Calc dialog box, which displays the calc information and allows you to view the calc definition, edit the calc information, create a new version of the calc (clone), copy the calc, or execute the calc.

Fields

- ID: the system generated calc ID.

- Sequence ID: the calc's sequence ID, which is its version number.

- Calc Role: an optional setting for a role that is applicable to the calc.

- Point Type Code: the code representing the object type (e.g., EM for electric meter).

- Category: a category that a calc is assigned to. [Editable]

- Name: the calc name. [Editable]

- Calc Lookup: a database lookup value; usually the calc name without spaces in all caps. [Editable]

- Processing Basis: specifies the time interval data to be analyzed (e.g., Daily, Monthly, Weekly). [Editable]

- Calc Description: an optional setting that describes the calc purpose. [Editable]

- Status: the calc status (Active/Inactive). [Editable]

- Lock Status: whether the calc is locked to editing.

- Start Time: the date/time when the calc became active.

- End Time: the end date for the calc. For active calcs, the end time is set to an arbitrary end date used in the system; when a new version of the calc becomes active, the end time is updated to equal the start time of the new version. See Cloning a Calc for information.

- Read Only: whether the calc is read-only or may be edited.

- Calc Definition: contains the XML link, which opens the Dataset Definition dialog box. See Viewing the Calc Definition XML for more information.

Buttons and Links

- Edit button: opens the Manage Calcs dialog box. See for more information.

- Clone button: creates a new version of the calc. The clone's Sequence ID is the original calc's Sequence ID plus one; all other calc information is the same. Cloning allows you to keep the initial calc active while working on modifications to the calc parameters. See Cloning a Calc for details.

- Copy button: creates a copy of the calc having a unique ID. See Copying a Calc for details.

- Set Current button: makes the calc the active executable version.

- Cancel button: closes the dialog box.

- Execute Now link: executes the calc. See Executing Calcs for more information.

Managing Calc Information

The Manage Calcs dialog box opens when you click the Edit link on the calc data table row or the Edit button on the View Calcs dialog box.

Fields

- ID: the system generated calc ID.

- Sequence ID: the calc's sequence ID, which is its version number.

- Calc Role: an optional setting for a role that is applicable to the calc.

- Point Type Code: the code representing the object type (e.g., EM for electric meter).

- Category: a category that a calc is assigned to. [Editable]

- Name: the calc name. [Editable]

- Calc Lookup: a database lookup value; usually the calc name without spaces in all caps. [Editable]

- Processing Basis: specifies the time interval data to be analyzed (e.g., Daily, Monthly, Weekly). [Editable]

- Calc Description: an optional setting that describes the calc purpose. [Editable]

- Status: the calc status (Active/Inactive). [Editable]

- Lock Status: whether the calc is locked to editing.

- Start Time: the date/time when the calc became active.

- End Time: the end date for the calc. For active calcs, the end time is set to an arbitrary end date used in the system; when a new version of the calc becomes active, the end time is updated to equal the start time of the new version. See Cloning a Calc for information.

- Read Only: whether the calc is read-only or may be edited.

- Calc Definition: contains the XML link, which opens the Dataset Definition dialog box. See Viewing the Calc Definition XML for more information.

Buttons and Links

- Save button: allows you to save any changes that have been made; the button is desensitized until a field has been modified.

- Clone button: creates a new version of the calc. The clone's Sequence ID is the original calc's Sequence ID plus one; all other calc information is the same. Cloning allows you to keep the initial calc active while working on modifications to the calc parameters. See Cloning a Calc for details.

- Copy button: creates a copy of the calc having a unique ID. See Copying a Calc for details.

- Set Current button: makes the calc the active executable version.

- Cancel button: closes the dialog box.

- Execute Now link: executes the calc. See Executing Calcs for more information.

Defining and Editing XML with the Calcs XML Editor

Once the calc has been added, a row appears in the data table on the Manage Calcs page. There are additional action link buttons that allow a user to view the Calc XML, edit the raw XML, or view the calc. A calc that is added from the Create Calcs dialog box does not have a calc definition.

The Calcs XML Editor allows you to add nodes and connect them with edges. There are two types of edges: an ordinary edge and a decision edge, which connects a decision node to another node (for example, a save node). A decision edge has a special property (isYesEdge) that allows you to control downstream execution based on the output of the connected decision nodes.

Note: The calc configuration gets saved in drm.calc_seq table as XML.

Buttons

- Save&Close: saves the calc in its current state and closes the editor. If the calc is structurally invalid, it will show a warning indicating that the Calc was saved, but it will fail execution.

- Validate&Save: validates the calc to make sure it is syntactically valid and shows validation errors, if any exist.

- Preview&Save: allows the user to view and save a calc image based on the nodes and its connectivity.

- Close: closes the Calcs XML Editor without saving changes.

Next in This Section

- Graph Components

- Variable References

Graph Components

The Calcs XML Editor uses graph terminology to describe the structure of the calc. The configuration is defined in the Graph Type, which contains Graph Header, Time Windows, Nodes, and Edges.

Next in This Section

- Graph Header

- Time Windows

- Nodes

- Edges

Graph Header

Graph Header is the header of the Calc Graph where name, description, and processing basis are defined.

Graph Name

The graph name is a user friendly name for the Calc Graph

Graph Desc

The graph description allows users to add valuable documentation information regarding the calc. It can be used as a placeholder to annotate information related to the calc logic, etc.

Graph Processing Basis

Graph processing basis is the same as the time basis/processing basis field that was earlier defined in the calc table. This information is redundant. This property comes into play when a task is run for an extended period of time. For example, a calc defined with a processing basis of daily is run for a week or a month, it will actually be run once for every day of the week or month. Also, the processing basis of a calc must match with the processing basis of the dataset that it is bound to. This check is ensured when the calc is defined and the dataset name selected from the drop-down.

Time Windows

The Time Windows section allows time windows to be added that are referenced by nodes in the calc. The need for a time window in a calc is limited. A time window is required only for Save Nodes.

Note: While the schema (XSD) dictates that all nodes of a calc reference a time window, the Calcs XML Editor hides this requirement by making the time window available only when configuring a save node. Behind the scenes, default time windows get added to other nodes.

One may ask what is the need to have time window even for a save node? Could we not simply use the process date to save the data? While saving data as the process date is the most common use case, there are situations where data needs to be saved as a different date that is offset from the process date. A time window helps achieve that. Using a time window attached to a save node data can be saved to any date. For example, a daily process that would run and derive Average Daily Usage (ADU) over the month and save it as the ADU for the month (Month start Date). In this case you would have an offset based time window with start date = offset of start, unit = month.

Nodes

Nodes define what a calc can do. The Nodes section allows nodes to be added. There are seven different types of nodes that may be used in a calc:

- Start

- Decision

- Union

- Logic

- Save

- Delete

- Move

Start Node

Where the calc starts execution. Any preceding nodes or any nodes that are not connected to a tree containing the start node will not be executed. The start node represents the head of the graph. It serves no purpose other than being the start point of the calc.

The start node only requires the Name to be entered; the description (Desc) is optional.

Decision Node

Contains custom code. If expression evaluates to TRUE, “yes” edges will be traversed. If not, “no” edges will be traversed.

In the decision node, custom code can be written that will dictate branching of the calc logic.. A decision node should always return a Boolean value (True/False). Based on the output of the decision node children node will be skipped or processed.

A decision node has two required fields Name and an Expression. The expression consists of free form code where variables from the dataset or other upstream nodes can be accessed.

Union Node

Children nodes will execute if at least ONE parent node executes.

A union node requires no configuration. When the calc engine encounters a union node, it will check its parent and execute children if at least one of the parent were executed (i.e., not skipped).

Logic Node

Contains custom code. What the expression evaluates to has no effect over what nodes are traversed (in contrast to decision node)

A logic node in calc is to setup initial variables or temporary variables that get accessed later by downstream children nodes. There is no restriction on the number of logic nodes but as a good practice, add a logic node right after the start node and initialize all temporary data upfront so that it is accessible by all other downstream nodes.

The configuration of a logic node is similar to that of the decision node. However, unlike decision node, logic node has no direct output, but rather creates data that can be accessed elsewhere in the program.

A logic node requires an expression field to be populated. Within the expression field, you may access dataset variables and other calc variables.

Save Node

A save node allows data to be saved for any fact type. When saving data we are concerned about four things:

1. Point ID: This is the calc point id for which execution takes place. Remember that every calc requires a dataset. The dataset returns iterators of points and the calc loops over this iterators. The point ID is thus the current point that the iterators returns.

2. Fact ID: In order to save data for the correct Fact ID, the Fact Attr attribute group needs to be configured. The save node is fact type code sensitive and provides options that match the fact type code being saved. For example, when saving a relation Parent Point Type Code needs to be specified while it is not a requirement for other fact types.

- In the save node, we do not specify the fact ID directly rather provide fact metadata via fact attributes that are then parsed by the calc engine and get resolved to a Fact ID

- The following is the list of Fact attributes required for non relation facts:

- Fact Lookup: A standardized lookup to uniquely identify a fact.

- Fact Name: A unique user friendly name to identify a fact.

- Fact Category: A logical group of facts.

- Fact Sequence: Priority of facts within a category.

- Time Basis: This in most cases should match the processing basis of the calc.

- Role Type Code: Role that was used to define the calc. The role should be inherited from the role used when adding a calc.

3. Value: When saving a relation fact Parent Point Name and Lookup are needed, while others require a value source. Value source could point to any dataset variable or a temporary variable that was set within the calc. There is no requirement that the value be set in the parent node of the save node.

4. Time Int: Time can be specified using a time window or by setting the Time source property of the save node. Refer t o the earlier section about Time Windows

Note: Time can be specified using time window source or time window. When a time source is provided it will always be chosen over the time returned by the time window.

The Delete If Skipped check-box provides a mechanism to avoid conflicts from "old" data by deleting previously saved data.

Delete Node

A delete node allows data to be deleted. Just like the save node a delete node requires configuring the fact attr attribute group, specifying a time int (using time source or time window). Value source is ignored.

Move Node

A move node allows data from a source fact to be moved to a target fact. One use case where move node is used is to move data that has failed validation to a different fact with a category of “Failed”.

Since moving involves two facts, a source fact and a target fact the configuration of move node requires specifying fact attrs separately for the source and target fact attrs.

Creating a Node

To create a node, do the following:

1. Click Create New Dr Nodes.

2. Select the branch with the new node in the tree-view.

3. Select the node type from the Type drop-down list.

Edges

Edges connect two nodes together to create a sequence for node execution.

There are two edge types:

1. Edge: An edge connects two nodes.

2. Decision Edge: an edge that connects a decision node to another node. If Yes is selected, the edge will be traversed when the decision is true.

Variable References

As has been specified before, free form python code may be written within a decision or logic node. This python code can access dataset variables as well as calc variables. In order to assist with calc development and provide a ready reference of all dataset variables and calc variables, two links are available at the top of the Calcs XML Editor: Dataset and Calc.

- Click the Dataset link to open a dialog box with fully qualified dataset variables and their description along with datatype information.

- Click the Calc link to open a dialog box with fully qualified calc variables with a usage summary for the variables within different calc nodes.

Next in This Section

- Accessing Variables

Accessing Variables

Dataset Variables

The notation followed in accessing a dataset variable is:

context.ds.<field_name>.<metric_name>

For example, if the dataset bound to the calc has a field called Account with a metric called Name, this variable would be accessed in the python expression as:

context.ds.Account.Name

Calc Variables

Getting Calc Variable: The notation followed in accessing a calc variable is:

context.nodes.<node_name>.<variable_name>

For example, if a calc has a node called vars with a temporary variable called average, this variable would be accessed as:

context.nodes.vars.average

Setting Calc Variables:

When setting a calc variable the notation to be followed is:

context.self.<variable_name> = xyz

Where self represents that node that you are within.

Cloning a Calc

The clone function, which is found in the View Calcs and Manage Calcs dialogs, creates a new version of the calc; the Sequence ID will be incremented by 1. The clone is unlocked, which allows it to be edited while the original is locked and ready to be executed (see Executing Calcs); once the new version is ready to be put in production, the previous version's End Time is set to the new version's Start Time.

The cloning function has the following workflow:

1. In the View Calcs or Manage Calcs dialog, click Clone.

2. Edit the clone (see Defining and Editing XML with the Calcs XML Editor for details).

3. When the clone is ready to be deployed, open its Manage Calcs dialog and click Set Current, which will set the original calc's Locked Status to Unlocked (see Managing Calc Information) and the clone's Locked Status to Locked.

Copying a Calc

The copy function, which is found in the View Calcs and Manage Calcs dialogs, creates a new calc with a unique ID. Copying allows you to start a new calc from an existing one. The copy has a Sequence ID of 1 since it is the first version of a new calc.

Viewing the Calc Definition XML

From the Calc Definition dialog box, you can view or edit the XML that specifies the calc parameters. The XML is typically defined with the Calc XML Editor, but the Calc Definition allows you to quickly modify XML parameters or define a calc using XML from a similar calc.

Viewing a Calc Graph

Calc graphs provide a flowchart-like view of the calc. Each edge (node connection) is displayed as the node symbols connected by an arrow representing a path in the calc.

Example

Create a calc that creates two lists based on whether consumption is below a value or not.

Logic Node

The logic node first creates a logic bucket named 'High' and then tests if the value of consumption data (based on the dataset looking for daily kWh) is below 7 (kWh).

context.self.bucket = 'HIGH'
if context.ds.basic.firstValue < 7: context.self.bucket = 'LOW'

- If the data value is below 7, the bucket will be set to 'LOW'.

- If not, the bucket will remain 'HIGH'.

Decision Nodes

The logic-dec_high edge checks to see if the dec_high node criteria is met.

if context.nodes.logic.bucket == 'HIGH': return True

The logic-dec_low edge checks to see if the dec_low node criteria is met.

if context.nodes.logic.bucket == 'LOW': return True

Save Nodes

The save nodes collect values meeting the criteria for the decision nodes. In this case, lists are created with values meeting the criteria:

- The dec_high-save_high decision edge saves all values in the HIGH bucket.

- The dec_low-save_low decision edge saves all values in the LOW bucket.

Executing Calcs

A calc can be executed from the View Calcs and Manage Calcs dialog boxes. Click Execute Now to open the Execute Calcs Details dialog box.

Enter the filter that the calc should operate on and the dates. Click Ok to execute the calc or Cancel to close the dialog box without executing.

Scenarios

Scenarios are a collection of calcs to be executed sequentially on a population found using a filter.

Note: Scenarios are run using a scenario task. See Manage Tasks for information.

Next in This Section

- Manage Scenarios Page

- Adding Scenarios

- Viewing Scenario Information

- Managing Scenario Information

Manage Scenarios Page

The Manage Scenarios page allows you to create, view, and edit scenarios.

Navigate to the Manage Scenarios page by selecting Scenarios from the Analytics section of the Manage menu. The scenarios data table will load with the scenarios that match the selected Type.

Scenario Fields

The top section of the Manage Scenarios page allows you to search for scenarios matching criteria added to the fields:

- Scenario ID: the system generated scenario ID.

- Scenario Category: a category that a scenario is assigned to.

- Scenario Name: the scenario name.

- Scenario Lookup: the alphanumeric lookup code for the scenario.

- Status: the scenario status (Active/Inactive).

- Locked Flag: whether the scenario is locked or unlocked.

Buttons

- Get Scenarios: allows you to initiate the search for scenarios matching the criteria set in the fields.

Note: Alternatively, you may press Enter when you finish adding content to a field and then click the Submit link when the system displays a message that the search criteria has changed.

- Add: opens the Create Scenarios dialog box from which you may create a new scenario. See Adding Scenarios for details.

Data Table

The data table contains information about the scenarios. The table contains the following columns:

- ID: the system generated scenario ID.

- Point Type Code: the code representing the object type (e.g., EM for electric meter).

- Scenario Category: a category that a scenario is assigned to.

- Name: the scenario name.

- Scenario Lookup: the alphanumeric lookup code for the scenario.

- Filter Lookup: the alphanumeric lookup code for the filter used in the scenario.

- Status: the scenario status (Active/Inactive).

- Create Time: the date/time when the scenario was created.

- Update Time: the date/time when the scenario was updated.

- Links:

- View: opens the View Scenarios dialog box, which displays the same data as provided in the data table row as well as buttons and links that allow you to edit the scenario. See Viewing Scenario Information for more information.

- Edit: opens the Manage Scenario dialog box, which displays an editable view of the scenario information. See Managing Scenario Information for more information.

Click the expand button () at the beginning of a row to reveal the data in the data table plus the following additional data:

- Scenario Role: an optional setting for a role that is applicable to the scenario.

- Calc List: lists the calcs associated with the scenario.

- Scenario Description: an optional setting that describes the scenario purpose.

- Lock Flag: whether the scenario is locked to editing.

Click the minimize button () to hide the extra information.

Note: The data table may be exported by clicking the export data icon () above the table's Search field.

Adding Scenarios

To add a scenario, click Add Scenario on the Manage Scenarios page. The Create Scenarios dialog box will open.

Fields

A scenario is defined by:

- Scenario Role: a scenario may be created with the role selected in the Drawer Menu or an empty role (i.e., n/a ).

- Category: A category allows scenarios to be grouped together for quick viewing from the UI. There is no special significance of the category field from a processing standpoint.

- Name: A user friendly name describing the scenario.

- Scenario Lookup: the standardized database lookup name without spaces or lowercase letters.

- Scenario Description: Description of the scenario with comments.

- Filter Lookup: The drop-down list shows all filters created for the role and point type.

- Status: enable or disable a scenario. Enabled scenarios will appear in the data table; disabled scenarios will be hidden.

- Locked Flag: the scenario lock status. Unlocked scenarios cannot be run.

Buttons

- Save: saves the scenario.

- Cancel: closes the dialog box without saving the scenario.

Create a Scenario

To create a calc:

1. On the Manage Scenarios page, click Add Scenario.

2. In the Create Scenarios dialog box, do the following:

1. Optionally, from the Scenario Role drop-down list, select a Role or leave the field blank.

2. From the Category drop-down list, select an appropriate category.

3. In the Name field, enter a descriptive name.

4. In the Scenario Lookup field, enter a value for the scenario lookup. The scenario lookup value can be any unique alphanumeric value without spaces in all caps.

5. In the Scenario Description field, enter a short description of the scenario features.

6. In the Filter Lookup drop-down list, select a filter to run the scenario against.

7. From the Status drop-down list, select whether the scenario is Active or Inactive.

8. From the Locked Flag drop-down list, select whether the scenario is Locked or Unlocked.

3. Click Save. The Create Scenarios dialog box will close.

Viewing Scenario Information

The View link in a dataset data table row launches the View Dataset dialog box, which displays the dataset information and allows you to view the dataset definition, edit the dataset information, clone the dataset, or execute the dataset.

Fields

- ID: the system generated scenario ID.

- Point Type Code: the code representing the object type (e.g., EM for electric meter).

- Category: a category that a scenario is assigned to.

- Name: the scenario name. [Editable]

- Scenario Lookup: the alphanumeric lookup code for the scenario.

- Calc List: lists the calcs associated with the scenario. [Editable]

- Scenario Description: an optional setting that describes the scenario purpose. [Editable]

- Filter Lookup: the alphanumeric lookup code for the filter used in the scenario.

- Status: the scenario status (Active/Inactive). [Editable]

- Lock Flag: whether the scenario is locked to editing. [Editable]

- Create Time: the date/time when the scenario was created.

- Update Time: the date/time when the scenario was updated.

Note: The fields marked as editable may be modified in the Manage Scenarios dialog.

Buttons

- Edit button: opens the Manage Scenarios dialog.

- Cancel button: closes the dialog box.

Managing Scenario Information

The Manage Scenarios dialog box opens when you click the Edit link on the scenario data table row or the Edit button on the View Scenarios dialog box. In addition to editing the editable fields (i.e., Name, Scenario Description, Status, and Locked Flag), the Manage Scenarios dialog box provides the Add Calc field, which allows you to add the calcs that will be run in the scenario.

Fields

- ID: the system generated scenario ID.

- Point Type Code: the code representing the object type (e.g., EM for electric meter).

- Category: a category that a scenario is assigned to.

- Name: the scenario name. [Editable]

- Scenario Lookup: the alphanumeric lookup code for the scenario.

- Calc List: lists the calcs associated with the scenario. [Editable]

- Scenario Description: an optional setting that describes the scenario purpose. [Editable]

- Filter Lookup: the alphanumeric lookup code for the filter used in the scenario.

- Status: the scenario status (Active/Inactive). [Editable]

- Lock Flag: whether the scenario is locked to editing. [Editable]

- Create Time: the date/time when the scenario was created.

- Update Time: the date/time when the scenario was updated.

Note: The fields marked as editable may be modified in the Manage Scenarios dialog.

Buttons and Links

- Assign Additional Calc: activates the Add Calc drop-down list. See Adding Calcs to a Scenario for information.

- Save button: saves changes.

- Cancel button: closes the dialog box without saving changes.

Next in This Section

- Adding Calcs to a Scenario

Adding Calcs to a Scenario

When you click the Assign Additional Calc link in the Manage Scenarios dialog box, the Add Calc field becomes an active drop-down list containing calcs appropriate to the scenario role and point type.

To add a calc:

1. Click the Assign Additional Calc link.

2. Select a calc from the Add Calc field.

3. Click the Save link to the right of the Add Calc field.

The calc will be listed in the Calc List field with a Remove link, which allows you to remove the calc from the scenario.

Administer

The Administer menu provides access to the Metadata, Security, and System features.

Next in This Section

- Administering Metadata

- Administer Security

- Administer System

Administering Metadata

The Administer Metadata category provides access to administer the following related items:

Next in This Section

- Administer Points

- Administer Facts

- Configuration

Administer Points

The Administer Points page allows you to view and, in some cases, edit the point metadata for an environment. The page contains the following components.

Filters Pane

The filters pane allows you to select the point type code that you are interested in. For example, to see electric meters, select Electric Meters from the drop-down list.

Fields

The top section of the Administer Points page provides filters that allow you to narrow the points that are returned by selecting the point type in the Filters pane. With the exception of Point ID, the text fields allow you to search for a string. If the whole string is unknown, a part of it can be entered with a percentage (%) symbol with as prefix, suffix, or both.

- Point ID: the point’s system-assigned, unique identifier.

- Point Name: the name that was given to the point when it was added.

- Point Lookup: the point’s secondary description, or nickname, that was assigned when the point was added.

- Point Tag: allows you to search by the tag category that was associated to the point when it was added.

- Status: allows you to search for points that are a specific status.

Buttons

- Get Points allows you to search for points matching the criteria set in the top section’s fields.

Data Table

The data table displays the filtered points in a tabular format with one row representing one point. Click the + button to expand a row to view the point data as text.

Click The selected points can also be copied to clipboard or saved as csv or pdf using the buttons on the up-per right corner. A search bar can be used to search for any text in the displayed point table.

- ID: the point’s system-assigned, unique identifier.

- Name: the name that was given to the point when it was added.

- Point Lookup: the point’s secondary description, or nickname, that was assigned when the point was added.

- Point Tag: the tag category that was associated to the point when it was added.

- Create Time: the date and time when the point was created.

- Update Time: the date and time when the point was updated.

- Links:

- View: opens the View Points dialog box, which displays the same information as provided in the data table row. In addition, it includes the point’s description. This dialog box also provides links that allow you to edit the point. (See Viewing Point Information for more information.)

- Edit: opens the Mange Points dialog box, which displays the same information as provided in the data table row. In addition, it allows you to edit the status. (See Managing Point Informationfor more information.)

Next in This Section

- Viewing Point Information

- Managing Point Information

Viewing Point Information

The View Points dialog box is accessed from the View link, which is found in the Administer Points data table.

Fields and Links

- Point ID: the point’s system-assigned, unique identifier.

- Point Name: the name that was given to the point when it was added.

- Point Lookup: the point’s secondary description, or nickname, that was assigned when the point was added.

- Point Tag: the tag category that was associated to the point when it was added.

- Point Description:

- Status: the point’s status (Active/Inactive).

- Create Time: the date and time when the point was created.

- Update Time: the date and time when the point was last updated.

Buttons

- Edit button: opens the Manage Points dialog box. (See Managing Point Information for more information.)

- Cancel button: closes the dialog box.

Managing Point Information

The Manage Points dialog box opens when you click the Edit link on the point data table row or the Edit button on the View Points dialog box.

Fields and Links

- Point ID: the point’s system-assigned, unique identifier.

- Point Name: the name that was given to the point when it was added.

- Point Lookup: the point’s secondary description, or nickname, that was assigned when the point was added.

- Point Tag: the tag category that was associated to the point when it was added.

- Point Description: a short description that was given to the point when it was added.

- Status: drop-down list that allows you to set the point’s status (Active/Inactive).

- Create Time: the date and time when the point was created.

- Update Time: the date and time when the point was updated.

Buttons

- Save button: allows you to save any changes that have been made; the button is disabled until a field has been modified.

- Cancel button: closes the dialog box.

Administer Facts

The Administer Facts page allows you to view and, in some cases, edit the fact metadata for an environment. There are a number of filters available on the left and central panels to help get to the facts of interest.

Filters Pane

The filters pane allows you to select the point type code that you are interested in. For example, to see electric meters, select Electric Meters from the drop-down list.

Fields

The top section of the Administer Facts page provides filters that can be used to search facts and a button to apply the filters and display the facts. While the drop downs display the possible values to select from, the text boxes have the capability to search based on a given string. If the whole string is unknown, a part of it can be entered with a percentage (%) symbol with as prefix, suffix or both.

- Fact ID: the fact’s system-assigned, unique identifier.

- Fact Type Code: allows you to search for facts that are a specific fact type code.

- Time Basis: allows you to search for facts that are based on a specific period of time. (See Time Basis for more information.)

- Fact Category: allows you to search for facts that are a specific fact category.

- Fact Name: the name that was given to the fact when it was added.

- Fact Lookup: the secondary description, or nickname, that was given to the fact when it was added. (This field is case sensitive.)

- Fact Source: the data or calculation that produced the fact.

- Role Type Code: allows you to search for facts associated to a specific role type.

- Aggregation Method: the mathematical function used to calculate the fact.

- Parent PTC: the point type code associated to a parent object.

- Status: allows you to search for facts that are a specific status.

Buttons

- Get Facts: allows you to search for facts matching the criteria set in the top sections fields.

Data Table

The data table displays the filtered facts in a tabular format with one row representing one fact. A drop-down on the upper left corner can be used to change the number of facts displayed per page (the default is 50 entries).

Every row has buttons to view, edit, or navigate to the fact audit screen for the fact in that row. The details can also be viewed by clicking the “+” in front of every row. The selected facts can also be copied to clipboard or saved as csv or pdf using the buttons on the upper right corner. A search bar can be used to search for any text in the displayed fact table.

- ID: the fact’s system-assigned, unique identifier.

- Fact Type Code: allows you to search for facts that are a specific fact type code.

- Time Basis: allows you to search for facts that are based on a specific period of time. (See Time Basis for more information.)

- Fact Category: allows you to search for facts that are a specific fact category.

- Fact Name: the name that was given to the fact when it was added.

- Fact Lookup: the secondary description, or nickname, that was given to the fact when it was added. (This field is case sensitive.)

- Fact Source: the data or calculation that produced the fact.

- Role Type Code: allows you to search for facts associated to a specific role type.

- Agg Method (Aggregation Method): the mathematical function used to calculate the fact.

- Parent PTC: the point type code associated to a parent object.

- Links:

- View: opens the Viewing Fact Informationdialog box, which contains the same information as the data table. In addition, it displays the associated Role and Fact Sequence.

- Edit: opens the Managing Fact Information dialog box, which contains the same information as the data table. In addition, it provides opportunities to update the fact name, aggregation method, and fact status.

- Fact Audit: navigates to the Fact Audit page and displays the current context to provide further information.

Next in This Section

- Viewing Fact Information

- Managing Fact Information

Viewing Fact Information

The Administer Facts dialog box is accessed from the View link, which is found in the Administer Facts data table.

Fields and Links

- Fact ID: the fact’s system-assigned, unique identifier.

- Fact Type Code: the fact type code that was associated to the fact when it was added.

- Time Basis: the period of time that is used to aggregate data in order to calculate the fact. (See Time Basis for more information.)

- Fact Category: the category that was associated to the fact when it was added.

- Fact Name: the name that was given to the fact when it was added.

- Fact Lookup: the secondary description, or nickname, that was given to the fact when it was added. (This field is case sensitive.)

- Fact Source: the data or calculation that produced the fact.

- Role: the role type that was associated to the fact when it was added.

- Agg Method (Aggregation Method): the mathematical function used to calculate the fact.

- Parent Point Type Code: the point type code associated to a parent object.

- Create Time: the date and time when the fact was created.

- Update Time: the date and time when the fact was last updated.

- Status: the status that was assigned to the fact.

Buttons

- Edit button: opens the Manage Facts dialog box. (See Managing Fact Information for more information.)

- Cancel button: closes the dialog box.

Managing Fact Information

The Administer Facts dialog box is accessed from the Edit link in the Administer Facts data table, or through the Edit button in the View Fact Information dialog box.

Fields and Links

- Fact ID: the fact’s system-assigned, unique identifier.

- Fact Type Code: the fact type code that was associated to the fact when it was added.

- Time Basis: the period of time that is used to aggregate data in order to calculate the fact. (See Time Basis for more information.)

- Fact Category: the category that was associated to the fact when it was added.

- Fact Name: the editable field allows you to review or update the name that was given to the fact when it was added.

- Fact Lookup: the secondary description, or nickname, that was given to the fact when it was added. (This field is case sensitive.)

- Fact Source: the data or calculation that produced the fact.

- Role: the role type that was associated to the fact when it was added.

- Agg Method (Aggregation Method): the drop-down menu allows you to review or update the mathematical function used to calculate the fact.

- Parent Point Type Code: the point type code associated to a parent object.

- Create Time: the date and time when the fact was created.

- Update Time: the date and time when the fact was last updated.

- Status: the drop-down menu allows you to review or update the status that was assigned to the fact.

Buttons

- Save button: allows you to save any changes that have been made; the button is disabled until a field has been modified.

- Cancel button: closes the dialog box.

Configuration

The Administer Configuration features are described in the Oracle DataRaker Cloud Service Configuration Guide.

Administer Security

The Security section contains the following pages:

- Administering DataRaker User Access

- Administer Roles

- Administer Groups

- User Audit

Next in This Section

- Administering DataRaker User Access

- Administer Roles

- Administer Groups

- User Audit

Administering DataRaker User Access

The DataRaker Administer Users page displays all users who have been created in Oracle Identity Manager and configured with the ODR User Provisioning resource.

Note: To access the Administer Users page, log in to DataRaker as an administrative or power user and select the Administer menu, then select Security, and then select Users.

The Administer Users page allows administrators to assign users to groups and roles, which determine the DataRaker features that are available to them:

- Group assignment determines general user interface characteristics (such as the menus that are displayed) and, consequently, which pages are accessible to the user.

- Role assignment determines the features that are available on the pages made available by the user's group privileges.

Note: Roles are unique to DataRaker modules so assigning a role automatically associates the user to a module.

For example, generally end users (non-power users) have access to environments with menus only for Explore and Export functionality based on their group assignment; what they see on the Explore and Export pages is determined by their role (for example, a user with a Billing role in the Meter to Bill module has different algorithms and panels on the Explore page than a user assigned to the Distribution Planning role in the Distribution Planning and Operations module).

Note: The user interface features defined for groups and roles are not customer configurable; rather, they are determined by licensing and implementation.

Fields

The top section of the page allows you to filter the list of users.

- Login: filter by a user's login name.

- First Name: filter by a user's first name.

- Last Name: filter by a user's surname, or family name.

- Email: filter by a user's email address.

- Department: filter by department name.

- Group: filter by group name.

- Role: filter by the role assigned to users.

- Status: filter by user status (Active/Inactive).

Buttons

- Get Users allows you to search for users matching criteria set in the fields. For example, the following image displays the data table filtered by first and last name of the user created in OIM.

Data Table

The data table lists users with columns containing the following information and links:

- Login: the user's login name.

- First Name: the user's first name.

- Last Name: the user's surname, or family name.

- Email: the user's email address.

- Created At: the date and time when the user was created.

- Updated At: the date and time when the user's information was last updated.

- Status: the user status (Active/Inactive).

- Links:

- View: opens the View Users dialog box, which displays user information.

- Edit: opens the Manage Users dialog box, which displays user information and allows you to edit their group and role associations, change their status, and provides a link to change their password in Oracle Identity Manager.

- Settings: field provides the Settings link, which opens the Settings dialog box.

Note: While the Settings dialog box is available, its functionality is currently inactive.

Note: click the expand button () to show the user's information as displayed in the table columns plus their department, group, and role.

Next in This Section

- Viewing User Information

- Managing User Environment Access

- Resetting Passwords

Viewing User Information

The View Users dialog box contains user information and provides links to edit the user's start page and group and role assignments. The dialog box contains the following information and links:

- Login: read-only field with the user's login.

- First Name: read-only field with the user's first name.

- Last Name: read-only field with the user's last name.

- Email: read-only field with the user's email address.

- Department: read-only field with the user's department.

- Group: read-only field with the user's group assignment(s).

- Role: read-only field with the user's role assignment(s).

- Created At: read-only field showing the date and time when the user was created.

- Updated At: read-only field showing the date and time when the user's information was last updated.

- Status: allows you to set the user's status (Active/Inactive).

- Settings: field provides the Settings link, which opens the Settings dialog box.

Note: While the Settings dialog box is available, its functionality is currently inactive.

Available Actions

- To edit the user's group and role assignments or status, click the Edit button. See Managing User Environment Access for details.

Managing User Environment Access

The Manage Users dialog box displays the following fields and links:

- Login: read-only field with the user's login.

- First Name: read-only field with the user's first name.

- Last Name: read-only field with the user's last name.

- Email: read-only field with the user's email address.

- Department: read-only field with the user's department.

- Group: dynamically updated text that lists groups chosen with the Assign Additional Group link.

- Add Group: field provides the Assign Additional Group link.

- Role: dynamically updated text that lists roles chosen with the Assign Additional Role link.

- Add Role: field provides the Assign Additional Role link.

- Created At: read-only field showing the date and time when the user was created.

- Updated At: read-only field showing the date and time when the user's information was last updated.

- Status: allows you to set the user's status (Active/Inactive).

- Settings: field provides the Settings link, which opens the Settings dialog box.

Note: While the Settings dialog box is available, its functionality is currently inactive.

Assigning a User to a Group

1. Click Assign Additional Group. The dialog will update with a drop-down list of the available groups.

Note: The Group options in this section are provided as examples only; your environment may have different group types or group names.

2. Select the appropriate group from the list.

3. Click Save to assign the user to the selected group.
The Manage Users dialog box will update the Group field with the assigned group.

4. If applicable, repeat the previous steps to assign the user to additional groups.

5. When finished adding groups, you may click the Cancel link next to the group selection to close the drop-down list, click the Cancel button on the dialog box to close the dialog box, or select an area in the browser window outside of the dialog box to close the dialog box.

Note: To remove a group assignment, click the Remove link next to the group name.

Assigning a User to a Role

1. Click Assign Additional Role. The dialog will update with a drop-down list of the available roles (based on the licensed module(s)).

2. Select the appropriate role from the drop-down list.

The dialog box will update with the selected role:

3. Click Save to assign the user to the selected role. The Manage Users dialog box will update the Role field with the newly assigned role.

4. If applicable, repeat the previous steps to assign the user to additional roles.

5. When finished adding roles, you may click the Cancel link under the role selection to close the drop-down list, click the Cancel button on the dialog box to close the dialog box, or select an area in the browser window outside of the dialog box to close the dialog box.

Note: To remove a role assignment, click the Remove link next to the role name.

Setting the User Status

The user's status may be set by selecting Active or Inactive from the Status field's drop-down list. Users are active by default, but you may set them to inactive to remove their access to DataRaker.

Resetting Passwords

The Manage Users dialog box Change Password link opens Oracle Identity Manager's Administration page for user administration. With the user selected, click the Reset Password link and follow the prompts to reset the user's password. The new password will be emailed to the user.

Administer Roles

DataRaker roles provide means to control the configuration of the system to refine data views for the tasks associated with the role. (See Using Filters for more information on product-defined roles.)

Fields

The top section of the Administer Roles page allows you to search for roles that match criteria in the fields.

- Role Type: allows you to search for roles that are associated to a specific type.

- Status: the drop-down menu allows you to search by active or inactive status.

- Role Name tenter the name that was given to the role when it was added.

Buttons

- Get Roles: allows you to search for roles matching the criteria set in the top section’s fields.

- Add

Data Table

- Role Type Code: the role type that was assigned to the role when it was added.

- Role Name: the name that was given to the role when it was added.

- Module Name: the DataRaker module that the role was assigned when it was added.

- Create Time: the time and date that the role was added.

- Last Updated: the time and date that a change was made to the role and saved to the system.

- Status: indicates whether the role is active or inactive.

- Links:

- View: opens the Viewing Role Information dialog box, which displays the same information as provided in the data table row. In addition, this displays the role ID, and environment name.

- Edit: opens the Managing Role Information dialog box, which displays the same information as provided in the data table row. In addition, the dialog box allows you to maintain the role type code, role name, module name, and status.

Next in This Section

- Viewing Role Information

- Managing Role Information

- Adding Roles

Viewing Role Information

The View Roles dialog box is accessed from the View link, which is found in the Administer Roles page’s data table. This feature offers additional information to what was displayed in the data table.

- ID: the system-assigned, unique role number.

- Environment Name: the associated environment’s name as assigned when the role was added.

- Role Type Code: the role type that was assigned to the role when it was added.

- Role Name: the name that was given to the role when it was added.

- Module Name: the DataRaker module that the role was assigned when it was added.

- Create Time: the time and date that the role was added.

- Last Updated: the time and date that a change was made to the role and saved to the system.

- Status: indicates whether the role is active or inactive.

Buttons

- Edit: opens the Manage Roles dialog box. See Managing Role Information for more information.

- Cancel: closes the dialog box.

Managing Role Information

The Manage Roles dialog box is accessed from the Edit link in the Administer Roles page’s data table and through the edit button in the View Roles dialog box. This feature offers an opportunity to maintain rile information.

- ID: the system-assigned, unique role number.

- Environment Name: the associated environment’s name as assigned when the role was added.

- Role Type Code: review or update the role type that was assigned to the role when it was added.

- Role Name: review or update the name that was given to the role when it was added.

- Module Name: use the drop-down menu to review or update the DataRaker module that the role was assigned when it was added.

- Create Time: the time and date that the role was added.

- Last Updated: the time and date that a change was made to the role and saved to the system.

- Status: use the drop-down menu to indicate whether the role is active or inactive.

Buttons

- Save: allows you to save any changes that have been made; the button is disabled until a field has been modified.

- Cancel: closes the dialog box.

Adding Roles

To add a role, do the following:

- Click Add Role.

- In the Create Roles dialog box, enter the Role Type Code, the Role Name, and set the Status.

- Click Save

Administer Groups

The Administer Groups page allows you to add and modify the groups to define user permissions.

Fields

The top section of the Administer Groups page allows you to find existing groups that match criteria added to the fields:

- Group Name: allows you to filter by the group name.

- Permission: allows you to filter groups having a specific permission setting.

- Status: allows you to filter by group status (Active/Inactive).

Buttons

- Get Groups: initiates a search for filters matching criteria set in the fields.

Data Table

- Group Name: the group name that was assigned when the group was added.

- Permissions: the rights, or permissions, that the group has been assigned.

- Create Time: the date and time that the group was added.

- Last Updated: the date and time that a change was saved to the group.

- Status: the group’s current status (Active or Inactive).

- Links:

- View: opens the Viewing Group Information dialog box, which contains similar information as displayed in the data table row. In addition, it displays the group’s ID.

- Edit: opens the Managing Group Information dialog box, which contains similar information as displayed in the data table row. In addition, it provides opportunities to update the group name, permissions, and group status.

Next in This Section

- Viewing Group Information

- Managing Group Information

- Adding Groups

- Duplicating Groups

Viewing Group Information

The View Groups dialog box is accessed from the View link, which is found in the Administer Groups page’s data table. This feature offers more information to what is displayed in the Administer Groups data table.

Fields

- ID: the system-assigned identifier.

- Group Name: the group name that was assigned when the group was added.

- Permissions: the rights, or permissions, that the group has been assigned.

- Create Time: the date and time that the group was added.

- Last Updated: the date and time that a change was saved to the group.

- Status: the group’s current status (Active or Inactive).

Buttons

- Edit: opens the Managing Group Information dialog box, which contains similar information as displayed in the data table row. In addition, it provides opportunities to update the group name, permissions, and group status.

- Clone: duplicates and renames the group. Once complete, the dialog box closes and you are returned to the Administer Groups data table to review and edit the new group. See Duplicating Groupsfor more information.

- Cancel: closes the dialog box.

Managing Group Information

The Manage Groups dialog box opens when you click the Edit link in the Administer Groups data table and when you click the Edit button in the View Group dialog box.

Fields

- ID: the system-assigned identifier.

- Group Name: review or edit the group name that was assigned when the group was added.

- Permissions: review or remove the rights, or permissions, that the group has been assigned.

- Add Permission: click the Assign Additional Permission link the add rights to the group.

- Create Time: the date and time that the group was added.

- Last Updated: the date and time that a change was saved to the group.

- Status: use the drop-down menu to review or change the group’s current status (Active or Inactive).

Buttons

- Save: allows you to save any changes that have been made; the button is disabled until a field has been modified.

- Clone: duplicates and renames the group. Once complete, the dialog box closes and you are returned to the Administer Groups data table to review and edit the new group. See Duplicating Groupsfor more information.

- Cancel: closes the dialog box.

Adding Groups

Groups can be added individually by following the procedure below, or by duplicating, cloning, an existing group (for more information about cloning groups, see Duplicating Groups

In order to add a new group, complete the following steps:

1. Navigate to Administer > Security > Groups.

2. Click the Add button.

3. When the Create Groups dialog box appears, enter in a unique Group Name.

4. Select Inactive from the Status drop-down menu if the group should not be in effect at this time.

5. Click Save.

6. Locate your group in the Administer Groups data table and click Edit.

7. In the Manage Groups dialog box:

- Update the Group Name, if necessary.

- Remove any permissions that should not be associated with this group.

8. Add any permissions that are missing by clicking the Assign Additional Permission link.

9. Review and update the status as necessary.

10. Click Save.

Duplicating Groups

In addition to creating groups by adding them, groups can also be duplicated, or cloned. For more information about adding groups without cloning, see Adding Groups.

In order to clone a new group, complete the following steps:

1. Navigate to Administer > Security > Groups.

2. Locate the template group and click either the View or Edit link.

3. When the associated dialog box appears, click the Clone button.

4. The system will create a duplicate of the group, rename it, and return you to the Administer Groups page.

5. Locate your group in the Administer Groups data table and click Edit.

6. In the Manage Groups dialog box:

- Update the Group Name, if necessary.

- Remove any permissions that should not be associated with this group.

7. Add any permissions that are missing by clicking the Assign Additional Permission link.

8. Review and update the status as necessary.

9. Click Save.

User Audit

Fields

The top section of the Administer User Audit page provides filters that allow you to narrow the user accounts that are returned.

- Type: allows you to choose between a summary and detail report. (Note: these are distinct reports. See the Data Table section for more information about the different columns that appear for each audit.)

- Role: search for user information based on a specific role.

- User Name: search by a user’s name

- Point Type: search for user information that is associated with a specific point type.

Buttons

- Get Logs: allows you to search for user logs matching the criteria set in the top section’s fields.

Data Table

The columns of the data table are different depending on whether a Summary (default) or Detail report is defined in the top section.

Columns associated to a summary audit:

- User Name: the assigned system name that was given to the user when they were added.

- Date: the date of the event.

- Hit Count: the number of references associated to the event.

- Distinct Fact Count: the number of facts associated to the event.

- Distinct Point Count: the number of points associated to the event.

Columns associated to a detail audit:

- User Name: the assigned system name that was given to the user when they were added.

- View:

- Point Type Code: the point type associated with the event.

- Point Name: the name that was given to the associated point when it was added

- Fact Type Code: the fact type that is associated with the event.

- Fact Type Category: the category of fact that is associated with the event.

- Fact Name: the name that was given to the associated fact when it was added.

- Datetime: the date and time associated with the event.

- Links:

- View: displays the View User dialog box, which contains the same information as the detailed audit’s data table row. In addition, the dialog box displays the fact time basis, start date, end date, and extra parameters associated with the row.

Administer System

The System section contains the following pages:

Next in This Section

- Administer Settings

- Caching

Administer Settings

The top section of the Administer Settings page allows you to search for tasks matching criteria in the fields:

- Setting ID: the setting’s system-assigned, unique identifier.

- Setting Type: drop-down list to search by the setting type.

- Setting Name: the setting’s name that was assigned to the setting when it was added.

- Setting Description: a short explanation of the settings purpose or function.

- Status: drop-down list to search by settings that can be applied (Active) or not (Inactive).

Buttons:

- Get Settings: executes the search based on the criteria entered in the fields in the top section.

Data Table:

- ID: displays the setting’s system-generated, unique identifier.

- Setting Type: the type that was assigned when the setting was added.

- Setting Name: the name that was assigned to the setting when it was added.

- Setting Description: the short explanation that was assigned to the setting when it was added.

- Status: indicates whether the setting can be applied (Active) or not (Inactive).

- Create Transaction ID: the system-generated, unique identifier that was assigned to the transaction that was created when the setting was added.

- Update Transaction ID: the system-generated, unique identifier that was assigned to the transaction that was created when the setting was updated, if applicable.

Caching

Buttons:

- Clear Cache: clears the server cache and displays a message related to the action.

- Get Server Stats: displays summary information about the server’s current processing.

Monitor

The Monitor menu provides access to the Queues, Database, Cluster, Distributed FS, and Nodes features.

Next in This Section

- Queues

Queues

The top section contains a single field, a button, and a refresh indicator.

- Refresh Seconds: enter the number of seconds before the system information updates.

- Get Data button: retrieves system information.

- Countdown timer: displays a graphical representation of how soon the displayed information will refresh. The grey bar decreases in size showing each second that has elapsed.

Note: When the data in the table refreshes, there is a brief yellow flash to indicate the change.

The display area shows a matrix of information about the server’s queues.

- Each field in the table displays the number of processes that are Active, Waiting, or Buried:

- Active: displays the number of processes that are currently running.

- Waiting: displays the number of processes that are on hold or pending their dependencies to complete.

- Buried:

- Each column in the table displays the type of process; for example the image above is divided displays Fact Audits, Runs, and Tasks.

- Each row in the table displays the server node that is processing the information; for example, the image above shows three nodes (rwssg01.us.oracle.com, rwssg03.us.oracle.com, and rwssg02.us.oracle.com).