Monitoring Pipeline Manager

For information about improving Pipeline Manager performance, see "Optimizing Pipeline Manager Performance".

Monitoring Pipeline Manager Memory Usage

You can use the MemoryMonitor module to monitor Pipeline Manager memory during startup and while it is processing files. You set a threshold for the amount or percentage of memory that determines when Pipeline Manager should issue a warning or gracefully shut down. You can set the thresholds as a percentage or as kilobytes or megabytes.

For example, if you set ShutdownFreeMemLimit to 50 and ScaleUnit to M, Pipeline Manager shuts down gracefully when the remaining free system memory reaches 50 MB. If you set WarningFreeMemLimit to 10 and ScaleUnit to P, Pipeline Manager logs a warning when the remaining free system memory reaches 10 percent.

Monitoring Pipeline Manager EDR Throughput

You can monitor the following statistics for each pipeline:

Number of event data records (EDRs) since startup.
Accumulated EDR processing time since startup.
Total number of EDRs since startup, independent of any transaction. This number is incremented after every processed EDR.
Total number of EDRs after the transaction ended. This number is not incremented until the current transaction has ended.
The real-time EDR count increments after each EDR is processed, while the transaction count increments EDR count only after transaction/file processing is ended.
Number of transactions since startup.
EDRs per second (throughput). This data includes the timestamp of when the measurement was taken.

You can use the Operations Management Framework (OMF) HTTP protocol to access the data.

Getting Recent Pipeline Log File Entries

You can display recent log file entries in the OMF HTTP server. The entries are also included in the Diagnostic Data Handler output file.

The log messages are stored in a circular buffer that stores the last 1000 log messages.

You can change the number of error messages stored in the buffer. To do so, edit the CircularBufferSize registry entry in the ITO section.

For example:

ProcessLog
{
    ModuleName = LOG
    Module
    {
        ITO
        {
            LogLevel            = Debug
...
            CircularBufferSize  = 100
         }

Using the pin_db_alert Utility to Monitor Key Performance Indicators

KPIs are metrics you use to quantify the health of your database and to alert you when potential issues exist. They identify database tables that must be archived or purged and indexes, triggers, and stored procedures that are missing or invalid.

The PROCEDURES KPI is used for Pipeline Manager.

The proceduresList module retrieves a list of stored procedures in the BRM system and writes the stored procedure names and status (VALID or INVALID) to the proceduresList_PROCEDURES.out file. This enables Pipeline Manager to compile data in parallel and to restore it from the precompiled data files file.

The proceduresList_validation module compares the list of stored procedures in the proceduresList validation configuration file to the procedures in the results file and writes missing procedures to the proceduresList_validation_PROCEDURES.out file.

Important: If you installed optional managers that use unique stored procedures or if you created custom stored procedures, you must add them to the proceduresList validation configuration file to monitor their status.

KPIs are monitored when you run the pin_db_alert.pl utility.

Collecting Diagnostic Information by Using RDA

Remote Diagnostic Agent (RDA) is an Oracle standard tool used to collect diagnostic data from your system applications environment.

You can use RDA to collect BRM and Pipeline Manager diagnostic information. The information collected from Pipeline Manager includes:

Configuration files

RDA collects the pipeline configuration data from the .reg (registry) and .dsc (description) files. For example, RDA collects the configuration data for wireless from the wireless.reg and containerDesc.dsc files.
Log files

RDA collects pipeline log data from the process log, pipeline log, and stream log files. For example, RDA collects the log data for wireless from the processWIRELESS.log file, the log_streamRT1.log file, and so on.
Other files

RDA collects pipeline installation and version details from the vpd.properties and piperev.dat files.

To find Pipeline Manager information, RDA looks at the registry files. To collect pipeline log files, verify that the INT_HOME environment variable is set to the Pipeline Manager installation directory.