Transaction event log settings

Overview

The Transaction Event Log provides a summary of each API Gateway message transaction, which is written to a log file, and used to generate metrics for API Gateway monitoring (for example, in the Monitoring view in API Gateway Analytics). In a distributed system with multiple API Gateway instances running, the events data is written to separate transaction event log files for each API Gateway instance.

The event log file data is processed by the local Node Manager every 5 minutes, aggregated into the appropriate metrics data, and then written to a database. API Gateway Analytics can use the data from the database to report on analytics in the system. Event log file data is written in JSON format, which also enables it to be integrated with third-party logging tools such as Splunk.

[Note] Note
Node Manager processing of event log data is not enabled by default. You must enable the Node Manager to write metrics to the database. For more details, see the section called “Configure the Node Manager to process event logs and update the metrics database”.

For more details how metrics are displayed, see the following:

Transaction event log formats

Event log files are located in the events directory of your API Gateway installation by default. For example:

INSTALL-DIR/apigateway/events/group-2_instance-1.log

When each event log file has been processed (every 5 minutes), it can be moved to a processed directory. For example:

INSTALL-DIR/apigateway/events/processed

By default, files are deleted after being processed.

Entries in the transaction event log file are generated for different event types (for example, header, system, transaction, and custom).

Event log header entries

Event log header entries contain details about the creation of the log file. For example, this includes when the log file is created, and on which host, domain, group, instance, and so on.

The following example shows the JSON format used for header events:

{
    "type": "header",
    "logCreationTime": "2015-01-23 12:25:00.120",
    "hostname": "Stephen-PC",
    "domainId": "cfbe55d1-be45-4968-8b4b-f06a4db858b8",
    "groupId": "group-2",
    "groupName": "QuickStart Group",
    "serviceId": "instance-1",
    "serviceName": "QuickStart Server",
    "version": "v7.4.0-Internal"
}
...

Event log system entries

Event log system entries contain details about the API Gateway system. For example, this includes details such as the amounts of disk space, memory, and CPU.

The following example shows the JSON format used for system events:

{
    "type": "system",
    "time": 1422015900120,
    "diskUsed": 30,
    "instUsed": 1,
    "sysCpu": 5,
    "instMem": 533436,
    "sysMem": 4641996,
    "sysMemTotal": 16759240
}
...

Event log transaction entries

Event log transaction entries contain details about a specific message transaction. For example, this includes details such as the protocol, method, bytes sent and received, IP addresses, ports, service name, and so on.

The following example shows the JSON format used for an HTTP transaction event with a service context and inbound and outbound transaction legs:

{
    "type": "transaction",
    "time": 1425291330502,
    "path": "/stockquote.asmx",
    "protocol": "http",
    "protocolSrc": "8080",
    "duration": 1842,
    "status": "success",
    "serviceContexts": [
        {
            "service": "StockQuote",
            "monitor": true,
            "client": null,
            "org": null,
            "app": null,
            "method": "GetQuote",
            "status": "success",
            "duration": 1824
        }
    ],
    "customMsgAtts": {},
    "correlationId": "4038f4540400788ebe4f84ca",
    "legs": [
        {
            "uri": "/stockquote.asmx",
            "status": 200,
            "statustext": "OK",
            "method": "POST",
            "vhost": null,
            "wafStatus": 0,
            "bytesSent": 1278,
            "bytesReceived": 612,
            "remoteName": "127.0.0.1",
            "remoteAddr": "127.0.0.1",
            "localAddr": "127.0.0.1",
            "remotePort": "49104",
            "localPort": "8080",
            "sslsubject": null,
            "leg": 0,
            "timestamp": 1425291328660,
            "duration": 1843,
            "serviceName": "StockQuote",
            "subject": null,
            "operation": "GetQuote",
            "type": "http",
            "finalStatus": "Pass"
        },
        {
            "uri": "/stockquote.asmx",
            "status": 200,
            "statustext": "OK",
            "method": "POST",
            "vhost": null,
            "wafStatus": 0,
            "bytesSent": 736,
            "bytesReceived": 1202,
            "remoteName": "www.webservicex.net",
            "remoteAddr": "173.201.44.188",
            "localAddr": "10.142.10.142",
            "remotePort": "80",
            "localPort": "49438",
            "sslsubject": null,
            "leg": 1,
            "timestamp": 1425291329916,
            "duration": 566,
            "serviceName": "StockQuote",
            "subject": null,
            "operation": "GetQuote",
            "type": "http",
            "finalStatus": null
        }
    ]
}
...

Configure the transaction event log

To configure the transaction event log in the Policy Studio tree, select the Server Settings node, and click Logging > Transaction Event Log. Alternatively, in the Policy Studio main menu, you can select Tasks > Manage Gateway Settings.

Configure the following fields to enable the API Gateway instance to write a transaction event log to a file:

Writing to Transaction Event Log:

Enables writing to an event log for all message transactions received by the API Gateway. This setting is enabled by default, and is required for API Gateway Analytics. For example, you could deselect this setting to optimize performance.

Write transaction event logs to directory:

Specifies the directory where transaction event logs are written. Defaults to ${environment.VDISTDIR}/events.

[Note] Note
If transaction event logs are being used to populate the metrics database, you must also update the sourceEventLogDir property in the Node Manager configuration if you change this directory. For more details, see Monitoring and reporting with API Gateway Analytics.

System event frequency (secs):

Specifies how often in seconds that a system entry is written to each event log file. Defaults to 60 seconds. For more details, see the section called “Event log system entries”.

Maximum disk space for event logs (MB):

Specifies the maximum amount of disk space used for event logs. When the directory reaches the specified limit, the oldest log files are deleted. Defaults to 1024 MB.

Check disk space interval (secs):

Specifies how often the amount of available disk space used for event logs is checked. Defaults to 600 seconds.

Select the message attributes to be stored in transaction events:

Enables you to specify custom message attributes to write to the transaction event logs (for example, the HTTP request URI). To specify an attribute, click Add, and enter the attribute name in the dialog.

[Note] Note

If the local Node Manager is configured to write transaction event logs to the metrics database, these attributes are written to the transaction_data table. Data from this table is not displayed in API Gateway Analytics. However, you can access this data using SQL queries.

You can disable writing of this data to the database by configuring the processCustomMessageAttributes option in the esexplorer tool. For details, see the section called “Configure additional options for event log processing in the Node Manager”.

To confirm updates to these settings, click Apply changes at the bottom right of the window. Click Deploy in the toolbar to deploy the updated configuration to the API Gateway.