DP log entry format

This topic describes the format of Data Processing log entries, including their message types and log levels.

The following is an example of a NOTIFICATION message resulting from the part of the workflow where DP connects to the Hive Metastore: 
[2015-07-28T11:45:08.502-04:00] [DataProcessing] [NOTIFICATION] [] [hive.metastore] 
   [host: web09.example.com] [nwaddr: 10.152.105.219] [tid: Driver] [userId: yarn] 
   [ecid: 0000KvLLfZE7ADkpSw4Eyc1LhuE0000002,0] Connected to metastore.
The format of the DP log fields (using the above example) and their descriptions are as follows:
Log entry field Description Example
Timestamp The date and time when the message was generated. This reflects the local time zone. [2016-04-28T11:45:08.502-04:00]
Component ID The ID of the component that originated the message. "DataProcessing" is hard-coded for the DP component. [DataProcessing]
Message Type The type of message (log level):
  • INCIDENT_ERROR
  • ERROR
  • WARNING
  • NOTIFICATION
  • TRACE
  • UNKNOWN
 [NOTIFICATION]
Message ID The message ID that uniquely identifies the message within the component. The ID may be null. []
Module ID The Java class that prints the message entry. [hive.metastore]
Host name The name of the host where the message originated. [host: web09.example.com]
Host address The network address of the host where the message originated [nwaddr: 10.152.105.219]
Thread ID The ID of the thread that generated the message. [tid: Driver]
User ID The name of the user whose execution context generated the message. [userId: yarn]
ECID The Execution Context ID (ECID), which is a global unique identifier of the execution of a particular request in which the originating component participates. Note that [ecid: 0000KvLLfZE7ADkpSw4Eyc1LhuE0000002,0]
Message Text The text of the log message. Connected to metastore.