Dgraph out log

The Dgraph out log is where the Dgraph's stdout/stderr output is remapped.

The Dgraph redirects its stdout/stderr output to the log file specified by the Dgraph --out flag. By default, the name and location of the file is: 
$BDD_HOME/logs/dgraph.out

You can specify a new out log location by changing the DGRAPH_OUT_FILE parameter in the bdd.conf file and then restarting the Dgraph.

The Dgraph stdout/stderr log includes startup messages as well as warning and error messages. You can increase the verbosity of the log via the Dgraph -v flag.

Dgraph out log format

The format of the Dgraph out log fields are:
  • Timestamp
  • Component ID
  • Message Type
  • Log Subsystem
  • Job ID
  • Message Text
The log entry fields and their descriptions are as follows:
Log entry field Description Example
Timestamp The local date and time when the message was generated, using use the following ISO 8601 extended format: 
YYYY-MM-DDTHH:mm:ss.sss(+|-)hh:mm

The hours range is 0 to 23 and milliseconds and offset timezones are mandatory.

2016-03-18T13:25:30.600-04:00
Component ID The ID of the component that originated the message. "DGRAPH" is hard-coded for the Dgraph. DGRAPH
Message Type The type of message (log level):
  • INCIDENT_ERROR
  • ERROR
  • WARNING
  • NOTIFICATION
  • TRACE
  • UNKNOWN
WARNING
Log Subsystem The log subsystem that generated the message. {dgraph}
Job ID The ID of the job being executed. [0]
Message Text The text of the log message. Starting HTTP server on port: 7010

Dgraph log subsystems

The log subsystems that can generate log entries in the Dgraph out log are the following:
  • background_merging — messages about Dgraph database maintenance activity.
  • bulk_ingest — messages generated by Bulk Load ingest operations.
  • cluster — messages about ZooKeeper-related cluster operations.
  • database — messages about Dgraph database operations.
  • datalayer — messages about Dgraph database file usage.
  • dgraph — messages related to Dgraph general operations.
  • eql — messages generated from the Endeca Query Language engine.
  • eve — messages generated from the EVE (Endeca Virtual Engine) query evaluator.
  • http — messages about Dgraph HTTP communication operations.
  • lexer — messages from the OLT (Oracle Language Technology) subsystem.
  • splitting — messages resulting from EVE (Endeca Virtual Engine) splitting tasks.
  • ssl — messages generated by the SSL subsystem.
  • task_scheduler — messages related to the Dgraph task scheduler.
  • text_search_rel_rank — messages related to Relevance Ranking operations during text searches.
  • text_search_spelling — messages related to spelling correction operations during text searches.
  • update — messages related to updates.
  • workload_manager — messages from the Dgraph Workload Manager.
  • ws_request — messages related to request exchanges between Web services.
  • xq_web_service — messages generated from the XQuery-based Web services.

All of these subsystems have a default log level of NOTIFICATION.

Dgraph startup information

The first log entry (that begins with "Starting Dgraph") lists the Dgraph version, startup flags and arguments, and path to the Dgraph databases directory. Later entries log additional start-up information, such as the amount of RAM and the number of logical CPUs on the system, the CPU cache topology, the created Web services, HTTP port number, and bulk load port number.

Dgraph shutdown information

As part of the Dgraph shutdown process, the shutdown details are logged, including the total amount of time for the shutdown. For example (note that timestamps have been removed for ease of reading): 
...
DGRAPH	NOTIFICATION  	{dgraph}	  [0]	Shutdown request received at Tue Mar 29 16:59:17 2016.  
                                     Shutdown will complete when all outstanding jobs are complete. 	
DGRAPH	NOTIFICATION  	{database}	[0]	Finished unmounting everything.	
DGRAPH	NOTIFICATION  	{dgraph}	  [0]	All dgraph transactions completed at Tue Mar 29 16:59:17 2016, exiting normally (pid=3701) 	
DGRAPH	WARNING       	{cluster} 	[0]	Lost connection to ZooKeeper: ZooKeeper connection lost (zk error -4)	
DGRAPH	NOTIFICATION  	{cluster}	 [0]	Finished closing zk connection	
DGRAPH	NOTIFICATION  	{dgraph}	  [0]	Overall shutdown took 1194 ms

Out log ingest example

The following snippets from a Dgraph out log show the entry format for an ingest operation. Note that timestamps have been removed for ease of reading. 
...
DGRAPH	NOTIFICATION  	{cluster}     [0]	Promoting to leader on database edp_f475de43
DGRAPH	NOTIFICATION  	{database}    [0]	Mounting database edp_f475de43	
DGRAPH	NOTIFICATION  	{dgraph}      [0]	Initial DL version: 2 	
DGRAPH	NOTIFICATION  	{bulk_ingest}	[0]	MessageParser constructor, parserCounter incremented, is now 1	
DGRAPH	NOTIFICATION  	{bulk_ingest}	[0]	Start ingest for collection: edp_f475de43 for database edp_f475de43
DGRAPH	NOTIFICATION  	{bulk_ingest}	[0]	Starting a bulk ingest operation for database edp_f475de43	
DGRAPH	NOTIFICATION  	{bulk_ingest}	[0]	batch 0 finish BatchUpdating status Success for database edp_f475de43	
DGRAPH	NOTIFICATION  	{bulk_ingest}	[0]	Ending bulk ingest at client's request for database edp_f475de43 - finalizing changes	
DGRAPH	NOTIFICATION  	{bulk_ingest}	[0]	Bulk ingest completed: Added 20000 records and rejected 0 records, for database edp_f475de43	
DGRAPH	NOTIFICATION  	{bulk_ingest}	[0]	Ingest end - 11.663MB in 7.042sec = 1.656MB/sec for database edp_f475de43

The bulk_ingest entries show the ingest of a data set with 20,000 records.