Dgraph request log and stdout/sterr log

Any Dgraph node in a data domain creates two logs.

You can use these Dgraph logs to troubleshoot queries, or to track performance of particular queries or updates.

By default, the Dgraph logs are stored in the $DOMAIN_HOME/EndecaServer/logs directory. For example, if your domain name is endeca_server_domain, the pathname on Windows is: 
C:\Oracle\Middleware\user_projects\domains\endeca_server_domain\EndecaServer\logs

You can specify another location for the logs by changing the path in the endeca-logs-dir parameter in the EndecaServer.properties configuration file.

Because the Dgraph logs are text files, you can use a text editor to view them.

Dgraph request log

The Dgraph request log (also called the query log) contains one entry for each request processed. The requests are sorted by their timestamp.

The default name of the Dgraph request log is: 
dataDomainName.reqlog
where dataDomainName is the name of the Endeca data domain that the Dgraph node is servicing.

Also, the /admin/dataDomainName?op=logroll command forces a query log roll, with the side effect of remapping the request log.

The format of the Dgraph request log consists of fourteen fields, which contain the following information:
  • Field 1: Timestamp (UNIX Time with milliseconds)
  • Field 2: Client IP Address
  • Field 3: Outer Transaction ID, if defined
  • Field 4: Request ID
  • Field 5: Response Size (bytes)
  • Field 6: Total Time (fractional milliseconds)
  • Field 7: Processing Time (fractional milliseconds)
  • Field 8: HTTP Response Code (0 on client disconnect)
  • Field 9: - (unused)
  • Field 10: Queue Status (on request arrival, number of requests in queue if positive, or number of available slots at the same priority if negative)
  • Field 11: Thread ID
  • Field 12: HTTP URL (URL encoded)
  • Field 13: HTTP POST Body (URL encoded; truncated to 64KBytes, by default; - if empty)
  • Field 14: HTTP Headers (URL encoded)

Note that a dash (-) is entered for any field for which information is not available or pertinent.

By default, the Dgraph truncates the contents of the body for POST requests at 64K. This default setting saves disk space in the log, especially during the process of adding large numbers of records to the data domain. If you need to review the log for the full contents of the POST request body, contact Oracle support.

Dgraph stdout/stderr log

The default name of the Dgraph stdout/stderr log is: 
dataDomainName.out

The Dgraph stdout/stderr log includes startup messages as well as warning and error messages. You can increase the verbosity of the log via the Dgraph -v flag. You can also set the logging variables to toggle logging verbosity for specified features, which are described in List of supported logging variables.

Note that the Dgraph stdout/stderr log reports startup and shutdown times (and other informational messages) using the system's local time zone, with no zone label displayed, but displays warning and error messages in UTC.

Using grep on the Dgraph request log

When diagnosing performance issues, you can use grep with a distinctive string to find individual requests in the Dgraph request log. For example, you can use the string: 
value%3D%22RefreshDate
If you have Studio, it is more useful to find the X-Endeca-Portlet-Id HTTP Header for the portlet sending the request, and grep for that. This is something like: 
X-Endeca-Portlet-Id: endecaresultslistportlet_WAR_endecaresultslistportlet_INSTANCE_5RKp_LAYOUT_11601
As an example, if you set: 
PORTLET=endecaresultslistportlet_WAR_endecaresultslistportlet_INSTANCE_5RKp_LAYOUT_11601
then you can look at the times and response codes for the last ten requests from that portlet with a command such as: 
grep $PORTLET Discovery.reqlog | tail -10 | cut -d ' ' -f 6,7,8

The command produces output similar to: 

20.61 20.04 200
80.24 79.43 200
19.87 18.06 200
79.97 79.24 200
35.18 24.36 200
87.52 86.74 200
26.65 21.52 200
81.64 80.89 200
28.47 17.66 200
82.29 81.53 200
There are some other HTTP headers that can help tie requests together:
  • X-Endeca-Portlet-Id — The unique ID of the portlet in the application.
  • X-Endeca-Session-Id — The ID of the user session.
  • X-Endeca-Gesture-Id — The ID of the end-user action (not filled in unless Studio has CLIENT logging enabled).
  • X-Endeca-Server-Query-Id — If multiple dgraph requests are sent for a single Endeca Server request, they will all have the same X-Endeca-Server-Query-Id.

For information on enabling Studio logging, see the Oracle Endeca Information Discovery Studio Administration and Customization Guide.