On UNIX systems, the
ps
command may report a number of defunct EAC-originated
processes. This is known and expected EAC behavior and it does not necessarily
indicate a problem.
For example, you
might see the following output from the
ps
command:
> ps -ef | grep endeca endeca 1924 1875 0 - ? 2:00 <defunct> [...]
Additionally, warning messages of this form appear in the
$ENDECA_CONF/logs/process.0.log
file on the affected
server:
Apr 17, 2009 11:24:17 AM com.endeca.esf.delegate.procctrl.ExecutableProcessHandle tryCleanShutdown WARNING: Process 1924 did not shutdown cleanly after 30 seconds. Terminating forcefully.
The cause of these warning messages is as follows. When the EAC shuts
down a child process like a Dgraph, it initially sends the correct exit command
for the process (admin?op=exit
in the case of the Dgraph)
and waits 30 seconds for the process to exit. However, if the Dgraph is
processing a long-running query, or if its request queue is long, it may not be
able to shut down within 30 seconds.
If the process does not exit after 30 seconds, the EAC logs the warning
message shown above and then kills the process with the operating system's
kill
command. When this occurs, the affected process is
reported by
ps
as being in a
<defunct>
state. In this state, it does not use
memory, disk space, or ports and should not be a problem for the system.
Alternatively, this can happen if you kill the EAC process directly
rather than by using the
shutdown.sh
script. In this case the EAC process
terminates immediately, leaving any chlid processes in a
<defunct>
state.
To avoid defunct EAC processing, consider the following recommendations:
The request log shows whether queuing or long processing times are preventing the Dgraph from responding in time to the
admin?op=exit
command. If this is the case, spreading traffic over a larger number of MDEX Engine mirrors (for queuing) or reducing query complexity (for long processing times) should allow the Dgraph to respond more quickly to the exit command.Another option may be to override the default 30-second timeout period for EAC shutdowns by modifying the value of the
com.endeca.eac.process.shutdownTimeoutSecs
setting in your server's$ENDECA_CONF/conf/eac.properties
file.This value shows the length of time in seconds that the EAC Agent on that server waits for a process to exit. Specifying a higher value for this setting may help prevent creation of
<defunct>
EAC child processes, but may also make EAC updates slower, because the EAC Agent will wait longer for all processes to exit.Note
Modifications to this setting will not take effect until the Endeca HTTP service (which contains the EAC) is restarted on the server.