Enabling Search Logging

In addition to the application server logs, Oracle ATG Web Commerce Search includes two additional log file types:

Note: Log files can quickly consume a great deal of space. Either clean out the logging directory often, or turn logging off when it is not required.

Enabling Search Activity Logs

The search activity files record the request input and response output for an engine. Errors in the response files can help with troubleshooting, and a missing response might indicate a timeout or other communication problem.

By default, the engine activity files log only request and response pairs corresponding to errors. You can use Dynamo Server Admin to enable logging for other activities. To enable logging:

  1. Go to http://host:port/dyn/admin/nucleus/atg/search/routing/Configuration (see the ATG Installation and Configuration Guide for information on Dynamo Server Admin).

  2. Change the loggingAllRequests property value to true to log all requests made to the search engine. You can also set the following individual properties to true to log only certain request types:

    • loggingDeploy

    • loggingIndex

    • loggingLoad

    • loggingSearch

    • loggingStart

    • loggingStop

    The indexing server writes out request and response XML to the following directory:

    <ATG10dir>\home\logs\searchEngineActivity

The names of the log files provide information about the contents. For example, the log file name 0006320004_m_2009-04-21-101254_26200058_mymachine_6100_search_request.xml includes the following:

  • 0006320004—Request ID

  • m—Indicates that the file is a merge file, representing the final results returned to the user. The merge files do not include a partition, host, or port number, since they include data from multiple partitions, and potentially multiple hosts.

  • 2009-04-21-101254—Timestamp in YYYY-MM-DD-HHMMSS format

  • 26200058—Partition ID

  • mymachine—Host name

  • 6100—Port number

  • search—Request type

  • request—File type

To change the directory where activity files are stored, edit the /atg/search/routing/Configuration.engineActivityPath property value and set it to the directory you want to use.

You can view the most recent activity logs directly in Dynamo Server Admin. To view activity logs, navigate to http://host:port/dyn/admin/nucleus/atg/search/routing/SearchEngineService. Note that the log history visible in Dynamo Server Admin is reset when the application server or search engine is restarted; however, files in the SearchEngineActivity directory are not affected by restarts.

Enabling Search Engine Logs

Each search engine has an associated engine log file. Errors in the log files can help with troubleshooting, and a missing log file can indicate a problem with starting that engine.

The engine log contains platform, build, and system information at the beginning of the file:

2009/05/04 15:56:14 : LOGGING : Server : 0 :      Platform: x86-Windows32
2009/05/04 15:56:14 : LOGGING : Server : 0 :      Build: ATGSearch_10_090504-
002806
2009/05/04 15:56:14 : LOGGING : Server : 0 :      Unicode Support: utf8
2009/05/04 15:56:14 : LOGGING : Server : 0 :      Processors: 2
2009/05/04 15:56:14 : LOGGING : Server : 0 :      Processor speed (MHz): 2993
2009/05/04 15:56:14 : LOGGING : Server : 0 :      Total physical memory (MB): 4029
2009/05/04 15:56:14 : LOGGING : Server : 0 :      Avail physical memory (MB): 1399
2009/05/04 15:56:14 : LOGGING : Server : 0 :      Total swap space (MB): 6238
2009/05/04 15:56:14 : LOGGING : Server : 0 :      Avail swap space (MB): 3744

The platform and build information can help you determine whether the correct version of the engine is installed. The system information can show whether the system has sufficient processors, physical memory, and swap space.

A search engine log also contains the index filename, partition name, and the Content ID (also known as Partition GUID) associated with it, as shown in the following example:

2009/05/04 15:56:19 : INFO : Server : 0 : Starting copy of
'C:\ATG\ATG10.2\Search10.2\SearchEngine\x86-win32\data\initial.index' to
'../indexFiles\400001\initial.index'

2009/05/04 15:56:25 : INFO : Server : 0 : Content ID: 0fd62287-c697-4fe4-
8639-716e2c0e2f07

This information can show whether the path to the deployment share directory is correct, and can be used to match the partition listed in the search activity logs.

Each step in the indexing process generates a separate log; a full index task generates a minimum of three logs, corresponding to the estimation, indexing, and deployment steps. If the index spans multiple physical partitions, each additional partition generates additional logs for the indexing and deployment steps.

For an estimation or indexing step, the search engine is launched with an initial partition. This is reflected in the logs by the index filename initial.index. For a deployment step, the engine log starts with an existing partition. This is reflected in the logs by filenames with idx and stg extensions. The content ID from the deployment step matches that of the prior indexing step; however the estimation step uses a different content ID. See the following example:

2009/05/04 15:57:14 : INFO : Server : 0 : Starting copy of
'C:\ATG\ATG10.2\Search10.2\bolt.100002.ContentSet1.400020.idx' to
'../indexFiles\400003\bolt.100002.ContentSet1.400020.idx'
2009/05/04 15:57:14 : INFO : Server : 0 : Starting copy of
'C:\ATG\ATG10.2\Search10.2\bolt.100002.ContentSet1.400020.stg' to
'../indexFiles\400003\bolt.100002.ContentSet1.400020.stg'
2009/05/04 15:57:15 : INFO : Server : 0 : Content ID: 9b6342d1-bfb2-4010-
992c-9793c7ff0cae

The default logging level is I (information). This level includes warnings and errors, but does not include requests and responses from the engine. Set the logging level to d (debug) to include the request and response XML in the output, as in the following example:

2009/05/04 18:11:09 : DEBUG : Server : 0 : request RequestXML: <query
minScore='300' RQText='ANSWER' exclusion='' sorting='SCORE' suggestion=
'TERMSET' version='2.0.1' autoRefine='false' docSetSort='None' debug='0'>
2009/05/04 18:11:09 : DEBUG : Server : 0 : response RequestXML: <?xml
version="1.0" encoding="utf-8"?>
<answer contentID="d2127cff-6209-4e3a-8ed4-
3800461c33ad" sorting="score" docSetSort="None"

To enable engine logging:

  1. Go to http://host:port/dyn/admin/nucleus/atg/search/routing/Configuration (see the ATG Installation and Configuration Guide for information on Dynamo Server Admin).

  2. Change the engineLogLevel property value to D to enable debug logging. This will log all requests and takes effect upon the next engine restart.

    The search engine writes out files named atgsearch_*.log to the following directory:

<ATG10dir>\Search10.2\SearchEngine\platform\bin

To change the directory where engine log files are stored, create a filelog_basepath environment variable and set it to the directory you want to use.

By default, engine log files are retained for two days. To configure how long the files are retained, edit the params property value of the /atg/search/routing/LaunchingService.properties file and add a –t flag as shown:

params=atgsearch,-p,$port$,-s,AESoapWaspConfig.xml,-a,$AEConfigXml$,
  -m,Messages.xml,-l,$loglevel$,-t,NN

The NN value indicates how long, in days, to retain the log files.

Controlling Slow Response Logging

Among other data, the search logs include warning information on search requests that take an unusual amount of time to process. An example of the warning logged is shown:

2009/08/03 13:34:13 : WARNING : Server : 0 : Slow request (5.18):
<query minScore='300' RQText='ANSWER' exclusion='' sorting='SCORE'
suggestion='TERMSET' version='2.0.1' autoRefine='false' docSetSort='None'
debug='0' sleepTimeMs="5000">

The Slow request (time) element shows the actual search processing time. Processing time excludes time spent in the queue before execution, so total elapsed time for the search could be longer under certain conditions.

If your logs are cluttered with extraneous warnings, or you want to troubleshoot particular slow searches, you can raise or lower the logging threshold for slow responses in the AEConfig.xml file located in <ATG10dir>/Search10.2/SearchEngine/platform/bin.

<SlowRequestThreshold>5000</SlowRequestThreshold>

The default is 5000 milliseconds.

Copyright © 1997, 2013 Oracle and/or its affiliates. All rights reserved. Legal Notices