glog.dataquery Properties

The data query breaks up matched objects into batches, where the batch size is given by this property, and defaults to 50. A batch creates a new process to hold the batch and publishes all of its lifetime events in parallel under that umbrella process. The batch size identifies the number of records to be simultaneously processed. When the data query starts, a full batch of records is sent to the event queues. As each record completes, another record takes its place.

If the data query is a long running task, the batch size also controls the frequency of progress updates and interruption checks. For large data queries, too frequent progress updates can impact performance.

The recommendations for batch size, then, are:

1. Size the batch to exceed the number of threads on the bottlenecked event queue.

E.g., if you are triggering an agent whose main work runs on agentPlanning and you have 80 agentPlanning threads, make sure the batch size >= 80.

2) Size the batch to avoid too frequent Long Running Task updates. E.g., if your data query matches a million records, the default batch size of 50 will generate 20,000 progress updates. Increasing this batch size to 1000 reduces the progress updates to 1000.

If the batch size is not defined on a data query, this value is used. If this property is not defined, then the glog.dataquery.eventBatchSize value is used.

Controls how many matching record a data query can support. If the matched count exceeds this maximum, the data query is canceled and an exception is written to the log. The default for glog.dataquery.maxCount is 100,000.