Bulk Plan Multi-threading

OTM provides two key ways to take advantage of multiple CPUs when running bulk plans. You can run several bulk-plans simultaneously by setting up multiple batch threads or by bulk plan partitioning. Bulk plan partitioning is a way to partition the order releases or order movements through pre-defined saved queries, and then run each partition in its own thread. In order to understand and effectively use multi-threading, it is important to understand how OTM’s multi-threading logic works. 

In addition to bulk plan partitioning, you can configure OTM to multi-thread several time consuming areas within the bulk plan logic. The following areas in bulk-plan support multi-threading:

• Shipment Building of Order Releases and Order Movements
• Multi-stop
• Committing Shipments
• Driver and Equipment Assignment Actions
• Continuous Moves

Running Multiple Bulk Plans Simultaneously

Setting Up Properties for Multi-threading

The threading for bulk plan is set up via thread group settings. The thread group name for the bulk plan is "batch" and the number of threads for this group is set as follows: glog.workflow.queueThreads.batch=2.

By default, the batch thread group is assigned 2 threads. Depending on the number of processors and the load on the system, you can increase this to more than 2. The group thread counts should be set before starting the server. 

You can also change the batch threads through the Event Queues page. However, the recommended approach is to set the group thread counts before starting the server. If the threads are changed through the Event Queues page, the changes will be lost once the server is brought down. The Event Queues page can be accessed via Configuration and Administration > Technical Support > Diagnostics and Tools > Event Management > Event Queues. You must be logged into OTM has DBA.ADMIN to access the Event Queues page.

Once on the screen, click on Show Threads button at the bottom to see all the threads. To increase the number of threads, click on Threads. To kill a thread (if it hangs up), click on the thread to terminate. Remember that killing a thread will not terminate any existing bulk plans. No future bulk plans will be scheduled to run on that thread though.

Running Bulk Plans

Once the threads are set up, you can run multiple bulk plans simultaneously. If there are more bulk plans than the number of threads, then the bulk plans will be queued up and will start executing as soon as the threads get freed up. 

Threads Inside Bulk Plans

While the entire set of bulk plans can be run in parallel, you can also set up threads for processing key functions inside a bulk plan in parallel. These functions are described in the table below. Similar to the bulk plan process thread group called "batch", the areas within the bulk plans are assigned many functional thread groups. These areas are also called task groups. The table below describes the default thread count settings for these task groups. These properties can be found in glog.task properties file and can be overridden in glog.properties file.

Functional Area (Task Type)	Description	Default Thread Group
Build Shipments (BuildShipments)	For every order bundle on a single leg, OTM plans them onto a shipment using the shipment building logic. Several bundles can be simultaneously built into shipments.	planningBuild
Build Shipment Graphs (BuildShipmentGraphs)	This is the logic for building the shipments on various legs of an itinerary.	planningBuild
Build Shipment Graph Collections (BuildShipmentGraphCollections)	This logic is used for building several shipment graphs for an order bundle based on order releases.	planningBuild
Build Shipment Graphs for Order Movements (BuildShipmentGraphsForOrderMovements)	The shipment building logic for building shipments for a bundle of order movements.	planningBuild
Commit Shipment Graphs (CommitShipmentGraphs)	This logic commits shipments to the database at the end of order release bulk plan.	planningCommit
Commit Shipment Graphs for Order Movements (CommitShipmentGraphsForOrderMovements)	This logic commits shipments to the database at the end of order movement bulk plan.	planningCommit
Compute Transit Time (ComputeTransitTime)	Compute transit time happens as part of Route Orders process of bulk plan that uses Network Routing	planningBuild
Delete Shipment Graph (DeleteShipmentGraph)	During unassign stage in bulk plan order releases, if the shipments are to be deleted instead of to be updated, the deleting process can be configured in multi-thread	planningCommit
Fleet Assignment (FleetAssignment)	Driver and Equipment assignment actions.	planningBuild
NR Network Rater (NRNetworkRater)	This logic computes rating information for itinerary and network legs inside the Network Routing logic.	planningBuild
SNRO Build Shipment for Order Movements (SNROBuildShipmentForOrderMovements)	This logic computes shipment information during the Show Network Routing Options - Ocean action.	planningBuild
Savings Algorithm (SavingsAlgorithm)	Inside the multi-stop algorithm, pairs of shipments are evaluated for combining, determining the potential savings. These tasks can be run in parallel to achieve potential savings.	planningBuild
Shortest Path (ShortestPath)	The shortest path logic inside Dispatch Plan Optimization (DPO) algorithm.	planningOptimization

 

Although several task groups inside the bulk plan use the same thread group, you can override the default and create your own thread group. For example, you can create a thread group for Fleet Assignment for the driver and equipment assignment actions, so that these actions do not contend with for the same threads that shipment building logic uses.

Definition of Terms

In order to understand how multi-threading works and how to set up the threads and their properties, it is important to know certain terms.

Tasks and Task Types

A task is an atomic unit of work. A task runs on a single thread. It cannot be distributed across threads. Examples of tasks in OTM are building a shipment for one order release bundle, building a shipment for one order movement bundle, evaluating the savings of combining two shipments, committing a shipment graph, etc. Each task is characterized by its Task Type – such as BuildShipment, CommitShipments, etc.

Task Batch and Batch Size

A task batch is a collection of similar tasks that are processed sequentially on a single thread. For example, a task batch can be comprised of one hundred order bundles (similar tasks). Each of these hundred order bundles are processed one at a time sequentially on a single thread to build shipments. The number of tasks within a task batch represents the batch size of the task batch.

Thread Group

A thread group is a collection of threads. In the table above, planningBuild is a thread group. A thread group contains a certain number of threads available for that group. All the tasks in a task batch are processed by a thread within a thread group sequentially. The task batches assigned to different threads in a thread group are run in parallel. Each thread group is allocated a certain number of threads as follows:

glog.workflow.queueThreads.planningBuild=16

Please note that setting the multi-valued thread group setting glog.workflow.topicGroup=<queue>,<threadCount> has been deprecated.  It has been replaced with glog.workflow.queueThreads.<queue>=<threadCount>.

In order to change the topic group assignment of PlanningBuild to 10, you can override the property in the CUSTOM property set as follows:

glog.workflow.queueThreads.planningBuild=10

Example

In the above example, the thread group “planningBuild” has 10 threads. When a task batch arrives at the thread group for processing, it will start processing the batch immediately using a free thread. If all the threads are busy, then the batch waits until one of the threads is freed up. It is important to note that the thread groups are common across all bulk plans. For example, if there are 4 bulk plans that are running simultaneously, and there are only 2 threads defined for planningCommit thread group, then only 2 shipment graphs can be committed at a given time across various bulk plans. It is highly recommended that the number of threads in a functional thread group be set higher than the Bulk Plan thread group.

Multi-thread Properties Associated to Each Task Type

Property	Definition	Default Value
glog.workflow.task.blockTimeout.<task type>	This is the timeout, in seconds, for a task list execution. Example: glog.workflow.task.blockTimeout.BuildShipment =300 Optional. The amount of the time that each batch processing is allowed before it times out.
glog.workflow.task.desiredBatchSize.<task type>	Number of tasks per batch | QUEUE Example: glog.workflow.task.desiredBatchSize.BuildShipment= QUEUE Optional. The desired number of tasks per batch. If QUEUE, the number will be determined by balancing the workload over the event queue threads. For example, 1000 tasks using a queue with 5 threads would result in 5 batches with 200 tasks each. Note that QUEUE presumes you have a dedicated queue for the task type. If, on the other hand, the desired batch size is 300, then 4 batches will be created with 300 tasks in the first 3 batches and 100 tasks in the 4th batch.	QUEUE
glog.workflow.task.flags.<task type>	Allowed flags are: WAIT_ON_LIFETIME. If set, the TaskList.execute method blocks on all activity related to the batches, including agents and notification. Otherwise, the method blocks only on the execution of the tasks. ABORT_ON_TIMEOUT. If set, any timeout in the TaskList.execute method removes any related batches from the event queues. Active batches are not interrupted. This flag results in indeterministic behavior and should only be used to reduce unnecessary work on the application server. LOG_BATCHES
glog.workflow.task.group.<task type>	The task type equals the event queue. Example: glog.workflow.task.group.BuildShipment=planningBuild glog.workflow.task.group.CommitShipment=planningCommit glog.workflow.task.group.Rating The thread group assigned to the task type. Configuring the “rating” queue with multiple threads will allocate same number of threads for evaluating multiple rate geos instead of single thread processing all rate geos.	Required – no defaults associated
glog.workflow.task.minimumBatchSize.<task type>	Minimum number of tasks per batch. Example: glog.workflow.task.minimumBatchSize.BuildShipment = 1 Optional. Batches can have overhead due to shared resources and workflow processing. This property allows you to assign a minimum number of tasks in any batch. If the total # of tasks is less than the minimum, OTM creates a single batch to process the tasks.
glog.workflow.task.parallelThreshold.<task type>	Minimum number of batches before allowing parallel execution. Example: glog.workflow.task.parallelThreshold.BuildShipment =2 Optional. If the total number of batches is less than the threshold, all tasks are processed on the calling thread (or single threaded). Otherwise, the batches are processed in parallel using the treads in the topic group.	2