Understanding Fulfillment Engine Processing Groups and Commit Levels
Processing groups and commit levels are used by the fulfillment engine to improve throughput of work being processed. When the processing group and commit level features are not being used, the fulfillment engine processes all demand lines in a single grouping. When using processing groups and commit levels, work is broken up into more manageable portions that can be tuned for overall processing performance improvements in different environments.
The fulfillment engine has these three main sections that must be clarified in order to understand how to use the processing group and commit level features:
Selection processing
This section of the fulfillment engine uses the selection criteria on the fulfillment request to select a grouping of demand lines for processing. Selected demand lines are laid into temporary tables in preparation for the validation and update sections of the process. If a single processing group is being used, then all demand lines selected into the temporary tables are passed down to the validation and update routines and processing continues for all work in a single grouping. If multiple processing groups are being used, then the first processing group is identified and passed down to the validation and update routines. When the first processing group is completed, the next processing group is sent downstream for processing. By varying the size of the processing groups, chunks of work can be sent through the process that best fit individual operating environments.
All demand lines in the fulfillment request's selection criteria are soft locked to prevent other processes from working with those demand lines. The term soft locked is used when a program updates the lock table with the demand line. Other processes recognize that the demand line is locked and will not attempt to do any processing on it until the process that set the soft lock is finished.
The fulfillment engine has an added feature that enables you to set the soft lock for only those items in the first processing group. By doing so, you free up the demand lines that are not being processed in the first processing group until it is absolutely necessary to lock other users out. This feature is set on the Setup Fulfillment - Fulfillment Engine page. This feature would be used in higher activity environments where fulfillment engine processes are being performed at the same time as other online and batch processing that may want access to these demand lines.
Note: The option to release soft locks is provided for times when you are working with multiple processing groups, when there is overhead necessary to determine the priority of demand lines being processed in the first processing group. When the soft locks are released, then this work has to be redone for each processing group. The need to establish processing groups and to hold or release the soft locks is dependent on processing variations at individual sites.
Validation processing
A series of validations are performed on demand lines before they are reserved or shipped. Wherever possible, the fulfillment engine performs these validations using set processing to take advantage of system performance features inherent in the database manager. The set of demand lines that the fulfillment engine works on is the processing group. The content of the processing group can be controlled using the different options available when defining processing groups either on the Setup Fulfillment - Fulfillment Engine Options page or by overriding these values when the fulfillment request is created.
Update processing
Update processing is performed on demand lines that have been validated and prepared in the fulfillment engine's temporary tables. When performing update processing, different inventory system tables must be updated in sequence to prevent database contention issues. When working with the item balance tables, items are sequenced and updated in item ID order. If commit level controls are not being used, then all items are updated in a single commit level. This means that from the point in the process that the item is updated, it will be locked by the database manager. This prevents other users from updating that item until all processing is complete. Because many processes throughout the system use the item balance tables, single level commit processing can create database contention issues.
Database contention slows down processing and in some cases creates deadlocking situations that force the reprocessing of the transaction request. Commit levels can be established when using the fulfillment engine to reduce the amount of time that an item ID is locked. This can reduce contentious situations and improve overall performance. Carefully consider how commit levels are established, as overuse of commit processing can slow down overall processing. Actual options established for commit processing are dependent on volume and activity considerations in different processing environments.
When using processing groups and commit groups, these rules apply:
Use only request level processing groups on front-end shipping fulfillment requests.
Reservation fulfillment requests can use request, order, and line processing levels.
Shipping and front-end shipping fulfillment requests always use a single commit level for a processing group.
You would use the processing level of a request along with an entry in the processing count field to control the number of commits performed on a group of shipping or front-end shipping transactions.
Reservations fulfillment requests can use commit levels of request, order, line, or item.
The commit group is always equal to, or a subset of the processing group.
If the commit level is set at a level higher than the processing level, then the fulfillment engine adjusts the processing level to that of the commit level. For example, if the processing level is set at the line level but the commit level is set at the request level, then the fulfillment engine uses processing groups set at the request level using a processing count that's the same as that set up for the commit count.
When the pass through feature is used on reservations fulfillment requests, then the pass through level overrides the processing group and commit level, if either are set to a value less then the pass through level.
You can never break transaction-based fulfillment requests across multiple processing groups.
If a processing group reaches its maximum number of orders or demand lines before including all demand lines requested on a transaction-based fulfillment request, then the processing group expands to include all orders or demand lines requested in the selection criteria of the transaction.
You can break run control based fulfillment requests across multiple processing groups.
Fulfillment transaction requests in the staging tables are soft locked when they are first selected by the selection section of fulfillment engine processing.
Once the first processing group is determined, soft locks on those transactions not included in the first processing group are released if the Release Additional Locks option is set on the Setup Fulfillment - Fulfillment Engine page. By releasing these locks, the transactions can be picked up and processed by concurrent fulfillment engine processes.