Resume Reading Feature

The purpose of this feature is to allow an application to read large files in parts instead of processing the whole file at once. Resume Reading allows your system to read files in a number of subsequent Business Rule executions, when you are using data streaming.

General Operation

The Resume Reading feature’s operation is achieved by keeping persistent information about the current successful file read operation, breaking, then resuming the next read operation from that last stored break position. As a result, the current file is read in parts, and the beginning and end of each part is determined by a predefined break condition.

You determine the break condition through the definition of your Business Rules. Since the Resume Reading feature operates based on reading one part of a file at a time per Business Rule, these rules must determine the break. Each Business Rule executes reading a part of the file, breaks, then passes to the next rule, which reads the next part up to the break, and so on, until the entire file is read.

A break condition can be any type of stopping point you determine in your Collaboration Rules. For example, this condition could be a fixed number of records, a delimiter, or reaching a specific character string.

The Client node in the OTD has a read-only property (ResumeReadingInProgress node) indicating whether there is a resume-reading operation in progress. This node is for informational purposes only. Also, the Resume Reading feature is available in the data-streaming mode only.

The feature has no special operational requirements besides setting the adapter configuration option. The adapter configuration has an option to enable or disable this feature. This option is also accessible at run time.

If this feature is enabled, the adapter always checks first for a resume-reading operation in progress. If this feature is not in progress, the adapter determines the next file based on the adapter configuration settings.

Step-by-step Operation

The diagram below illustrates how the Resume Reading feature operates along with pre- and post-file-transfer commands. In this example, the Collaboration executes the same Business Rule four times. Each execution causes the Collaboration to read another section of the file. When the Collaboration reads the final records, it executes the Post-Transfer commands

Figure 8 Resume Reading Operation

In this example, the reading happens in the following steps:

1. The adapter starts reading the file then reaches a break condition after a partial data read (the end of Part 1), the adapter’s pre-transfer commands have already been executed. The resume-reading state is stored, and no post-transfer commands are executed. The adapter is waiting for the next execution of the Business Rule.

2. The resume-reading operation is in progress but still attains only partial data reads. The adapter reads from one break condition to the next (Part 2 and Part 3 in the figure) The resume-reading state is stored in each case, and the adapter executes the Business Rule once per each part.

3. The resume-reading operation is in progress and completes its data read during the final execution of the Business Rule (Part 4). The adapter reads from a break condition to the end of a file. No resume-reading state is stored, and any post-transfer commands are then executed.

In all of the previous steps, the Business Rule is executed repeatedly, and the current read position in the file changes on each execution. If the file is smaller than Part 1 in the figure, the adapter does not reach a break condition. The operation is normal, and no resume-reading state is stored. The pre- and post-transfer commands are executed.

Operation Without Resume Reading Enabled

If the Resume Reading feature is not enabled:

• Data-read Stop Then Restart: Any unread data at the end of the file is ignored.

• Resume Reading in Progress: If there is a resume-reading operation in progress from a previous execution, an error is generated, and an exception is thrown.

  ---

  Note –

  If there is a resume-reading operation in progress it cannot be interrupted and must be completed. The executeBusinessRules() method must be called enough times to fully consume the file. In other words, do not discontinue processing the file before it has been completely consumed.

  ---

To Avoid Storing a Resume Reading State

Sometimes a partial data-stream read is necessary even when the Resume Reading feature is enabled. For example, there could be some application logic on top of the record parsers, which might abandon the rest of the file because of a corrupted record and close the file successfully after reading only part of the file’s content.

In this case, you must set the LocalFileOTD.Configuration.ResumeReading node to False before calling finish(). This setting tells the BatchLocalFile OTD to complete the operation without storing a resume-reading state. You can set up the Collaboration Rule to then send notifications or take other measures, as desired.