BatchRecord OTD

BatchRecord, the Batch Adapter’s record-processing OTD, allows you to parse (extract) records from an incoming payload (payload data) or to create an outgoing payload consisting of records. Understanding the operation of this OTD and how to use it requires an explanation of some of these terms.

The word payload here refers to an in-memory buffer, that is, a sequence of bytes or a stream. Also, records in this context are not records in the database sense. Instead, a record simply means a sequence of bytes with a known and simple structure, for example, fixed-length or delimited records.

For example, each of the following record types can be parsed or created by this OTD:

• A large data file that contains a number of SAP IDocs, each with 1024 bytes in length.

• A data file that contains a large number of X12 purchase orders, each terminated by a special sequence of bytes.

The record-processing OTD can handle records in the following formats:

• Fixed length: Each record in the payload is exactly the same size.

• Delimited: Each record is followed by a specific sequence of bytes, for example, CR,LF.

• Single: The entire payload is the record.

When using character delimiters with DBCS data, use single byte character(s) or equivalent hex values with hex values that do not coincide with either byte of the double byte character.

BatchRecord OTD Structure

The BatchRecord OTD contains two top-level nodes, Client, Configuration, PersistentState, and StateManager. Expand these nodes to reveal additional sub-nodes.

Figure 9 BatchRecord OTD Structure

OTD Structure and Operation

Each field node under the Configuration node of the OTD, corresponds to one of the adapter’s record-processing configuration parameters.

Record-processing OTD Node Functions

The following list explains these primary nodes in the record-processing OTD, including their functions:

• BatchRecord: Represents the OTD’s root node.

• Configuration: Each sub-node within this node corresponds to an adapter configuration parameter and contains the corresponding settings information, except for the Parse or Create parameter. See BatchRecord Connectivity Map Properties for details.

  • InputStreamAdapter and OutputStreamAdapter: Allow you to use and control the data-streaming features of the OTD. For details on their operation, see Streaming Data Between Components.

    ---

    Note - You can transfer data using the Payload node or by using data streaming (InputStreamAdapter and OutputStreamAdapter nodes), but you cannot use both methods in the same OTD.

    ---

    ---

    Note - For the record-processing OTD, these configuration nodes are read-only. They are provided only for the purpose of accessing and checking the configuration information at run time.

    ---

• Record: A properties node that represents either:

  • The current record just retrieved via the get() method, if the call succeeded

  • The current record to be added to the data payload when put() is called

• Payload: The in-memory buffer containing the data payload byte array you are parsing or creating.

  ---

  Caution - It is a good practice to use a byte array in all cases. Failure to do so can cause loss of data.

  ---

  • put(): Adds whatever is currently in the Record node to the data payload. The method returns true if the call is successful.

  • get(): Retrieves the next record from the data payload (or stream), and populates the Record node with the record retrieved. get() returns true if the call is successful.

  • finish(): Allows you to indicate a successful completion of either a parse or create loop for both put() and get().

    ---

    Note - Use reset() to indicate any errors and to allow the OTD to clean up any unneeded internal data structures.

    ---

Using the Record-processing OTD

This OTD has the following basic uses:

• Parsing a payload: When the payload comes from an external system

• Creating a payload: Before sending the payload to an external system

A single instance of the OTD is not designed to be used for both purposes at the same time in the same Collaboration. To enforce this restriction, there is a setting under the adapter’s General Settings parameters called Parse or Create Mode, for which you can select either Parse or Create.

Using get() and put()

The get() and put() methods are the heart of the OTD’s functionality. If you call either method, the record retrieved or added is assumed to be of the type specified in the adapter configuration, for example, fixed-length or delimited.

The get() method can throw an exception, but generally this action only happens when there is a severe failure. One such failure is an attempt to call get() before the payload data (or stream if you are streaming) has been set. However, the best practice is to code the Collaboration to check the return value from a get() call. A return of true means a successful get operation; a false means the opposite.

Choosing the Parse or Create Mode

The adapter checks to ensure that the proper calls are made according to your mode setting. For example, calling put() in a parse-mode environment would cause the adapter to throw an exception with an appropriate error message explaining why. Calling get() in the create mode would also result in an error.

The adapter requires these restrictions because:

• If you are processing an inbound payload, you are calling get() to extract records from the payload (parsing). In this situation it makes little sense to call put(). Doing so at this point would alter the payload while you are trying to extract records from it. Calling put() would overwrite the payload and destroy the data you are trying to obtain.

• Conversely, when you are creating a payload by calling put(), you have no need to extract or parse data at this point. Therefore, you cannot call get().

As a result, you can place the OTD on the source or destination side of a given Collaboration, as desired, and use the OTD for either parsing or creating a payload. However, you cannot parse and create at the same time. Implement your OTD in a Collaboration using the Java CAPS Collaboration Rules Editor.

Creating a Payload

When you want the payload data sent to an external system, you can place the OTD on the outbound side of the Collaboration interfacing with that system. Successive calls to put() build up the payload data in the format defined in the adapter configuration.

Once all the records have been added to the payload, you can drag and drop the payload onto the node or nodes that represent the Collaboration’s outbound destination. Also, you can set an output stream as the payload’s destination.

When you are building a data payload, you must take into account the type and format of the data you are sending. The adapter allows you to use the following formats:

• Single Record: This type of payload represents a single record to be sent. Each successive call to put() has the effect of growing the payload by the size of the data being put, and the payload is one contiguous stream of bytes.

• Fixed-size Records: This type of payload is made up of records, with each being exactly the same size. An attempt to put() a record that is not of the size specified causes an exception to be thrown.

• Delimited Records: This type of payload is made up of records that have a delimiter at the end. Each record can be a different size. Do not add any delimiters to this data type when it is passed to put(). The delimiters are added automatically by the adapter.

• User Defined: In this type of payload, the semantics are fully controlled by your own implementation.

Parsing a Payload

To represent payload data inbound from an external system, you map the data to the payload node in the OTD (from the Collaboration Rules Editor). In addition, you can specify an input stream as a source.

Either way, each successive call to get() extracts the next record from the payload. The type of record extracted depends on the parameters you set in the adapter’s configuration, for example, fixed size or delimited.

You must design the parsing Collaboration with instructions on what to do with each record extracted. Normally, the record can be sent to another Collaboration where a custom OTD describes the record format and carries on further processing.

Fully Consuming a Payload

It is possible to fully consume a payload. That is, after a number of successive calls to get(), you can retrieve all the records in the payload. After this point, successive calls to get() return the Boolean false. You must design the business rules in the subject Collaboration to take this possibility into account.

Using Record Processing with Data Streaming

If you are using the record-processing OTD with data streaming, you must be careful not to overwrite the output files. If the OTD is continually streaming to a BatchLocalFile OTD that uses the same output file name, the OTD can write over files on the output side.

To avoid this problem, you must use either file sequence numbering or change the output file names in the Collaboration Rules. Sequence numbering allows the BatchLocalFile OTD to distinguish individual files by adding a sequence number to them. If you use target file names, post-transfer file names, or both, you can change the name of the output file to a different file name.

For more information on how to use these features, see Sequence Numbering and Pre/Post File Transfer Commands.