Large File Non-XML Sample

This sample presents a design pattern appropriate for application architectures that require manipulation of a large file into a series of non-XML records. In this pattern a File control reads a large XML file and then splits the file into individual records. The individual records are then inserted into a database.

Note: This sample is available on dev2dev and is not supported by BEA.

About the Sample

The sample consists of a single WebLogic Integration application with one synchronous business process and associated artifacts, including a File Event Generator, File Control, and Schema. A File Event Generator is used to poll a directory for large XML Files. The File Event Generator is configured to pass by file name. A File control is then used to read the file record by record. The record is parsed using MFL. For the sample, a CSV file containing customer records is used. The customer records are inserted into the CUSTOMER table of the WLI Sample Domain.

The CSV file is in the following format:

     28,CUSTOMER,2315 N First St,San Jose,CA,95131,408,570-8884

The following data is inserted into the CUSTOMER table in the samples domain:

     CUSTID INT 10
     NAME VARCHAR 40
     ADDRESS VARCHAR 40
     CITY VARCHAR 40
     STATE CHAR 2
     ZIP VARCHAR 11
     AREA_CODE CHAR 3
     PHONE CHAR 9

Workflow of the Sample

The sample provides an implementation that uses a Batch Insert DB Control. This control is built as a wrapper around the existing DB Control. The implementation uses the Batch Insert DB control within a loop bounded by a transaction block to limit the number of records processed before transaction timeout. This prevents JTA timeout when processing large files.

The following diagram illustrates the sample architecture:

Required Software

This sample requires the following software:

• WebLogic Integration 9.2, MP2 or later (includes Jython 2.1). Download WebLogic Integration 9.2 from the download page on dev2dev.
• WLI Common Utilities, version 2.5. Download WLI Common Utilities from the download page.
• ebLogic Server Scripting Tool (WLST) (included with WLI Common Utilities). WLST can also be downloaded from the download page.
•  
• Note: WLST Offline and WLST Online are available for download and evaluation from BEA's dev2dev site, but have not been formally included in the WebLogic Platform 9.2 product. WLST is supported through BEA newsgroups only, and the utility and APIs are subject to change. BEA intends to formally support this capability in a future release of WebLogic Platform.

Supporting Materials

In addition to providing application code, the sample includes the following items:

• Instructions and scripts for configuring a BEA WebLogic Integration cluster to support the application.
• Instructions and scripts for deploying the application to an appropriately configured cluster.
• Instructions, guidelines, and caveats for designing processes that will process large files.

Download

You can download the Large File Non-XML sample and WLI Common Utilities from the sample download page on dev2dev.