Large File Non-XML Sample

This sample presents a design pattern appropriate for application architectures that require manipulation of a large file into a series of non-XML records. In this pattern a File control reads a large XML file and then splits the file into individual records. The individual records are then inserted into a database.

Note: This sample is provided on dev2dev for your convenience and is not supported by BEA.

What the Sample Does

The sample consists of a single WebLogic Integration application containing one synchronous business process and associated artifacts, including a File Event Generator, File Control, and Schema. A File Event Generator is used to poll a directory for large XML Files. The File Event Generator is configured to pass by file name. A File control is then used to read the file record by record. The record is parsed using MFL. For the sample, a CSV file containing customer records is used. The customer records are inserted into the CUSTOMER table of the WLI Sample Domain.

The CSV file is in the following format:

     28,CUSTOMER,2315 N First St,San Jose,CA,95131,408,570-8884

The following data is inserted into the CUSTOMER table in the samples domain:

     CUSTID INT 10
     NAME VARCHAR 40
     ADDRESS VARCHAR 40
     CITY VARCHAR 40
     STATE CHAR 2
     ZIP VARCHAR 11
     AREA_CODE CHAR 3
     PHONE CHAR 9

How It Works

The sample provides an implementation that uses a Batch Insert DB Control. This control is built as a wrapper around the existing DB Control. The implementation uses the Batch Insert DB control within a loop bounded by a transaction block to limit the number of records processed before transaction timeout. This prevents JTA timeout when processing large files.

The following diagram illustrates the sample architecture:

Required Software

This sample requires the following software:

• WebLogic Integration 8.1, SP4 or later (includes Jython 2.1). You can download WebLogic Integration 8.1 from the download page on dev2dev.
• WLI Common Utilities, version 2.5. You can download WLI Common Utilities from the download page.
• ebLogic Server Scripting Tool (WLST) (included with WLI Common Utilities). You can also download WLST from the download page.
•  
• Note: WLST Offline and WLST Online are available for download and evaluation from BEA's dev2dev site, but have not been formally included in the WebLogic Platform 8.1 product. WLST is supported through BEA newsgroups only, and the utility and APIs are subject to change. BEA intends to formally support this capability in a future release of WebLogic Platform.

Supporting Materials

In addition to providing application code, the sample includes the following items:

• Instructions and scripts for configuring a BEA WebLogic Integration cluster to support the application.
• Instructions and scripts for deploying the application to an appropriately configured cluster.
• Instructions, guidelines, and caveats for designing processes that will process large files.

How to Download

You can download the Large File Non-XML sample and WLI Common Utilities from the download page on dev2dev.