Sun Data Integrator Development Phase

The development phase consists of standard tasks for specifying source and target databases and advanced tasks for further customizing the data transformation logic.

Standard Development Tasks

The following steps outline the basic procedure for developing an ETL process using Sun Data Integrator.

1. Connect to the source and target databases from the Services window in NetBeans.

2. Create a new Data Integrator Module project in NetBeans.

3. Using the Data Integrator Wizard, specify the source database and tables and the target database and tables.

4. Using either the Data Integrator Wizard or the ETL Collaboration Editor, specify join conditions and map the source fields or columns to the target fields or columns.

5. Specify the execution strategy.

6. Add the ETL service to a composite application.

7. Build and deploy the composite application.

Advanced Development Tasks

You can perform additional tasks during the development phase to customize your ETL application further.

• Customized Data Transformation – Use data transformation operators to define advanced standardization and cleansing rules for the source data.

• Master Index Staging – Create a staging database populated with the initial bulk data that will be loaded into a new master index application. The staging database is used by the master index Data Cleanser and Data Profiler prior to loading the data.

• ETL Process Integration - Call an ETL collaboration from a BPEL business process, web service, Java client, or other application.

• Extraction Scheduling - Configure a time or event that triggers a data extraction from the data source. You can extract data in batch mode or continuously based on database triggers.

• Parallel Processing - Configure the ETL process to run on multiple threads for better performance and faster execution.

Data Integrator Wizard

The Data Integrator Wizard takes you through each step of the ETL setup process and, based on the information you specify, creates a collaboration that defines the configuration of the ETL process.

Figure 8 Select Target Table on the Data Integrator Wizard

ETL Collaboration Editor

Once you define the data integration framework using the wizard, you use the ETL Collaboration Editor to further customize its configuration.

Figure 9 ETL Collaboration Editor