Designing Data Integrator Projects

Setting the Batch Size for Joined Tables

To increase performance during collaboration execution, you can configure the batch size for the temporary tables created for joined source tables. By tuning the batch size you can load data more efficiently into source tables.

By default, 5000 rows are populated at the same time into a source table. There is no upper limit to the batch size. The limit is determined by the amount of internal memory available on the machine running the collaboration. Generally, the lower the number the better, but adjust the value to determine the optimum performance.


Note –

The source table batch size only affects temporary source tables. To limit the number of rows fetched at a time, specify the batch size in the Properties panel for the target table.


ProcedureTo Set the Batch Size for Joined Tables

  1. Open the collaboration you want to edit.

  2. Right-click the source table to set the batch size for, and then select Properties.

    The Properties panel appears.

    Figure shows the Source Table – Properties window.
  3. In the Batch Size property (under the Expert heading), enter the number of rows to populate at the same time into the temporary source table.

  4. Click OK.