Parallel Data Load for EIM tables Using bcp

Microsoft SQL Server allows data to be bulk copied into a single EIM table from multiple clients in parallel, using the bcp utility or BULK INSERT statement. Use the bcp utility or BULK INSERT statement when the following conditions are true:

  • SQL Server is running on a computer with more than one processor.

  • The data to be bulk copied into the EIM table can be partitioned into separate data files.

These recommendations can improve the performance of data load operations. Perform the following tasks, in the order in which they are presented, to bulk copy data into SQL Server in parallel:

  1. Set the database option truncate log on checkpoint to TRUE using sp_dboption.

  2. Set the database option select into/bulkcopy to TRUE using sp_dboption.

    In a logged bulk copy all row insertions are logged, which can generate many log records in a large bulk copy operation. These log records can be used to both roll forward and roll back the logged bulk copy operation.

    In a nonlogged bulk copy, only the allocations of new pages to hold the bulk copied rows are logged. This significantly reduces the amount of logging that is needed and speeds the bulk copy operation. Once you do a nonlogged operation, immediately back up so that transaction logging can be restarted.

  3. Make sure that the table does not have any indexes, or, if the table has an index, make sure that it is empty when the bulk copy starts.

  4. Make sure that you are not replicating the target table.

  5. Make sure that the TABLOCK hint is specified using bcp_control with eOption set to BCPHINTS.

    Note: Using ordered data and the ORDER hint will not affect performance because the clustered index is not present in the EIM table during the data load.
  6. After data has been bulk copied into a single EIM table from multiple clients, any clustered index on the table must be recreated using DBCC DBREINDEX.