Parallel Data Load for EIM tables Using bcp
Microsoft SQL Server allows data to be bulk copied into a single
EIM table from multiple clients in parallel, using the bcp
utility or BULK INSERT
statement. Use the bcp
utility or BULK INSERT
statement when
the following conditions are true:
SQL Server is running on a computer with more than one processor.
The data to be bulk copied into the EIM table can be partitioned into separate data files.
These recommendations can improve the performance of data load operations. Perform the following tasks, in the order in which they are presented, to bulk copy data into SQL Server in parallel:
Set the database option truncate log on checkpoint to
TRUE
using sp_dboption.Set the database option select into/bulkcopy to
TRUE
using sp_dboption.In a logged bulk copy all row insertions are logged, which can generate many log records in a large bulk copy operation. These log records can be used to both roll forward and roll back the logged bulk copy operation.
In a nonlogged bulk copy, only the allocations of new pages to hold the bulk copied rows are logged. This significantly reduces the amount of logging that is needed and speeds the bulk copy operation. Once you do a nonlogged operation, immediately back up so that transaction logging can be restarted.
Make sure that the table does not have any indexes, or, if the table has an index, make sure that it is empty when the bulk copy starts.
Make sure that you are not replicating the target table.
Make sure that the
TABLOCK
hint is specified using bcp_control with eOption set toBCPHINTS
.Note: Using ordered data and theORDER
hint will not affect performance because the clustered index is not present in the EIM table during the data load.After data has been bulk copied into a single EIM table from multiple clients, any clustered index on the table must be recreated using
DBCC DBREINDEX
.