16Bulk Data Export

This chapter contains the following:

You can extract large volumes of data using bulk export. You can either extract a full set of records for an object, or perform incremental extracts. For example, you can extract complete set of account data or extract updated set of records every week. Bulk export creates comma separated or tab delimited files, which are attached to the export process.

Bulk data export process flow

You can extract large volumes of data using bulk export. You can either extract full set or records for an object, or perform incremental extracts. Bulk export creates comma separated or tab delimited files, which are attached to the export process after exporting the data.

To perform bulk data export:

  1. Create the export process definition. The export process definition is made up of the export map and the processing schedule.

  2. In the export map of the export process definition:

    1. Select attributes for the export

    2. Filter data for the export

  3. Schedule the export job.

  4. Activate the export job.

Define Bulk Export

After defining the mapping files, you create an export process to export an object's data using the mapping files created.

To define a bulk export process:

  1. Navigate to the following in the Setup and Maintenance work area:

    • Offerings: Sales

    • Functional Area: Data Import and Export

    • Task: Schedule Export Processes

  2. Select Create from the Actions menu to view the Create Export Process Definition: Enter Basic Information page.

  3. Enter basic information about the export process, such as name and export map.

  4. Click Next to view the Create Export Process Definition: Configure Export Objects page.

    In this page, you define the export object you want to export and the mapping you want use for the export process.

  5. Optionally, you can enter a name for the mapping if you would like to reuse this mapping for other export processes. An object's mapping file determines what data is exported every time an object is exported. You can have multiple mapping files for an export object for various object details you want to export.

    Note: The export mapping is optional. You don't need to select an existing mapping to perform an export.
  6. Select Create from the Actions menu to view the Manage Export Objects dialog box.

  7. Select objects from the Available Objects list and move them to the Selected Objects list and click Done.

  8. Define the attributes you want to export in the Detail region of the page. You can define:

    • Attributes you want to export. Select Enabled for the attributes you want to export.

    • Header text of the attributes. Edit the header text in the Header text column. This value is used as the column header in the files generated by the export process

  9. Define the filters to determine the data you want to export by clicking the button in the Edit Filter Criteria column.

    Note: For the incremental extracts, you can create filters using time stamps to determine which rows to export.
  10. Click Next to view Create Export Process Definition: Create Schedule page.

  11. Select if you want to run the export process immediately or at a later time. Two types of scheduled exports are supported:

    1. Incremental Export

    2. Normal Export

  12. Click Next to view the Create Export Process Definition: Review page.

  13. Review the export process details and click Activate.

After each export process executes and completes, a comma or tab delimited data file is created and stored as an attachment to the data export process. If the number of records matching the view criteria exceeds one million records, or if the generated export file size exceeds 100 MB, then the data export process generates multiple files. The generated export files are numbered using the extension "_NNN" starting at "_001".

You can also use web services to schedule and start an export process. In this case, you can use the getAttachment web service to download the data file.

You use bulk data export to export data, by leveraging export maps. To export files using bulk export, you create a process definition, map files for full and incremental extract processes, and then schedule the data export process.

Bulk Data Export Process Components

This image shows the bulk data export process components comprising the process name, export process ID, and the export map ID.

Bulk export process components

Bulk data export process includes these three components:

  • Process name: A name for the export process that lets you easily refer the export process rather than using the computer generated ID such as 100000019897192.

  • Export process ID: A unique, application generated identifier for the export process definition for the export process. The process ID ties the export map with its export objects, filters, and the export schedule.

  • Export map ID: A unique identifier for the export map. You can reuse the export map in different process definitions. For example, you create a process definition to export all the data from the Customer export object. You can then reuse that export map and apply a new filter on the data to create an incremental export, such as data accrued since the last export date.

FAQs for Manage Bulk Data Export

Review the requirements for the data to be exported and determine the source view objects that hold the attributes you want.

Full sets of data aren't always required for export. To create a subset of data, use filter criteria to determine the time frame or scope of data, based on values of the attributes. To find activities for a specific date range, for example 1/1/11 through 3/31/11, navigate to the Export Objects Detail Sub Page and click the filter icon. Fill in the filter criteria dialog for the project start dates to select the data to be exported. You run the export by navigating to the following in the Setup and Maintenance work area:

  • Offerings: Sales

  • Functional Area: Data Import and Export

  • Task: Schedule Export Processes

You can view the Schedule Export Processes, Overview page to see the History subpage. The column Exported Data File shows a hyperlink to your output file This file will be a comma separated variable or a tab delimited file. Click that link to open the file and see the exported data.

Changing the sequence number changes the order of the attributes in the exported data file. Changing the header text enables you to give a more intuitive meaning to the attribute and the associated data.

Select as many view objects as required for the export process. Select the individual attributes required from every export object.