Process Large Data Sets Asynchronously with Different Bulk Import Operations

The Marketo Bulk API enables you to handle huge data sets asynchronously with different bulk operations. For every bulk operation, the Marketo application creates a job. The job contains a nonempty CSV file that is 9 MB or less in size. Because the jobs are processed in parallel, no execution order is followed.

This use case describes how to configure the Marketo Adapter to import a large volume of program member records into Marketo. Similarly, you can import a large volume of records for leads and custom objects. To perform this operation, you create FTP Adapter and Marketo Adapter connections in Oracle Integration. In this use case, a CSV file is used as input.

1. Create a scheduled orchestration integration.
2. Drag an FTP Adapter into the integration canvas.
3. Configure the FTP Adapter as follows:
   1. On the Operations page, select Download File from the Select Operation list.
   2. Select ASCII from the Select a Transfer Mode list.
   3. Provide the input directory, file name, and download directory.
   4. Review your selections on the Summary page.
4. Drag a stage file action into the integration canvas below the FTP Adapter and configure it to read the downloaded file. The stage file action helps the Marketo Adapter fetch data in the form of segments (a single segment contains 200 records).
   1. On the Basic Info page, provide a name.
   2. On the Configure Operation page, select Read Files in Segments from the Choose Stage File Operation field.
   3. Specify the XPath expression for the file name in the Specify the File Name field.
   4. Specify the directory name in the Specify the Directory to read from field.
   5. On the Scheme Options page, select Create a new schema from a CSV file from the Do you want to create a new schema or select an existing one list.
   6. On the Format Definition page, click Choose File and upload the sample CSV file in the Select a New Delimited Data File field.
   7. Review your selections on the Summary page.
5. Drag a Marketo Adapter inside the stage file action and configure it with the required bulk operation.
   1. On the Basic Info page, provide a name.
   2. On the Action page, select Import Bulk Data.
   3. On the Operations page, select the Program Members object, and then select the Import Program Member operation.
   4. Review your selections on the Summary page.
6. In the mapper, map the stage file action elements to the Marketo input payload.
   
   
   Description of the illustration marketo_mapper.png
   
   
7. Drag a Marketo Adapter below the stage file action and configure it with the Final Submit operation.
   1. On the Basic Info page, provide a name.
   2. On the Action page, select Import Bulk Data.
   3. On the Operations page, select the Final Submit operation in the Select Operation field.

      Note:

      You cannot select the object on which to perform the operation because this Marketo Adapter connection performs the activity for the operation that was configured for the Marketo endpoint inside the stage file action.
   4. Review your selections on the Summary page.
8. Drag a for-each action into the canvas, select batchId as the repeating element to get from the Final Submit response, provide a name, and provide a current element name.
9. Drag a Marketo Adapter to configure the Import Program Members Status operation inside the for-each action.
   1. On the Basic Info page, provide a name.
   2. On the Action page, select Import Bulk Data.
   3. On the Operations page, select the Program Members object, and then select the Import Program Member Status operation.
   4. Review your selections on the Summary page.
10. In the mapper, map the current element name batchId to the Program Member Status request.
11. Drag an FTP Adapter connection below the Marketo Adapter and configure the FTP endpoint to write the Marketo Program Member Status response to a file for future use.
    1. On the Basic Info page, provide a name.
    2. On the Operations page, select Write File from the Select Operation list.
    3. Select ASCII from the Select a Transfer Mode list.
    4. Specify the output directory and file name pattern.
    5. On the Scheme page, select Sample delimited document (CSV) from the Which one of the following choices would be used to describe the structure of the file contents list.
    6. On the File Contents-Definition page, click Choose File and upload the schema file in the Select a New Delimited Data File field.
    7. Review your selections on the Summary page.
12. In the mapper, map the Marketo Program Member Status response to the FTP write request.
13. Specify the tracking variable.
    1. Click Tracking.
    2. Drag the required variable to use for the tracking reference (for this example, startTime is selected).
14. Activate the integration.
15. After successful activation, you can submit the integration and monitor the runtime in Oracle Integration.