Retrieving large volumes of data

Retrieving large volumes of data from Eloqua requires multiple requests using the offset and limit query parameters. This tutorial will walk you through how to retrieve data surpassing the limit of 50,000 per request, including example HTTP requests and cURL.

In this tutorial:

Understanding the offset and limit query parameters

The query parameters limit and offset are used to specify how many records to return, and which batch of records to return. When exporting large amounts of data surpassing 50,000, these query parameters are required to retrieve the next batch of records since the maximum amount of records you can retrieve at once within a single batch is 50,000.

Query parameter Description Default Maximum Example
limit A URL parameter that specifies the maximum number of records to return. This can be any positive integer between 1 and 50000 inclusive. If not specified, the default is 1000.

Note: Only the /data endpoints have a maximum limit of 50000. Refer to the documentation for specific endpoints to learn more about limitations.

1000 50000 ?limit=50000
offset Specifies an offset that allows you to retrieve the next batch of records. Any positive integer. For example, if your limit is 1000, specifying an offset of 1000 will return records 1000 through 2000. If not specified, the default is 0. 0 N/A ?offset=1000

Note the following:

  • It is best practice to export data using the syncs data endpoint.
  • By default, data is returned in JSON format unless an Accept header is added to accept text/csv format.

Example requests

For example, you have 150,000 records in your database. The maximum amount of records you can retrieve at once is 50,000, so you will need to make three separate calls to retrieve 50,000 records at a time. The following examples will walkthrough how to form these requests.

Note: The example requests below assume that our Eloqua pod is 3 and our sync id is 8.

Retrieving records 0 - 50000

To return records 0 to 50,000, we'll specify an offset of 0 and a limit of 50000.

HTTP Request

GET https://secure.p03.eloqua.com/api/bulk/2.0/syncs/8/data?limit=50000&offset=0
			

cURL request

curl --user "APITest\API.User" --request GET https://secure.p03.eloqua.com/api/bulk/2.0/syncs/8/data?limit=50000&offset=0
			

Response

{
  "totalResults": 150000,
  "limit": 50000,
  "offset": 0,
  "count": 50000,
  "hasMore": true,
  "items": [
    ...
  ],
  
}			

The response displays items 0 to 50,000.

Retrieving records 50,000 - 100,000

To return records 50,000 to 100,000, we'll specify an offset of 50000 and a limit of 50000.

HTTP request

GET https://secure.p03.eloqua.com/api/bulk/2.0/syncs/8/data?limit=50000&offset=50000
			

cURL request

curl --user "APITest\API.User" --request GET https://secure.p03.eloqua.com/api/bulk/2.0/syncs/8/data?limit=50000&offset=50000
			

Response

{
  "totalResults": 150000,
  "limit": 50000,
  "offset": 50000,
  "count": 50000,
  "hasMore": true,
  "items": [
    ...
  ],
  
}

The response displays items 50,000 to 100,000.

Retrieving records 100,000 - 150,000

To return records 100,000 to 150,000, we'll specify an offset of 100000 and a limit of 50000.

HTTP request

GET https://secure.p03.eloqua.com/api/bulk/2.0/syncs/8/data?limit=50000&offset=100000
			

cURL request

curl --user "APITest\API.User" --request GET https://secure.p03.eloqua.com/api/bulk/2.0/syncs/8/data?limit=50000&offset=100000
			

Response

{
  "totalResults": 150000,
  "limit": 50000,
  "offset": 100000,
  "count": 50000,
  "hasMore": false,
  "items": [
    ...
  ],
  
}

The response displays items 100,000 to 150,000.

Related

Export data from Eloqua

Sending API requests using cURL

cURL request formatting