Exporting large numbers of records

Endeca Server does not have a dedicated facility for the bulk export of records. Instead, you can pin a version of the index, and request a large number of records from this version, with paging through the results. The records are returned in an XML format and thus require parsing as a post-processing step.

The following procedure implies that you use the Endeca Server only, but do not use Studio. This procedure relies on direct methods within the Endeca Server. If you have installed Studio, you can use its facility to export large numbers of records.

To export records in bulk from the Endeca Server with the Conversation Web Service:

  1. Pin a data version, using a request with PinDataVersion element, specifying an optional_pin_timeout. For details, see Holding on to a data version.
    For example: 
    ...
<ns:Request>
 <!--Optional:-->
  <ns:PinDataVersion>optional_pin_timeout</ns:PinDataVersion>
<ns:State>
  ....
</ns:State>
    The request should return the version number in its header: X-Endeca-Served-Data-Version. This is the version number that you pinned.
  2. Issue a subsequent request that references this version number in the DataVersionRequested element. This request should also include a RecordListConfig type. For details, see Retrieving large numbers of records.

    If the request is successful, it returns the requested records.

It is up to you, as the front-end application developer, to determine what to do with the retrieved records. For example, you can display each record’s attribute values. You can also write code to properly format the returned attribute values for export to an external file, such as a Microsoft Excel spreadsheet, or a CSV file. To help with performance of parsing received records, use a SAX parser. See, for example, Parsing an XML file using SAX.