Creating an application data set from an uploaded file

The file upload option for creating an application data set allows you to upload data from an Excel or JSON file.

Before uploading data from a file, make sure to clean up the data to remove:
  • Tabs
  • Line breaks
  • Carriage returns

If these are present in an attribute value, then users may not be able to use that value for refinement.

You should also remove any unnecessary spaces.

For a data set created from an uploaded file, Studio always generates an attribute called dataSetName_eid-reserved-spec, which acts as the unique identifier for each record.

After selecting the file upload option, to create a data set from an uploaded file:

  1. Click the radio button to indicate whether you want to upload an Excel spreadsheet or a JSON file.
    Selecting the file type for a file upload data set
  2. To search for and select the file to upload, click the Browse button.

    Note that Excel spreadsheets must have been created using Excel 2000 or later, and must have the .xls or .xlsx file extension.

    JSON files must have the .json file extension, and must use UTF-8 encoding.

    Studio processes the file, and displays a subset of the data.

    For Excel spreadsheets, Studio displays data from the first worksheet. The columns may not be in exactly the same order as they are in the file.

    For JSON files, when Studio parses the file, it also identifies the available arrays in the file. The list of arrays is used to populate the Get records from drop-down list for that file. In the data set, Studio creates a record for each item in the selected array.

    Also note that date/time values are displayed using the default Studio date/time format, which does not include the time. However, the full date/time value will be included in the data set.

    If an attribute is not present in the first 1000 records, then it is not displayed on the attribute configuration page, and is not added to the data set.

    If the JSON file is malformed, then Studio parses as much of the file as it can.

  3. For an Excel file:
    Preview of data for a selected Excel file
    1. If there are multiple worksheets, then from the Select the sheet to use drop-down list, select the worksheet to use for the upload.
    2. If the data has a header row, check the My data includes header row checkbox.
    3. Click Next.

      The next page, used to define the data set attributes, is displayed.

  4. For a JSON file:
    Preview of data from a JSON file
    1. From the Get records from drop-down list, select the array to use to create the data set.

      Studio creates a record for each item in the selected array.

      To create the data set from the root level of the JSON file, select Document root.

    2. Click Next.

      The next page, used to define the data set attributes, is displayed.

  5. On the next page, under Define Data Set:
    Add Data Set page with automatic refinement rules setting
    1. In the Data set name field, type the name of the data set.

      By default, the data set name is the file name without the file extension.

    2. From the Default Search Language drop-down list, select the language to use to index the data for search.

      You can select from any of the languages supported by the Endeca Server.

    3. When adding an additional data set to an existing application, to automatically create refinement rules between the two data sets, check the Automatically create refinement rules checkbox.

      If you enable the automatic refinement rules, then Studio creates rules to link attributes that have the same name, multi-value setting, data type, and refinement behavior.

    4. Configure the attributes for the data set. See Reviewing and revising the attributes for a new data set.
  6. After configuring the attributes, click Done.

    Studio creates the data set and displays the application.

    While Studio is creating the data set, the Endeca Server domain is temporarily set to be read-only. Once the data set is complete, it becomes writeable again.

    When you add a new data set to an existing application, Studio adds a new page to the application. The page is named for the data set.

    Note that if there is a problem creating the data set, the new page may still be created. You can either update the components to link to views from other data sets, or remove the page.