3 Data Processing Configuration

This section describes configuration for attribute searchability, date formats, and configuration for Spark. It also discusses how to add a SerDe JAR to the Data Processing workflows.

• Date format configuration
  The dateFormats.txt file provides a list of date formats supported by Data Processing workflows. This topic lists the defaults used in this file. You can add or remove a date format from this file if you use the formats supported by it.
• Spark configuration
  Data Processing uses a Spark configuration file, sparkContext.properties. This topic describes how Data Processing obtains the settings for this file and includes a sample of the file. It also describes options you can adjust in this file to tweak the amount of memory required to successfully complete a Data Processing workflow.
• Adding a SerDe JAR to DP workflows
  This topic describes the process of adding a custom Serializer-Deserializer (SerDe) to the Data Processing (DP) classpath.