[/map {"- map/map "}) [/map/title {"- topic/title "}) Working with Data Domains (title][/map/topicref {"- map/topicref "}) [/map/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicmeta/navtitle {"- topic/navtitle "}) Working with Data Domains (navtitle][/map/topicref/topicmeta/linktext {"- map/linktext "}) Working with Data Domains (linktext][/map/topicref/topicmeta/shortdesc {"- map/shortdesc "}) This section documents the creation, population, and maintenance of data domains. (shortdesc] (topicmeta][/map/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) About working with data domains (navtitle][/map/topicref/topicref/topicmeta/linktext {"- map/linktext "}) About working with data domains (linktext][/map/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) This topic describes general information about working with data domains. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Recommended data domain creation and configuration process (navtitle][/map/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Recommended data domain creation and configuration process (linktext][/map/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) This topic outlines the recommended process for creating, configuring, and loading a data domain. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) About data domain data (navtitle][/map/topicref/topicref/topicmeta/linktext {"- map/linktext "}) About data domain data (linktext][/map/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) This section describes the data loaded into the data domain. (shortdesc] (topicmeta][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Supported data types (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Supported data types (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) This topic lists the Integrator ETL native data types and specifies which of them are supported in the Endeca data domain's Dgraph process. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Supported languages (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Supported languages (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) Endeca data stores support upload of data in a number of languages. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Default values for new attributes (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Default values for new attributes (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) New standard and managed attributes created during an ingest are given a set of default values. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Creating mdexType Custom properties (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Creating mdexType Custom properties (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) Integrator ETL allows you to create an mdexType Custom property that you can use to explicitly specify the MDEX type to which a particular Endeca standard attribute should map. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Specifying multiple record delimiters (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Specifying multiple record delimiters (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) By using an OR operator, you can specifying multiple record delimiters in the metadata. (shortdesc] (topicmeta] (topicref] (topicref][/map/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Creating data domains (navtitle][/map/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Creating data domains (linktext][/map/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) This section describes options for creating a data domain. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Loading and maintaining data in a data domain (navtitle][/map/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Loading and maintaining data in a data domain (linktext][/map/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) Integrator ETL provides four components to load and maintain data in a data domain: Bulk Add/Replace, Merge Records, Modify Records, and Delete Records. (shortdesc] (topicmeta][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Determining which updates to run (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Determining which updates to run (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) This topic discusses at a high level which types of updates are typically run, which helps you decide which types of graphs you need to create in Integrator ETL for your update purposes. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Choosing a data loading strategy (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Choosing a data loading strategy (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) Strategies for loading data into Endeca Server fall into two general categories: baseline updates and incremental updates. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Choosing a loader (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Choosing a loader (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) Integrator ETL provides two loader components to support the loading of data into an Endeca data domain: The Bulk Add/Update Records component (often called the "Bulk Loader") and the Merge Records component. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Processing multi-value data (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Processing multi-value data (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) You can load multi-value data into a data domain by specifying the delimiter used to separate multiple values.But if you want to process multi-value data before loading it, convert it to a list data type (shortdesc] (topicmeta] (topicref] (topicref][/map/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Load graphs (navtitle][/map/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Load graphs (linktext][/map/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) Loading graphs share a similar implementation, whether they were developed for initial load of data or for a subsequent baseline.The Integrator ETL Sample Applications include a full load graph, named LoadData, that you can use as a model for building your own load graphs. (shortdesc] (topicmeta][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Source data format (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Source data format (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) Integrator ETL reader components can read a variety of formats, including delimited, JDBC, and XML. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Configuring a Reformat component to generate a primary key (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Configuring a Reformat component to generate a primary key (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) The typical practice when generating a primary key is to use CTL to concatenate the value of two or more standard attributes to generate the key value. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Configuring the Bulk Add/Replace Records component for initial load (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Configuring the Bulk Add/Replace Records component for initial load (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) This topic describes the configuration of the Bulk Add/Replace Records component when used in an initial load graph. (shortdesc] (topicmeta] (topicref] (topicref][/map/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Adding records and assignments (navtitle][/map/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Adding records and assignments (linktext][/map/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) Use the Merge Records component to add new records to the data domain, or to add new assignments to existing records. (shortdesc] (topicmeta][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Merge Records input (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Merge Records input (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) The input to the Merge Records component is an arbitrary array of property names and values that you want to use to select the records you want to modify, and the values you want to add to the associated properties in the file. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Merge Records graph (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Merge Records graph (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) A Merge Records graph usually consists of two components: a reader component and the Merge Records component. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Configuring the Merge Records component (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Configuring the Merge Records component (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) This topic describes how to configure the Merge Records component to load records and assignments. (shortdesc] (topicmeta] (topicref] (topicref][/map/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Modifying records (navtitle][/map/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Modifying records (linktext][/map/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) Use the Modify Records component to modify the assignments on a record.You can add new assignments, replace existing assignments, or delete assignments from the record. (shortdesc] (topicmeta][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Modify Records input (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Modify Records input (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) The input to the Modify Records component is an arbitrary array of property names and values that you want to use to select the records you want to modify, and the values you want to add to the associated properties in the file. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Modify Records graph (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Modify Records graph (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) A Modify Records graph usually consists of two components: a reader component and the Modify Records component. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Specifying the Record Set Specifier Attributes (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Specifying the Record Set Specifier Attributes (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) This topic describes how to specify the Record Set Specifier Attributes property on components that include it. (shortdesc] (topicmeta] (topicref] (topicref][/map/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Backing up and restoring data domains (navtitle][/map/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Backing up and restoring data domains (linktext][/map/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) Use operations in the Manage Web Service to back up and restore data domains. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Deleting data from the data domain (navtitle][/map/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Deleting data from the data domain (linktext][/map/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) You can delete all data from the data domain, delete complete records from the data domain, or delete specific assignments from a record. (shortdesc] (topicmeta][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Deleting records from the data domain (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Deleting records from the data domain (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) Use the Delete Records component to delete records from a data domain. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Delete Records graph (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Delete Records graph (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) A Delete Records graph can include either one or two components. (shortdesc] (topicmeta] (topicref][/map/topicref/topicref/topicref {"- map/topicref "}) [/map/topicref/topicref/topicref/topicmeta {"- map/topicmeta "}) [/map/topicref/topicref/topicref/topicmeta/navtitle {"- topic/navtitle "}) Configuring the Delete Records component (navtitle][/map/topicref/topicref/topicref/topicmeta/linktext {"- map/linktext "}) Configuring the Delete Records component (linktext][/map/topicref/topicref/topicref/topicmeta/shortdesc {"- map/shortdesc "}) This topic describes how to configure the Delete Records component to delete records from the data domain. (shortdesc] (topicmeta] (topicref] (topicref] (topicref] (map]