1. Analytical Application Infrastructure User Guide
  2. Data Management Framework
  3. Data Mapping

Data Mapping

Data Mapping refers to the process of retrieving unstructured data from data sources for further data processing, storage, or migration. The intermediate extraction process can be followed by data transformation and metadata addition before exporting it to the staging area or to the Business Data Model.

Data movement can be from:

  • RDBMS source to RDBMS target (T2T)
  • RDBMS source to Flat File target(T2F)
  • RDBMS source to HDFS-Hive target (T2H)
  • HDFS-Hive source to RDBMS target(H2T)
  • HDFS-Hive source to HDFS target (H2H)
  • HDFS/Local-WebLog Source to HDFS Target (L2H)
  • HDFS-Hive source to Flat File target (H2F)
  • Flat File to RDBMS target (F2T)
  • Flat File present in Local File System (LFS) to HDFS target or HDFS file to HDFS target(F2H)

Note:

File present in the HDFS system cannot be loaded into RDBMS target Infodom.

F2T and F2H can be defined from the Data Mapping window. There is no separate Data File Mapping window.

Data movement between Hive and RDBMS can be enhanced using third-party tools like SQOOP and Oracle Loader for Hadoop (OLH). You must set parameters from the DMT Configurations window. For details, see the DMT Configurations section. For details on the configurations for SQOOP and OLH, see OFSAAI Administration Guide available in the OHC Documentation Library.

For the configurations required to support WebLog ingestion (L2H), see the Data Movement of WebLog Source to HDFS target section in the OFSAAI Administration Guide available in the OHC Documentation Library.

The roles mapped to Data Mapping are as follows:

  • DMACCESS
  • DMREAD
  • DMWRITE
  • DMPHANTOM
  • DMAUTH
  • DMADV

For all the roles, functions and descriptions, see Appendix A.

Figure 7-4 Data Mappings window


This image displays the Data Mappings window.

The Data Mappings window displays the list of pre-defined Data Mapping definitions with Record Status as Executable with details such as Code, Name, Source, Type, Created By, Creation Date, Version, and Active. You can add, view, modify, delete, or purge Data Mapping definitions. You can make any version of a Data Mapping definition as the latest. For more information, see Versioning and Make Latest Feature of Data Mapping.

For sorting the fields, mouse-over at the end of the Column heading and click to sort in the ascending order or click the Downward Arrow icon to sort the fields in the descending order.

You can search for a Data Mapping definition based on Code, Name, Type (F2T, T2F, and T2T), Source, and Record status. The options for Record Status are Executable, Active, Inactive, and Deleted.

  • Executable - Displays all active versions of Data Mapping definitions and inactive versions of the same Data Mapping definitions with distinct sources.
  • Active - Displays only the active version of all Data Mapping definitions.
  • Inactive - Displays all the inactive versions of Data Mapping definitions.
  • Deleted - Displays all the deleted Data Mapping definitions.