Data Mapping refers to the process of retrieving unstructured data from data sources for further data processing, storage, or migration. The intermediate extraction process can be followed by data transformation and metadata addition before exporting it to the staging area or to the Business Data Model.
Data movement can be from:
· RDBMS source to RDBMS target (T2T)
· RDBMS source to Flat File target(T2F)
· RDBMS source to HDFS-Hive target (T2H)
· HDFS-Hive source to RDBMS target(H2T)
· HDFS-Hive source to HDFS target (H2H)
· HDFS/Local-WebLog Source to HDFS Target (L2H)
· HDFS-Hive source to Flat File target (H2F)
· Flat File to RDBMS target (F2T)
· Flat File present in Local File System (LFS) to HDFS target or HDFS file to HDFS target(F2H)
|
NOTE |
File present in HDFS system cannot be loaded into RDBMS target Infodom. F2T and F2H can be defined from Data Mapping window. There is no separate Data File Mapping window. |
Data movement between Hive and RDBMS can be enhanced using third party tools like SQOOP and OLH (Oracle Loader for Hadoop). You need to set parameters from the DMT Configurations window. For details, see DMT Configurations section. For details on the configurations for SQOOP and OLH, see OFSAAI Administration Guide available in OHC Documentation Library.
For the configurations required to support Weblog ingestion (L2H), refer Data Movement of WebLog Source to HDFS target section in OFSAAI Administration Guide available in OHC Documentation Library.
The roles mapped to Data Mapping are as follows:
· DMACCESS
· DMREAD
· DMWRITE
· DMPHANTOM
· DMAUTH
· DMADV
|
NOTE |
Both old functions ETLDEF and ETLUSER and the aforementioned new functions will be supported if you set Allow Old Functions as Yes in the DMT Configurations window. Ensure the new roles are mapped to the required User Groups. |
For all the roles, functions and descriptions, see Appendix A.
The Data Mappings window displays the list of pre-defined Data Mapping definitions with Record Status as Executable with details such as Code, Name, Source, Type, Created By, Creation Date, Version, and Active. You can add, view, modify, delete, or purge Data Mapping definitions. You can make any version of a Data Mapping definition as latest. For more information, see Versioning and Make Latest Feature of Data Mapping section.
For sorting the fields, mouse-over at the end of the Column heading and click 
to sort in the ascending order or click 
to sort the fields in the descending order.
You can search for a Data Mapping definition based on Code, Name, Type (F2T, T2F and T2T), Source and Record status. The options for Record Status are Executable, Active, Inactive, and Deleted.
§ Executable- Displays all active versions of Data Mapping definitions and inactive versions of the same Data Mapping definitions with distinct sources.
§ Active- Displays only active version of all Data Mapping definitions
§ Inactive- Displays all inactive versions of Data Mapping definitions
§ Deleted- Displays all the deleted Data Mapping definitions.