JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Analyzing and Cleansing Data for a Master Index     Java CAPS Documentation
search filter icon
search icon

Document Information

Analyzing and Cleansing Data for a Master Index

Related Topics

Data Cleansing and Analysis Overview

About the Data Profiler

About the Data Cleanser

Data Cleansing and Profiling Process Overview

Required Format for Flat Data Files

Generating the Data Profiler and Data Cleanser

To Generate the Data Profiler and Data Cleanser

Configuring the Environment

To Configure the Environment

Extracting the Legacy Data

Determining the Fields to Analyze

Defining the Data Analysis Rules

To Define Data Analysis Rules

Performing the Initial Data Analysis

To Perform the Initial Data Analysis

Reviewing the Data Profiler Reports

Configuring the Data Cleansing Rules

To Configure the Data Cleansing Rules

Cleansing the Legacy Data

To Cleanse the Data

Performing Frequency Analyses on Cleansed Data

Adjusting the Master Index Configuration

Data Profiler Rules Syntax

Data Profiler Processing Attributes

Data Profiler Global Variables

Simple Frequency Analysis Rules

Constrained Frequency Analysis Rules

Pattern Frequency Analysis Rules

Data Cleanser Rules Syntax

Data Cleanser Processing Attributes

Data Cleanser Global Variables

Data Validation Rules

dataLength

dateRange

matchFromFile

patternMatch

range

reject

return

validateDBField

Data Transformation Rules

assign

patternReplace

replace

truncate

Conditional Data Rules

dataLength

equals

isnull

matches

Conditional Operators

Data Profiler Report Samples

Simple Frequency Analysis Report Samples

Constrained Frequency Analysis Report Samples

Pattern Frequency Analysis Report Samples

Generating the Data Profiler and Data Cleanser

In order to use the data analysis tools, you need to generate the Data Profiler and Data Cleanser from the master index application for which they will be used. These tools are based on the information you specified in the Master Index Wizard and changes you made to the configuration files. You can generate and extract the Data Profiler and Data Cleanser to a Windows or UNIX machine.

To Generate the Data Profiler and Data Cleanser

  1. In the NetBeans Project window, right-click the main project of the master index application and then select Generate Cleanser Zip.

    The file is generated and downloaded to NetBeans_Projects/Project_Name/cleanser-generated.

  2. After the Data Cleanser is downloaded, right-click the main project again and then select Generate Profiler Zip.

    The file is generated and downloaded to NetBeans_Projects/Project_Name/profiler-generated.

  3. On your computer, navigate to NetBeans_Projects/Project_Name/cleanser-generated and extract the contents of cleanser.zip.
  4. After the file are extracted, navigate to NetBeans_Projects/Project_Name/profiler-generated and extract the contents of profiler.zip.

Next Steps

Continue to Configuring the Environment.