The Perfect Balance feature of Oracle Big Data Appliance distributes the reducer load in a MapReduce application so that each reduce task does approximately the same amount of work. While the default Hadoop method of distributing the reduce load is appropriate for many jobs, it does not distribute the load evenly for jobs with significant data skew.
Data skew is an imbalance in the load assigned to different reduce tasks. The load is a function of:
The number of keys assigned to a reducer.
The number of records and the number of bytes in the values per key.
The total run time for a job is extended, to varying degrees, by the time that the reducer with the greatest load takes to finish. In jobs with a skewed load, some reducers complete the job quickly, while others take much longer. Perfect Balance can significantly shorten the total run time by distributing the load evenly, enabling all reducers to finish at about the same time.
Your MapReduce job can be written using either the mapred or mapreduce APIs; Perfect Balance supports both of them.