When you run your modified Java code, you can set the Perfect Balance properties by using the standard hadoop
command syntax:
bin/hadoop jar application_jarfile.jar ApplicationClass \ -conf application_config.xml \ -conf perfect_balance_config.xml \ -D application_config_property \ -D perfect_balance_config_property \ -libjars application_jar_path.jar...
Example 5-4 runs a script named pb_balanceapi.sh
, which runs the InvertedIndexMapreduce
class example packaged in the Perfect Balance JAR file. The key load metric properties are set to the values recommended in the Job Analyzer report shown in Figure 5-1.
To run the InvertedIndexMapreduce class example, see "About the Perfect Balance Examples."
Example 5-4 Running the InvertedIndexMapreduce Class
$ cat pb_balanceapi.sh BALANCER_HOME=/opt/oracle/orabalancer-<version>-h2 APP_JAR_FILE=/opt/oracle/orabalancer-<version>-h2/jlib/orabalancer-<version>.jar export HADOOP_CLASSPATH=${BALANCER_HOME}/jlib/orabalancer-<version>.jar:${BALANCER_HOME}/jlib/commons-math-2.2.jar:$HADOOP_CLASSPATH hadoop jar ${APP_JAR_FILE} oracle.hadoop.balancer.examples.invindx.InvertedIndexMapreduce \ -D mapreduce.input.fileinputformat.inputdir=invindx/input \ -D mapreduce.output.fileoutputformat.outputdir=jdoe_outdir_api \ -D mapreduce.job.name=jdoe_invindx_api \ -D mapreduce.job.reduces=10 \ -D oracle.hadoop.balancer.linearKeyLoad.keyWeight=93.981394 \ -D oracle.hadoop.balancer.linearKeyLoad.rowWeight=0.001126 \ -D oracle.hadoop.balancer.linearKeyLoad.byteWeight=0.0 $ sh ./balanceapi.sh 14/04/14 15:03:51 INFO balancer.Balancer: Creating balancer 14/04/14 15:03:51 INFO balancer.Balancer: Starting Balancer 14/04/14 15:03:51 INFO input.FileInputFormat: Total input paths to process : 5 14/04/14 15:03:54 INFO balancer.Balancer: Balancer completed 14/04/14 15:03:55 INFO input.FileInputFormat: Total input paths to process : 5 14/04/14 15:03:55 INFO mapreduce.JobSubmitter: number of splits:5 14/04/14 15:03:55 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1397066986369_3510 14/04/14 15:03:55 INFO impl.YarnClientImpl: Submitted application application_1397066986369_3510 . . . File Input Format Counters Bytes Read=112652976 File Output Format Counters Bytes Written=384974202