If you get Java "out of heap space" or "GC overhead limit exceeded" errors on the client node while running the Perfect Balance sampler, then increase the client JVM heap size for the job.
Use the Java JVM -Xmx
option.You can specify client JVM options before running the Hadoop job, by setting the HADOOP_CLIENT_OPTS
variable:
$ export HADOOP_CLIENT_OPTS="-Xmx1024M $HADOOP_CLIENT_OPTS"
Setting HADOOP_CLIENT_OPTS
changes the JVM options only on the client node. It does not change JVM options in the map and reduce tasks. See the invindx
script for an example of setting this variable.
Setting HADOOP_CLIENT_OPTS
is sufficient to increase the heap size for the sampler, regardless of whether oracle.hadoop.balancer.runMode
is set to local
or distributed
. When runMode=local
, the sampler runs on the client node, and HADOOP_CLIENT_OPTS
sets the heap size on the client node. When runMode=distributed
, Perfect Balance automatically sets the heap size for the sampler Hadoop job based on the -Xmx
setting you provide in HADOOP_CLIENT_OPTS
. Perfect Balance never changes the heap size for the map and reduce tasks of your job, only for its sampler job.