public class JobUtils
extends java.lang.Object
Contains convenience methods used among demo jobs
Constructor and Description |
---|
JobUtils() |
Modifier and Type | Method and Description |
---|---|
static <K,V> java.lang.Class<? extends <any>> |
getMapredInputFormat(JobConf conf, java.lang.Class<?> rawClass)
Checks if the given class is an org.apache.hadoop.mapred.InputFormat or an org.apache.hadoop.mapreduce.lib.input.FileInputFormat and returns the appropriate adapter if needed.
|
static Path[] |
getPaths(java.lang.String inStr, Configuration conf) |
static void |
jobCleanup(Configuration conf)
Performs cleaning tasks once a job has finished.
|
static Path |
setupInputPath(java.lang.String inStr, Job job) |
static Path |
setupInputPath(java.lang.String inStr, JobConf conf)
A convenience method to infer a job's input provided by the user.
|
static Path |
setupOutputPath(java.lang.String outStr, Path basePath, Job job) |
static Path |
setupOutputPath(java.lang.String outStr, Path basePath, JobConf conf)
Convenience method for setting a job's output path.
|
static void |
validateInputDataSet(InputDataSet inputDs, Configuration conf) |
static void |
validateMapRedInputDataSet(InputDataSet inputDs, JobConf jobConf) |
public static <K,V> java.lang.Class<? extends <any>> getMapredInputFormat(JobConf conf, java.lang.Class<?> rawClass)
conf
- the job configurationrawClass
- a class to be used as input formatpublic static Path[] getPaths(java.lang.String inStr, Configuration conf) throws java.io.IOException
java.io.IOException
public static void jobCleanup(Configuration conf)
conf
- the job configurationpublic static Path setupInputPath(java.lang.String inStr, Job job) throws java.io.IOException
java.io.IOException
public static Path setupInputPath(java.lang.String inStr, JobConf conf) throws java.io.IOException
inStr
- the input data location. It may be a file's path, a comma-separated paths, a glob expression or an expression of the form path/<filename as regex>, that is, a path where the file name is given by a regular expression.conf
- the job configurationjava.io.IOException
public static Path setupOutputPath(java.lang.String outStr, Path basePath, Job job) throws java.io.IOException
java.io.IOException
public static Path setupOutputPath(java.lang.String outStr, Path basePath, JobConf conf) throws java.io.IOException
outStr
- the output string as specified by the userbasePath
- a base folder to be used in case outStr is a relative pathconf
- the job configurationjava.io.IOException
public static void validateInputDataSet(InputDataSet inputDs, Configuration conf) throws java.lang.Exception
java.lang.Exception
public static void validateMapRedInputDataSet(InputDataSet inputDs, JobConf jobConf) throws java.lang.Exception
java.lang.Exception
Copyright © 2017, 2019 Oracle and/or its affiliates. All Rights Reserved.