oracle.spatial.hadoop.vector.util

Class JobUtils

java.lang.Object

oracle.spatial.hadoop.vector.util.JobUtils

public class JobUtils
extends java.lang.Object

Contains convenience methods used among demo jobs

Author:

hsaucedo

Constructor Summary

Constructors

Constructor and Description
JobUtils()

Method Summary

Modifier and Type	Method and Description
static <K,V> java.lang.Class<? extends <any>>	getMapredInputFormat(JobConf conf, java.lang.Class<?> rawClass) Checks if the given class is an org.apache.hadoop.mapred.InputFormat or an org.apache.hadoop.mapreduce.lib.input.FileInputFormat and returns the appropriate adapter if needed.
static Path[]	getPaths(java.lang.String inStr, Configuration conf)
static void	jobCleanup(Configuration conf) Performs cleaning tasks once a job has finished.
static Path	setupInputPath(java.lang.String inStr, Job job)
static Path	setupInputPath(java.lang.String inStr, JobConf conf) A convenience method to infer a job's input provided by the user.
static Path	setupOutputPath(java.lang.String outStr, Path basePath, Job job)
static Path	setupOutputPath(java.lang.String outStr, Path basePath, JobConf conf) Convenience method for setting a job's output path.
static void	validateInputDataSet(InputDataSet inputDs, Configuration conf)
static void	validateMapRedInputDataSet(InputDataSet inputDs, JobConf jobConf)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JobUtils

public JobUtils()

Method Detail

getMapredInputFormat

public static <K,V> java.lang.Class<? extends <any>> getMapredInputFormat(JobConf conf,
                                                                          java.lang.Class<?> rawClass)

Checks if the given class is an org.apache.hadoop.mapred.InputFormat or an org.apache.hadoop.mapreduce.lib.input.FileInputFormat and returns the appropriate adapter if needed.

Parameters:

conf - the job configuration

rawClass - a class to be used as input format

Returns:

an input format class (instance of org.apache.hadoop.mapred.InputFormat)

getPaths

public static Path[] getPaths(java.lang.String inStr,
                              Configuration conf)
                       throws java.io.IOException

Throws:

java.io.IOException

jobCleanup

public static void jobCleanup(Configuration conf)

Performs cleaning tasks once a job has finished.

Parameters:

conf - the job configuration

setupInputPath

public static Path setupInputPath(java.lang.String inStr,
                                  Job job)
                           throws java.io.IOException

Throws:

java.io.IOException

setupInputPath

public static Path setupInputPath(java.lang.String inStr,
                                  JobConf conf)
                           throws java.io.IOException

A convenience method to infer a job's input provided by the user. The input path is determined using the following criteria:

1. If the input is a comma-separated paths, it is set as is and the method returns null
2. If the input exists, it is set as is and the method returns the given input as a path
3. The input is considered a glob. If files were found the method returns the input glob as a path
4. The input's file name is considered a regular expression. A file pattern filter is set to use the regular expression and the method returns the input's parent folder as a path.

Parameters:

inStr - the input data location. It may be a file's path, a comma-separated paths, a glob expression or an expression of the form path/<filename as regex>, that is, a path where the file name is given by a regular expression.

conf - the job configuration

Returns:

a path in case of using a single path or base path or null in case of the input being a comma-separated list of paths

Throws:

java.io.IOException

setupOutputPath

public static Path setupOutputPath(java.lang.String outStr,
                                   Path basePath,
                                   Job job)
                            throws java.io.IOException

Throws:

java.io.IOException

setupOutputPath

public static Path setupOutputPath(java.lang.String outStr,
                                   Path basePath,
                                   JobConf conf)
                            throws java.io.IOException

Convenience method for setting a job's output path. If the output string is a relative path and basePath is specified, it will set the output to be a file relative to basePath. Additionally, it will remove the output if it already exists.

Parameters:

outStr - the output string as specified by the user

basePath - a base folder to be used in case outStr is a relative path

conf - the job configuration

Returns:

the final output path

Throws:

java.io.IOException

validateInputDataSet

public static void validateInputDataSet(InputDataSet inputDs,
                                        Configuration conf)
                                 throws java.lang.Exception

Throws:

java.lang.Exception

validateMapRedInputDataSet

public static void validateMapRedInputDataSet(InputDataSet inputDs,
                                              JobConf jobConf)
                                       throws java.lang.Exception

Throws:

java.lang.Exception