oracle.spatial.hadoop.rasterapi.core

Class HadoopConfiguration

java.lang.Object

oracle.spatial.hadoop.rasterapi.core.HadoopConfiguration

public class HadoopConfiguration
extends java.lang.Object

Represents Hadoop Configuration to be used when executing the raster jobs. Configuration may be set using a properties file or one by one using included method setProperty.

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.conf.Configuration	conf

Constructor Summary

Constructors

Constructor and Description
HadoopConfiguration() Class Constructor creates a new Configuration object.
HadoopConfiguration(java.util.Properties prop) Class Constructor creates a new Configuration object and sets the related properties.

Method Summary

Modifier and Type	Method and Description
RasterJob	createRasterJob() Creates a raster job based on this configuration, if it was already created, no new job is created.
RasterJob	createRasterLoaderJob() Creates a raster loader job based on this configuration, if it was already created, no new job is created.
RasterJob	createRasterProcessorJob() Creates a raster processsing job based on this configuration, if it was already created, no new job is created.
org.apache.hadoop.conf.Configuration	getConf() Gets the hadoop configuration object
org.apache.hadoop.fs.FileSystem	getFileSystem() Gets the filesystem associated to this configuration
RasterJob	getJob() Gets the job related to this configuration
java.lang.String	getProperty(java.lang.String key) Gets the property for the related key on this configuration
org.apache.hadoop.fs.Path	getYarnLogDir() Gets the path for yarn logs
void	printProperties() Dumps the configuration to system console, for debugging.
void	setBlockSize(int mb) Sets the HDFS block size
void	setGdalDataPath(java.lang.String gdalDataPath) Sets GDAL_DATA environment variable to this configuration, location of this folder must be shared to all instances of the cluster.
void	setHdfsPathPrefix(java.lang.String hdfsPathPrefix) Sets the default HDFS path prefix for this configuration.
void	setJobHistoryAddress(java.lang.String jobHistoryAddress) Sets the job history server address for this configuration.
void	setMapreduceJobJar(java.lang.String jobJar) Sets the jar that contains mapreduce application to attach in this configuration.
void	setProperty(java.lang.String key, java.lang.String value) Sets a property to this configuration.
void	setResourceManagerAddress(java.lang.String resourceManagerAddress) Sets the address of the applications manager interface in the Resource Manager.
void	setResourceManagerScheduler(java.lang.String resourceManagerScheduler) Sets the Resource Manager scheduler interface for this configuration.
void	setUser(java.lang.String user) Sets the ugi user to execute jobs created from this configuration.
void	setYarnApplicationClasspath(java.lang.String classpath) Sets the yarn application classpath to be used in this job

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

conf

protected org.apache.hadoop.conf.Configuration conf

Constructor Detail

HadoopConfiguration

public HadoopConfiguration(java.util.Properties prop)
                    throws java.lang.Exception

Class Constructor creates a new Configuration object and sets the related properties.

Parameters:

prop - properties containing the element to set in the configuration object.

Throws:

java.lang.Exception - if there is a problem during configuration creation

HadoopConfiguration

public HadoopConfiguration()

Class Constructor creates a new Configuration object.

Method Detail

getProperty

public java.lang.String getProperty(java.lang.String key)

Gets the property for the related key on this configuration

Parameters:

key - key value to search in this configuration

Returns:

the value stored in this configuration for the related key

getYarnLogDir

public org.apache.hadoop.fs.Path getYarnLogDir()

Gets the path for yarn logs

Returns:

path for yarn logs

setProperty

public void setProperty(java.lang.String key,
                        java.lang.String value)

Sets a property to this configuration.

Parameters:

key - key value to set in this configuration

value - value related to set in this configuration

printProperties

public void printProperties()
                     throws java.lang.Exception

Dumps the configuration to system console, for debugging.

Throws:

java.lang.Exception - if there is a problem to get the PrintWriter

getJob

public RasterJob getJob()

Gets the job related to this configuration

Returns:

the job related to this configuration

createRasterJob

public RasterJob createRasterJob()

Creates a raster job based on this configuration, if it was already created, no new job is created.

Returns:

RasterJob created for this configuration or previously existing one

createRasterLoaderJob

public RasterJob createRasterLoaderJob()

Creates a raster loader job based on this configuration, if it was already created, no new job is created.

Returns:

RasterLoaderJob created for this configuration or previously existing one

getConf

public org.apache.hadoop.conf.Configuration getConf()

Gets the hadoop configuration object

Returns:

hadoop configuration object

setGdalDataPath

public void setGdalDataPath(java.lang.String gdalDataPath)

Sets GDAL_DATA environment variable to this configuration, location of this folder must be shared to all instances of the cluster.

Parameters:

gdalDataPath - is the path for GDAL_DATA directory

createRasterProcessorJob

public RasterJob createRasterProcessorJob()

Creates a raster processsing job based on this configuration, if it was already created, no new job is created.

Returns:

RasterProcessorJob created for this configuration or previously existing one

getFileSystem

public org.apache.hadoop.fs.FileSystem getFileSystem()
                                              throws java.io.IOException

Gets the filesystem associated to this configuration

Returns:

the FileSystem object associated with this Configuration.

Throws:

java.io.IOException - if there is a problem to get HDFS pointer

setUser

public void setUser(java.lang.String user)

Sets the ugi user to execute jobs created from this configuration.

Parameters:

user - user to set in this configuration

setHdfsPathPrefix

public void setHdfsPathPrefix(java.lang.String hdfsPathPrefix)

Sets the default HDFS path prefix for this configuration.

Parameters:

hdfsPathPrefix - HDFS path prefix to set in this configuration

setBlockSize

public void setBlockSize(int mb)

Sets the HDFS block size

Parameters:

mb - block size in megabytes to set

setMapreduceJobJar

public void setMapreduceJobJar(java.lang.String jobJar)

Sets the jar that contains mapreduce application to attach in this configuration.

Parameters:

jobJar - HDFS path prefix to set in this configuration

setYarnApplicationClasspath

public void setYarnApplicationClasspath(java.lang.String classpath)

Sets the yarn application classpath to be used in this job

Parameters:

classpath - yarn application classpath

setJobHistoryAddress

public void setJobHistoryAddress(java.lang.String jobHistoryAddress)

Sets the job history server address for this configuration.

Parameters:

jobHistoryAddress - job history server address to set in this configuration

setResourceManagerScheduler

public void setResourceManagerScheduler(java.lang.String resourceManagerScheduler)

Sets the Resource Manager scheduler interface for this configuration.

Parameters:

resourceManagerScheduler - scheduler interface to set in this configuration

setResourceManagerAddress

public void setResourceManagerAddress(java.lang.String resourceManagerAddress)

Sets the address of the applications manager interface in the Resource Manager.

Parameters:

resourceManagerAddress - addredd of the applications manager interface to set in this configuration