oracle.spatial.hadoop.rasterapi.core

Class HdfsUtil

java.lang.Object

oracle.spatial.hadoop.rasterapi.core.HdfsUtil

public class HdfsUtil
extends java.lang.Object

Utilitary methods to create and execute the jobs

Constructor Summary

Constructors

Constructor and Description
HdfsUtil()

Method Summary

Modifier and Type	Method and Description
static java.io.File	copyFromHDFStoLocal(java.lang.String hdfsFile, java.lang.String localFile, org.apache.hadoop.conf.Configuration conf) Copies specified HDFS file to Local FileSystem
static void	deleteFile(java.lang.String path, org.apache.hadoop.conf.Configuration conf) Deletes file from HDFS
static org.apache.hadoop.conf.Configuration	getConf(org.apache.hadoop.fs.FileSystem fs) Return the configuration for this fileSystem
static org.apache.hadoop.fs.FileStatus	getFile(java.lang.String path, org.apache.hadoop.conf.Configuration conf) Gets the specified file from HDFS
static org.apache.hadoop.fs.FileStatus[]	getFiles(org.apache.hadoop.fs.Path hdfsPath, org.apache.hadoop.conf.Configuration conf) Gets the list of files contained in the specified folder
static org.apache.hadoop.fs.Path	getHomeHDFSDirectory(org.apache.hadoop.conf.Configuration conf) Return the current user's home directory in this filesystem
static org.apache.hadoop.fs.FileStatus[]	listFolderContent(org.apache.hadoop.conf.Configuration conf, java.lang.String path) Lists the content of the specified HDFS folder
static org.apache.hadoop.fs.FSDataInputStream	open(org.apache.hadoop.fs.Path p, org.apache.hadoop.conf.Configuration conf) Opens the specified file.
static boolean	validatePathExistence(org.apache.hadoop.conf.Configuration conf, java.lang.String path) Validates if the specified HDFS path exists

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HdfsUtil

public HdfsUtil()

Method Detail

copyFromHDFStoLocal

public static java.io.File copyFromHDFStoLocal(java.lang.String hdfsFile,
                                               java.lang.String localFile,
                                               org.apache.hadoop.conf.Configuration conf)
                                        throws java.io.FileNotFoundException,
                                               java.io.IOException

Copies specified HDFS file to Local FileSystem

Parameters:

hdfsFile - path to source HDFS file

localFile - path to destiny local file

conf - configuration object used to get access to HDFS

Returns:

the File object created in local FS

Throws:

java.io.FileNotFoundException - if the file could not be copied to local filesystem

java.io.IOException - if there is a problem during read of source file or write of final local file system

validatePathExistence

public static boolean validatePathExistence(org.apache.hadoop.conf.Configuration conf,
                                            java.lang.String path)
                                     throws java.io.IOException

Validates if the specified HDFS path exists

Parameters:

conf - Hadoop HDFS configuration to get access into file system

path - path to validate if exists or not

Returns:

true if the path exists or else otherwise

Throws:

java.io.IOException - if there is a problem while trying to access the path

listFolderContent

public static org.apache.hadoop.fs.FileStatus[] listFolderContent(org.apache.hadoop.conf.Configuration conf,
                                                                  java.lang.String path)
                                                           throws java.io.IOException

Lists the content of the specified HDFS folder

Parameters:

conf - Hadoop configuration object used to get access to HDFS

path - path to folder containing items to list

Returns:

an array with the list of files found in the specified folder

Throws:

java.io.IOException - if there is a problem during the access to the folder

getHomeHDFSDirectory

public static org.apache.hadoop.fs.Path getHomeHDFSDirectory(org.apache.hadoop.conf.Configuration conf)
                                                      throws java.io.IOException

Return the current user's home directory in this filesystem

Parameters:

conf - Hadoop configuration object used to get access to HDFS

Returns:

the HDFS home directory for the user in this configuration

Throws:

java.io.IOException - if there is a problem during access to this folder

deleteFile

public static void deleteFile(java.lang.String path,
                              org.apache.hadoop.conf.Configuration conf)
                       throws java.io.IOException,
                              java.io.FileNotFoundException

Deletes file from HDFS

Parameters:

path - path to the file object of deletion

conf - Hadoop configuration object used to get access to HDFS

Throws:

java.io.IOException - if there is problem during access to the specified file

java.io.FileNotFoundException - if the specified file to delete does not exist

getFile

public static org.apache.hadoop.fs.FileStatus getFile(java.lang.String path,
                                                      org.apache.hadoop.conf.Configuration conf)
                                               throws java.io.IOException,
                                                      java.io.FileNotFoundException

Gets the specified file from HDFS

Parameters:

path - path to the HDFS file that is specified to be selected

conf - Hadoop configuration object used to get access to HDFS

Returns:

the file obtained from HDFS

Throws:

java.io.IOException - if there is problem during access to the specified file

java.io.FileNotFoundException - if the specified file to get does not exist

getFiles

public static org.apache.hadoop.fs.FileStatus[] getFiles(org.apache.hadoop.fs.Path hdfsPath,
                                                         org.apache.hadoop.conf.Configuration conf)
                                                  throws java.io.IOException,
                                                         java.io.FileNotFoundException

Gets the list of files contained in the specified folder

Parameters:

hdfsPath - path of the HDFS folder to pull files from

conf - Hadoop configuration object used to get access to HDFS

Returns:

the list of files included in the specified HDFS folder

Throws:

java.io.IOException - if there is problem during access to the specified file

java.io.FileNotFoundException - if the specified folder to access does not exist

getConf

public static org.apache.hadoop.conf.Configuration getConf(org.apache.hadoop.fs.FileSystem fs)

Return the configuration for this fileSystem

Parameters:

fs - the HDFS to pull configuration from

Returns:

the Configuration object attached to the specified HDFS

open

public static org.apache.hadoop.fs.FSDataInputStream open(org.apache.hadoop.fs.Path p,
                                                          org.apache.hadoop.conf.Configuration conf)
                                                   throws java.io.IOException,
                                                          java.io.FileNotFoundException

Opens the specified file.

Parameters:

p - path to the HDFS file to open

conf - Hadoop configuration object used to get access to HDFS

Returns:

the input stream for the requested HDFS file

Throws:

java.io.IOException - if there is problem during access to the specified file

java.io.FileNotFoundException - if the specified folder to access does not exist