com.nt.udc.ei.transport

Class FileEITransport

java.lang.Object

com.nt.udc.ndk.node.EITransport

com.nt.udc.ei.transport.FileEITransport

All Implemented Interfaces:

DataProviderIfc, DCTransport, FileDataProviderIfc, java.lang.Runnable

Direct Known Subclasses:

GenericFlatFileProcessor, NAREIFileProcessor

public abstract class FileEITransport
extends EITransport

Used to represent a class that provides the processing "engine" for files that are retrieved or sent from network elements. This class provides a mechanism to monitor a directory for files to process either via a "SynchronizedDirectory" object or just a standard File Object. The "SynchronizedDirectory" object is used when objects of this class need to be informed when files are available, and the standard File object can be used when objects of this class monitor a directory themselves.

Field Summary

Fields

Modifier and Type	Field and Description
protected static int	DEFAULT_POLLING_INTERVAL Default polling interval if none is specified in config file:
protected static java.lang.String	DELETE_LOCAL_FILES Config string in the node's general config file:
protected boolean	deleteLocalFiles Determines whether to delete the local files after processing them.
protected boolean	fileLevelTransaction
protected FileEISequenceCheck	fileSeqCheck
protected boolean	fsChecked
static java.lang.String	INPUT_BACKUP_DIR
protected SMGLock	ioLock Used to synchronize the thread in which this object runs and the EI Node's main thread when shutting odwn.
protected boolean	isOrdered
protected java.lang.String	itsFileSuffix Specifies the suffix of the files this node will process
protected java.io.File	localDir Represents the local directory from which this object gets files to process.
protected LoggerIfc	logger Represents the logging facility for the EI node using this object.
protected boolean	mtChecked
static java.lang.String	MULTI_THREADED
static java.lang.String	ORDER_REQUIRED
protected static java.lang.String	POLLING_INTERVAL Config string in the node's general config file:
protected SynchronizedDirectory	scratchDir Represents the local directory from which this object gets files to process.
protected boolean	useSynchronizedDirectory Determines whether to use a SynchronizedDirectory or just a File object for monitoring for files to process.

Fields inherited from class com.nt.udc.ndk.node.EITransport

dataReceiver, fdChecked, filedataReceiver, fileduplicateCheck, fileToProcess, inputCount, lockFile, movedFile, node, pollingInterval, processing, processingData, running

Constructor Summary

Constructors

Constructor and Description
FileEITransport(DCNode node, java.io.File sd, java.lang.String suffix, SMGLock lock) Use this form when monitoring a local directory directly, as opposed to waiting to be notified when files are ready to process.
FileEITransport(DCNode node, SynchronizedDirectory sd, java.lang.String suffix, SMGLock lock) Use this form when you want to be notified when files are ready to be processed.

Method Summary

Modifier and Type	Method and Description
boolean	deleteLocalFiles() Returns "delete local files" flag:
protected java.io.FilenameFilter	getActFilter()
DCFieldContainer[]	getBulkData() Allows a DataReceiver to obtain the available data in a known format.
DCFieldContainer	getData() Allows a DataReceiver to obtain the available data in a known format.
java.lang.String	getInputBackupDir()
protected long	getLength(java.io.File file) Returns the length stored in the scratch file.
boolean	isDataAvailable() Allows a DataReceiver to determine whether some data is available for processing
boolean	isHealthy() This method is used to check if the transport is fully functional.
boolean	isLessThan(java.lang.String value1, java.lang.String value2)
void	moveCurrentFileToBackUp(java.io.File currentFile, java.lang.String newFilename)
protected abstract void	processFiles(java.util.Vector filesToProcess) Here's where we actually parse the files and extract the data of interest.
protected long	readLength(java.io.RandomAccessFile recoveryFile) This method is used to read the number of bytes already read and processed.
void	run() This is what gets executed as this object runs in its thread: Basically, wait for files that need to be processed and then process them.
protected void	setActFilter(java.io.FilenameFilter newFilter)
void	setDubCheckObj(FileEIDuplicateCheck fileduplicateCheck)
void	setDupCheckFlag(boolean fdChecked)
void	setSeqCheckFlag(boolean fsChecked)
void	setSeqCheckObj(FileEISequenceCheck fileSeqCheck)
void	shutdown() Specifies actions needed to shut down the thread in which this object runs.
java.lang.String[]	sortFiles(java.lang.String[] filesToProcess)
protected void	writeLength(java.io.File scratchfile, long len) Record a long in a file - used to write a length in a scratch file to keep track of how much of a file has been processed.
protected void	writeLength(java.io.RandomAccessFile recoveryFile, long len) This method is used to store the current input file's length in the recovery file.

Methods inherited from class com.nt.udc.ndk.node.EITransport

getCurrArchFile, getData, getDataReceiver, getDubCheckObj, getDupCheckFlag, getFileBulkData, getFileData, getFileDataReceiver, getFileList, getMovedFileStatus, getNode, getPollingInterval, getProcessedArchFile, handleRejectOfFile, incrementIn, incrementIn, isEndOfFile, isFileDataAvailable, isProcessing, isProcessingData, isStartOfFile, openInputFile, rejectFile, removeProcessedFile, setCountsSuspect, setDataReceiver, setFileDataReceiver, setFileToProcess, setMovedFileStatus, setNode, setPollingInterval, setProcessedArchFile, setProcessing, signalBatchCompletion, validOIMinorType

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

POLLING_INTERVAL

protected static java.lang.String POLLING_INTERVAL

Config string in the node's general config file:

DELETE_LOCAL_FILES

protected static java.lang.String DELETE_LOCAL_FILES

Config string in the node's general config file:

MULTI_THREADED

public static final java.lang.String MULTI_THREADED

See Also:

Constant Field Values

ORDER_REQUIRED

public static final java.lang.String ORDER_REQUIRED

See Also:

Constant Field Values

INPUT_BACKUP_DIR

public static final java.lang.String INPUT_BACKUP_DIR

See Also:

Constant Field Values

DEFAULT_POLLING_INTERVAL

protected static int DEFAULT_POLLING_INTERVAL

Default polling interval if none is specified in config file:

deleteLocalFiles

protected boolean deleteLocalFiles

Determines whether to delete the local files after processing them.

logger

protected LoggerIfc logger

Represents the logging facility for the EI node using this object.

scratchDir

protected SynchronizedDirectory scratchDir

Represents the local directory from which this object gets files to process.

localDir

protected java.io.File localDir

Represents the local directory from which this object gets files to process.

useSynchronizedDirectory

protected boolean useSynchronizedDirectory

Determines whether to use a SynchronizedDirectory or just a File object for monitoring for files to process.

itsFileSuffix

protected java.lang.String itsFileSuffix

Specifies the suffix of the files this node will process

ioLock

protected SMGLock ioLock

Used to synchronize the thread in which this object runs and the EI Node's main thread when shutting odwn. This "lock" object can be used to ensure that all underlying threads (like the one in which this guy runs) are finished with whatever they were in the middle of doing.

mtChecked

protected boolean mtChecked

isOrdered

protected boolean isOrdered

fileLevelTransaction

protected boolean fileLevelTransaction

fileSeqCheck

protected FileEISequenceCheck fileSeqCheck

fsChecked

protected boolean fsChecked

Constructor Detail

FileEITransport

public FileEITransport(DCNode node,
                       java.io.File sd,
                       java.lang.String suffix,
                       SMGLock lock)
                throws NodeStartException

Use this form when monitoring a local directory directly, as opposed to waiting to be notified when files are ready to process. (This is the form used by EI's that process local files.)

Parameters:

node - specifies the DCNode using this processing engine

sd - specifies the directory from which this node gets files to process

suffix - specifies the suffix of the files this node will process

lock - used to properly shutdown the EI and all its threads

Throws:

NodeStartException

FileEITransport

public FileEITransport(DCNode node,
                       SynchronizedDirectory sd,
                       java.lang.String suffix,
                       SMGLock lock)
                throws NodeStartException

Use this form when you want to be notified when files are ready to be processed. (This is the form used by EI's that retrieve files via FTP.)

Parameters:

node - specifies the DCNode using this processing engine

sd - specifies the directory from which this node gets files to process

suffix - specifies the suffix of the files this node will process

lock - used to properly shutdown the EI and all its threads

Throws:

NodeStartException

Method Detail

shutdown

public void shutdown()

Specifies actions needed to shut down the thread in which this object runs.

Specified by:

shutdown in interface DCTransport

Specified by:

shutdown in class EITransport

getData

public DCFieldContainer getData()

Allows a DataReceiver to obtain the available data in a known format.

Specified by:

getData in interface DataProviderIfc

Specified by:

getData in class EITransport

Returns:

DCFieldContainer object containing the data that is available for processing

getBulkData

public DCFieldContainer[] getBulkData()

Allows a DataReceiver to obtain the available data in a known format.

Specified by:

getBulkData in interface DataProviderIfc

Specified by:

getBulkData in class EITransport

Returns:

DCFieldContainer objects containing the data that is available for processing

isDataAvailable

public boolean isDataAvailable()

Allows a DataReceiver to determine whether some data is available for processing

Specified by:

isDataAvailable in interface DataProviderIfc

Specified by:

isDataAvailable in class EITransport

Returns:

true, if data is available

run

public void run()

This is what gets executed as this object runs in its thread: Basically, wait for files that need to be processed and then process them. Continue until signalled to shut down.

Specified by:

run in interface java.lang.Runnable

Specified by:

run in class EITransport

processFiles

protected abstract void processFiles(java.util.Vector filesToProcess)

Here's where we actually parse the files and extract the data of interest. Make sure you handle shutting down the node correctly as well as synchronizing all necessary threads. The SynchronizedDirectory class provides a synchronizing capability for this class and the transport class. There is also an SMGLock object passed into the constructor of this class for synchronizing this thread with the parent EI node thread (when shutting down). The trick is to never let this method get wrapped up in a time-consuming processing loop (in case the user wants to shut down this node in the middle of such a loop).

Parameters:

filesToProcess - list of filenames that need to be processed

deleteLocalFiles

public boolean deleteLocalFiles()

Returns "delete local files" flag:

Returns:

value of flag

writeLength

protected void writeLength(java.io.File scratchfile,
                           long len)

Record a long in a file - used to write a length in a scratch file to keep track of how much of a file has been processed. Used for recovering when the node is shut down prematurely (or crashes or dies unexpectedly). To use this, call it from the appropriate place in your version of the "processFiles" method.

getLength

protected long getLength(java.io.File file)

Returns the length stored in the scratch file. Used to keep track of how much of a file has been processed.

Parameters:

file - Specifies the scratch file.

Returns:

number of bytes that have already been read and processed.

isHealthy

public boolean isHealthy()

This method is used to check if the transport is fully functional.

Specified by:

isHealthy in interface DCTransport

Overrides:

isHealthy in class EITransport

Returns:

true if the transport is healthy, false otherwise

writeLength

protected void writeLength(java.io.RandomAccessFile recoveryFile,
                           long len)

This method is used to store the current input file's length in the recovery file. Subclasses should call this method.

Parameters:

recoveryFile - Specifies the recovery file.

readLength

protected long readLength(java.io.RandomAccessFile recoveryFile)

This method is used to read the number of bytes already read and processed. Subclasses should call this method.

Parameters:

recoveryFile - Specifies the recovery file.

getActFilter

protected java.io.FilenameFilter getActFilter()

setActFilter

protected void setActFilter(java.io.FilenameFilter newFilter)

sortFiles

public java.lang.String[] sortFiles(java.lang.String[] filesToProcess)

isLessThan

public boolean isLessThan(java.lang.String value1,
                          java.lang.String value2)

setDupCheckFlag

public void setDupCheckFlag(boolean fdChecked)

setSeqCheckFlag

public void setSeqCheckFlag(boolean fsChecked)

setDubCheckObj

public void setDubCheckObj(FileEIDuplicateCheck fileduplicateCheck)

setSeqCheckObj

public void setSeqCheckObj(FileEISequenceCheck fileSeqCheck)

getInputBackupDir

public java.lang.String getInputBackupDir()

moveCurrentFileToBackUp

public void moveCurrentFileToBackUp(java.io.File currentFile,
                                    java.lang.String newFilename)
                             throws java.io.IOException

Overrides:

moveCurrentFileToBackUp in class EITransport

Throws:

java.io.IOException