com.beasys.commerce.axiom.document.loader
Class BulkLoader

java.lang.Object
  |
  +--com.beasys.commerce.axiom.document.loader.BulkLoader

public class BulkLoader

extends java.lang.Object

implements java.io.FilenameFilter

The reference document repository bulk loader application.

This class is mainly designed to run as a command-line application, via a "java com.beasys.commerce.axiom.document.loader.BulkLoader" command-line. To see a usage, give it a -h flag or read the Usage.txt in this package.

Additionally, BulkLoader objects can be created and used to provide the functionality in other places. The lifecycle of a BulkLoader is as follows:

• Construction
• parseArgs() if not passed in to constructor.
• validateArgs() to make sure the loader has valid arguments.
• initialize() to setup internal state.
• go() to execute the load.
• shutdown() to cleanup everything.

Not calling parseArgs(), validateArgs(), and initialize(), in that order, will cause the BulkLoader object to most likely ungracefully fail. However, once those methods have been invoked, the utility methods (like go(), doLoad(), printSchema(), deleteDoc(), and insertDoc()) can invoked as needed. The commit() and rollback() method will control the BulkLoader's transaction with the database. When finished, be sure to call the shutdown() method to release internal resources.

If manually constructing and utilizing a BulkLoader object, be certain to synchronize all access to the object. Since the command-line program is single-threaded, BulkLoader objects are not thread-safe by design.

See Also:

MetaParser, FileCache

Inner Class Summary

static interface

BulkLoader.ShowUsageException Quick inner exception thrown on parseArgs() to say we should just print a usage report.

Field Summary

static java.lang.String	DEF_MD_FILE_EXT The default file extension for metadata property files.
static java.lang.String	DEF_MIME_TYPE The default mime type.
static java.lang.String	DEF_SCHEMA_NAME The default schema name.
static java.lang.String	DEF_SCHEMA_PATH The default path for the schema file.
static java.lang.String	DEF_WLS_PROPS_PATH The default weblogic properties file path.
static java.lang.String	deleteDocSql The preparable sql to remove a document from the database.
static java.lang.String	deleteMDSql The preparable sql to remove a document's implicit metadata from the database.
static java.lang.String	DOC_MD_TABLE The wlcs_document_metadata table name.
static java.lang.String	DOC_TABLE The wlcs_document table name.
static java.lang.String	insertDocSql The preparable sql to insert a document into the database.
static java.lang.String	insertMDSql The preparable sql to insert a document metadata into the database.

Constructor Summary

BulkLoader()
Constructor a BulkLoader without command-line arguments.

BulkLoader(java.lang.String[] args)
Construct a BulkLoader from the given command-line arguments.

Method Summary

boolean	accept(java.io.File dir, java.lang.String name) Implement the FilenameFilter interface method to use our match and ignore lists.
static void	close(java.lang.Object o) Close an object which has a close() method, ignoring any exceptions.
void	commit() Commit the transaction.
void	debug(java.lang.String mesg) Out put a debug message.
void	deleteDoc(java.lang.String path) This will remove the document with id of path from the database, including all of its implicit metadata.
void	doLoad() Do the actual bulk load logic on the file list.
void	doLoad(java.io.File baseDir, java.lang.String path, java.util.Properties dirProps) Load the given path into the database.
void	error(java.lang.String mesg) Output an error message.
void	error(java.lang.String mesg, java.lang.Throwable ex) Output an error message.
java.lang.String	fixPath(java.lang.String path) Fix up a path to be forward-slash style and to not have empty path parts.
java.util.Properties	getMetadataProperties(java.io.File base, java.util.Properties p) Get the metadata properties for the given file or directory.
void	go() Let the bulkloader go.
void	initialize() Initialize the bulkloader from the current state.
void	insertDoc(java.lang.String path, java.io.File f, java.util.Properties p, java.lang.String mimeType) Insert a document and metadata into the database.
static boolean	isHidden(java.io.File f) Check if the specified file is a hidden file.
boolean	isHtmlFile(java.lang.String name) Tell if the specified file name is an HTML file to the loader.
static boolean	isReadableDirectory(java.lang.String name) Check if the specified file name is a directory that we can get into.
static int	main(BulkLoader loader, java.lang.String[] args) The main method invoked on a BulkLoader instance.
static void	main(java.lang.String[] args) Command-line entry point.
void	parseArgs(java.lang.String[] args) Parse the given input arguments.
void	printSchema() Print the schema xml for all the metadata we've loaded so far.
void	printSchema(java.io.PrintWriter out, java.lang.String enc) Print the schema xml for all the metadata we've loaded so far to the given output stream.
void	rollback() Rollback the transaction.
boolean	shouldIgnore(java.lang.String name) Tell if the loader should ignore the specified file name.
boolean	shouldInclude(java.lang.String name) Tell if the loader should include the specified file name.
void	shutdown() Shutdown this bulk loader.
static java.util.Properties	splitToProperties(java.lang.String str, java.lang.String on) Split a WLS connection pool style string in a Properties object of the name=value pairs.
void	usage(java.io.PrintStream out) Print the usage of the application.
void	usage(java.io.PrintWriter out) Print the usage of the application.
void	validateArgs() Validate that we have been passed correct arguments.
void	warning(java.lang.String mesg) Output a warning message.
void	warning(java.lang.String mesg, java.lang.Throwable ex) Output a warning message.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DOC_TABLE

public static java.lang.String DOC_TABLE

The wlcs_document table name.

See Also:

RefDocumentProvider.DOC_TABLE

DOC_MD_TABLE

public static java.lang.String DOC_MD_TABLE

The wlcs_document_metadata table name.

See Also:

RefDocumentProvider.DOC_MD_TABLE

DEF_MD_FILE_EXT

public static final java.lang.String DEF_MD_FILE_EXT

The default file extension for metadata property files.

DEF_SCHEMA_PATH

public static final java.lang.String DEF_SCHEMA_PATH

The default path for the schema file.

DEF_SCHEMA_NAME

public static final java.lang.String DEF_SCHEMA_NAME

The default schema name.

DEF_WLS_PROPS_PATH

public static final java.lang.String DEF_WLS_PROPS_PATH

The default weblogic properties file path.

DEF_MIME_TYPE

public static final java.lang.String DEF_MIME_TYPE

The default mime type.

deleteMDSql

public static final java.lang.String deleteMDSql

The preparable sql to remove a document's implicit metadata from the database.

deleteDocSql

public static final java.lang.String deleteDocSql

The preparable sql to remove a document from the database.

insertDocSql

public static final java.lang.String insertDocSql

The preparable sql to insert a document into the database.

insertMDSql

public static final java.lang.String insertMDSql

The preparable sql to insert a document metadata into the database.

Constructor Detail

BulkLoader

public BulkLoader()

Constructor a BulkLoader without command-line arguments.

BulkLoader

public BulkLoader(java.lang.String[] args)
           throws java.lang.IllegalArgumentException

Construct a BulkLoader from the given command-line arguments.

Throws:

java.lang.IllegalArgumentException - thrown on invalid args

See Also:

parseArgs(java.lang.String[])

Method Detail

accept

public boolean accept(java.io.File dir,
                      java.lang.String name)

Implement the FilenameFilter interface method to use our match and ignore lists.

Specified by:

accept in interface java.io.FilenameFilter

parseArgs

public void parseArgs(java.lang.String[] args)
               throws java.lang.IllegalArgumentException

Parse the given input arguments.

Parameters:

args - the input arguments.

Throws:

BulkLoader.ShowUsageException - thrown if the caller should show a usage report.

java.lang.IllegalArgumentException - thrown on bad arguments.

usage

public void usage(java.io.PrintStream out)

Print the usage of the application.

usage

public void usage(java.io.PrintWriter out)

Print the usage of the application.

validateArgs

public void validateArgs()
                  throws java.lang.IllegalStateException

Validate that we have been passed correct arguments.

This does not validate that the arguments are valid. That will be done in initialize().

initialize

public void initialize()
                throws java.sql.SQLException,
                       java.lang.IllegalStateException

Initialize the bulkloader from the current state.

This calls setJDBCInfo() and then creates a JDBC connection.

Throws:

java.sql.SQLException - thrown on an error connecting to the database.

java.lang.IllegalStateException - thrown on an initialization error.

See Also:

#setJDBCInfo

go

public void go()
        throws java.sql.SQLException,
               java.io.IOException

Let the bulkloader go.

Throws:

java.sql.SQLException - if the load/cleanup fails.

java.io.IOException - if printing the schema fails.

shutdown

public void shutdown()

Shutdown this bulk loader.

doLoad

public void doLoad()
            throws java.sql.SQLException

Do the actual bulk load logic on the file list.

doLoad

public void doLoad(java.io.File baseDir,
                   java.lang.String path,
                   java.util.Properties dirProps)
            throws java.sql.SQLException

Load the given path into the database.

If path is a directory, all files underneath it that match our patterns will be included. If path is a file, it will be loaded.

Parameters:

baseDir - the base directory (can be used to get absolute file paths).

path - the path to the file or directory (this can be multi-part, not just name).

dirProps - the base md properties for file (this should be a clone this method can modify as needed).

Throws:

java.sql.SQLException - thrown on a database error.

printSchema

public void printSchema()
                 throws java.io.IOException

Print the schema xml for all the metadata we've loaded so far.

Throws:

java.io.IOException - thrown on an error outputting the schema xml.

printSchema

public void printSchema(java.io.PrintWriter out,
                        java.lang.String enc)
                 throws java.io.IOException

Print the schema xml for all the metadata we've loaded so far to the given output stream.

Parameters:

out - the output stream.

enc - the file encoding (will go in xml head if not null).

Throws:

java.io.IOException - thrown on an I/O error.

getMetadataProperties

public java.util.Properties getMetadataProperties(java.io.File base,
                                                  java.util.Properties p)
                                           throws java.io.IOException

Get the metadata properties for the given file or directory.

This does not do a META data parse.

Parameters:

base - the file or directory base path.

p - the properties to load into (null to create new).

Returns:

the properties (p if p was not null).

Throws:

java.io.IOException - on an error reading the properties file.

deleteDoc

public void deleteDoc(java.lang.String path)
               throws java.sql.SQLException

This will remove the document with id of path from the database, including all of its implicit metadata.

Parameters:

path - the document path

Throws:

java.sql.SQLException - thrown on a database error.

insertDoc

public void insertDoc(java.lang.String path,
                      java.io.File f,
                      java.util.Properties p,
                      java.lang.String mimeType)
               throws java.sql.SQLException

Insert a document and metadata into the database.

Parameters:

path - the document path id.

f - the file of the document (can be null).

p - the implicit properties.

mimeType - the preferred mime type of the document (null for default).

Throws:

java.sql.SQLException - thrown on a database error.

commit

public void commit()

Commit the transaction.

rollback

public void rollback()

Rollback the transaction.

shouldInclude

public boolean shouldInclude(java.lang.String name)

Tell if the loader should include the specified file name.

shouldIgnore

public boolean shouldIgnore(java.lang.String name)

Tell if the loader should ignore the specified file name.

isHtmlFile

public boolean isHtmlFile(java.lang.String name)

Tell if the specified file name is an HTML file to the loader.

fixPath

public java.lang.String fixPath(java.lang.String path)

Fix up a path to be forward-slash style and to not have empty path parts.

debug

public void debug(java.lang.String mesg)

Out put a debug message.

Subclasses can override this method to change where messages go.

warning

public void warning(java.lang.String mesg,
                    java.lang.Throwable ex)

Output a warning message.

Subclasses can override this method to change where messages go.

warning

public void warning(java.lang.String mesg)

Output a warning message.

error

public void error(java.lang.String mesg,
                  java.lang.Throwable ex)

Output an error message.

Subclasses can override this method to change where messages go.

error

public void error(java.lang.String mesg)

Output an error message.

isReadableDirectory

public static boolean isReadableDirectory(java.lang.String name)

Check if the specified file name is a directory that we can get into.

isHidden

public static boolean isHidden(java.io.File f)

Check if the specified file is a hidden file.

Under UNIX, the File.isHidden() reports that "/weblogicCommerce/dmsBase/." is a hidden file, which it is not. So, this fixes that problem by getting canonicals paths for directories before calling isHidden(). That seems to do the trick.

splitToProperties

public static java.util.Properties splitToProperties(java.lang.String str,
                                                     java.lang.String on)

Split a WLS connection pool style string in a Properties object of the name=value pairs.

close

public static void close(java.lang.Object o)

Close an object which has a close() method, ignoring any exceptions.

main

public static int main(BulkLoader loader,
                       java.lang.String[] args)

The main method invoked on a BulkLoader instance.

This will take a BulkLoader through the bulk loading steps. Output will be sent via the BulkLoader's debug(), warning(), and error() methods.

This will not call System.exit().

Parameters:

args - the command-line args.

Returns:

the exit code (0 for success, non-zero for failure).

See Also:

parseArgs(java.lang.String[]), validateArgs(), initialize(), go(), commit()

main

public static void main(java.lang.String[] args)

Command-line entry point.

This will call System.exit() on invalid args or error. To invoke a bulk load from your own code, create and manipulate a BulkLoader object. You can use the other main method, which does not exit.

Parameters:

args - the command-line args.

See Also:

main(BulkLoader, String[])