|
BEA Systems, Inc. | ||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: INNER | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object | +--com.beasys.commerce.axiom.document.loader.BulkLoader
The reference document repository bulk loader application.
This class is mainly designed to run as a command-line application, via a "java com.beasys.commerce.axiom.document.loader.BulkLoader" command-line. To see a usage, give it a -h flag or read the Usage.txt in this package.
Additionally, BulkLoader objects can be created and used to provide the functionality in other places. The lifecycle of a BulkLoader is as follows:
parseArgs()
if not passed in to constructor.
validateArgs()
to make sure the loader has
valid arguments.
initialize()
to setup internal state.
go()
to execute the load.
shutdown()
to cleanup everything.
If manually constructing and utilizing a BulkLoader object, be certain to synchronize all access to the object. Since the command-line program is single-threaded, BulkLoader objects are not thread-safe by design.
To load the default LoaderFilters
, the BulkLoader
looks for the com/beasys/commerce/axiom/document/loader/loader.properties
file in the CLASSPATH. From that it reads the list of default LoaderFilter
class names from the "loader.defFilters" property. To not use any of the
default filters, specify +filters in the command-line args.
MetaParser
,
FileCache
Inner Class Summary | |
static interface |
BulkLoader.ShowUsageException
Quick inner exception thrown on parseArgs() to say we should just print a usage report. |
Field Summary | |
protected java.sql.Connection |
connection
The connection this loader uses. |
java.lang.String |
conPoolName
The connection pool name. |
static java.lang.String |
DEF_MD_FILE_EXT
The default file extension for metadata property files. |
static java.lang.String |
DEF_MIME_TYPE
The default mime type. |
static java.lang.String |
DEF_SCHEMA_NAME
The default schema name. |
static java.lang.String |
DEF_SCHEMA_PATH
The default path for the schema file. |
static java.lang.String |
DEF_WLS_PROPS_PATH
The default weblogic properties file path. |
static java.lang.String |
deleteDocSql
The preparable sql to remove a document from the database. |
protected java.sql.PreparedStatement |
deleteDocStmt
The delete document statement. |
static java.lang.String |
deleteMDSql
The preparable sql to remove a document's implicit metadata from the database. |
protected java.sql.PreparedStatement |
deleteMDStmt
The delete metadata statement. |
static java.lang.String |
DOC_MD_TABLE
The wlcs_document_metadata table name. |
static java.lang.String |
DOC_TABLE
The wlcs_document table name. |
java.lang.String |
docBase
The docBase. |
boolean |
doCleanUp
Are we supposed to do a cleanup. |
boolean |
doDelete
Are we deleting entries. |
boolean |
doMetaParse
Are we supposed to parse '*.htm' and '*.html' files for META tags. |
java.lang.String |
fileEncoding
The file enconding (null for VM default). |
java.util.List |
fileList
The list of files/directories to scan over. |
java.util.List |
htmlMatchList
The list of patterns that represent HTML file names. |
java.util.List |
ignoreList
The list of file name patterns to ignore. |
boolean |
includeHidden
Are we supposed to include hidden files and directories. |
boolean |
inheritProps
Are we supposed to inherit metadata properties when recursing directories? |
static java.lang.String |
insertDocSql
The preparable sql to insert a document into the database. |
protected java.sql.PreparedStatement |
insertDocStmt
The insert document statement. |
static java.lang.String |
insertMDSql
The preparable sql to insert a document metadata into the database. |
protected java.sql.PreparedStatement |
insertMDStmt
The insert document metadata statement. |
protected java.util.Properties |
jdbcProps
The JDBC connection properties. |
protected java.lang.String |
jdbcUrl
The JDBC connection url. |
java.util.List |
loaderFilters
The list of LoaderFilters to try. |
java.util.List |
matchList
The list of file name patterns to include. |
java.lang.String |
mdFileExt
The file extension of metadata property files. |
protected java.util.Collection |
metadataNames
The metadata properties we find along the way. |
boolean |
recurse
Do we recurse over directories? |
java.lang.String |
schemaName
The name of the schema in the schema file. |
java.lang.String |
schemaPath
The path to the schema file to output. |
boolean |
testOnly
Are we running in test mode? |
boolean |
verbose
Do we spew out messages. |
java.lang.String |
wlsPropsPath
The weblogic.properties file path. |
Constructor Summary | |
BulkLoader()
Constructor a BulkLoader without command-line arguments. |
|
BulkLoader(java.lang.String[] args)
Construct a BulkLoader from the given command-line arguments. |
Method Summary | |
boolean |
accept(java.io.File dir,
java.lang.String name)
Implement the FilenameFilter interface method to use our match and ignore lists. |
static void |
close(java.lang.Object o)
Close an object which has a close() method, ignoring any exceptions. |
void |
commit()
Commit the transaction. |
void |
debug(java.lang.String mesg)
Out put a debug message. |
void |
deleteDoc(java.lang.String path)
This will remove the document with id of path from the database, including all of its implicit metadata. |
void |
doLoad()
Do the actual bulk load logic on the file list. |
void |
doLoad(java.io.File baseDir,
java.lang.String path,
java.util.Properties dirProps)
Load the given path into the database. |
void |
error(java.lang.String mesg)
Output an error message. |
void |
error(java.lang.String mesg,
java.lang.Throwable ex)
Output an error message. |
protected void |
finalize()
Called when this to be finialized. |
java.lang.String |
fixPath(java.lang.String path)
Fix up a path to be forward-slash style and to not have empty path parts. |
static java.lang.String |
fixString(java.lang.String in)
Fix empty strings to be nulls. |
java.util.Properties |
getLoaderFilterProperties(java.io.File f,
java.util.Properties p)
Get the properties from the BulkLoader's LoaderFilters for the given file. |
java.util.Properties |
getMetadataProperties(java.io.File base,
java.util.Properties p)
Get the metadata properties for the given file or directory. |
void |
go()
Let the bulkloader go. |
void |
initialize()
Initialize the bulkloader from the current state. |
void |
insertDoc(java.lang.String path,
java.io.File f,
java.util.Properties p,
java.lang.String mimeType)
Insert a document and metadata into the database. |
static boolean |
isHidden(java.io.File f)
Check if the specified file is a hidden file. |
boolean |
isHtmlFile(java.lang.String name)
Tell if the specified file name is an HTML file to the loader. |
static boolean |
isReadableDirectory(java.lang.String name)
Check if the specified file name is a directory that we can get into. |
static int |
main(BulkLoader loader,
java.lang.String[] args)
The main method invoked on a BulkLoader instance. |
static void |
main(java.lang.String[] args)
Command-line entry point. |
void |
parseArgs(java.lang.String[] args)
Parse the given input arguments. |
void |
printSchema()
Print the schema xml for all the metadata we've loaded so far. |
void |
printSchema(java.io.PrintWriter out,
java.lang.String enc)
Print the schema xml for all the metadata we've loaded so far to the given output stream. |
void |
rollback()
Rollback the transaction. |
protected void |
setJDBCInfo()
Set the jdbc connection url and properties from the current weblogic.properties file. |
boolean |
shouldIgnore(java.lang.String name)
Tell if the loader should ignore the specified file name. |
boolean |
shouldInclude(java.lang.String name)
Tell if the loader should include the specified file name. |
void |
shutdown()
Shutdown this bulk loader. |
static java.util.Properties |
splitToProperties(java.lang.String str,
java.lang.String on)
Split a WLS connection pool style string in a Properties object of the name=value pairs. |
void |
usage(java.io.PrintStream out)
Print the usage of the application. |
void |
usage(java.io.PrintWriter out)
Print the usage of the application. |
void |
validateArgs()
Validate that we have been passed correct arguments. |
void |
warning(java.lang.String mesg)
Output a warning message. |
void |
warning(java.lang.String mesg,
java.lang.Throwable ex)
Output a warning message. |
Methods inherited from class java.lang.Object |
clone,
equals,
getClass,
hashCode,
notify,
notifyAll,
toString,
wait,
wait,
wait |
Field Detail |
public static java.lang.String DOC_TABLE
RefDocumentProvider.DOC_TABLE
public static java.lang.String DOC_MD_TABLE
RefDocumentProvider.DOC_MD_TABLE
public static final java.lang.String DEF_MD_FILE_EXT
public static final java.lang.String DEF_SCHEMA_PATH
public static final java.lang.String DEF_SCHEMA_NAME
public static final java.lang.String DEF_WLS_PROPS_PATH
public static final java.lang.String DEF_MIME_TYPE
public static final java.lang.String deleteMDSql
public static final java.lang.String deleteDocSql
public static final java.lang.String insertDocSql
public static final java.lang.String insertMDSql
public boolean verbose
public boolean testOnly
public boolean recurse
public boolean doDelete
public boolean doMetaParse
public boolean doCleanUp
public boolean includeHidden
public boolean inheritProps
public java.lang.String docBase
public java.lang.String mdFileExt
This should start with a ".".
public java.lang.String schemaPath
public java.lang.String schemaName
public java.lang.String fileEncoding
public java.util.List matchList
Empty to include all.
public java.util.List ignoreList
public java.util.List htmlMatchList
public java.util.List fileList
public java.lang.String wlsPropsPath
public java.lang.String conPoolName
public java.util.List loaderFilters
protected java.lang.String jdbcUrl
protected java.util.Properties jdbcProps
protected java.sql.Connection connection
protected java.sql.PreparedStatement deleteMDStmt
protected java.sql.PreparedStatement deleteDocStmt
protected java.sql.PreparedStatement insertDocStmt
protected java.sql.PreparedStatement insertMDStmt
protected java.util.Collection metadataNames
Constructor Detail |
public BulkLoader()
public BulkLoader(java.lang.String[] args) throws java.lang.IllegalArgumentException
parseArgs(java.lang.String[])
Method Detail |
protected void finalize() throws java.lang.Throwable
public boolean accept(java.io.File dir, java.lang.String name)
public void parseArgs(java.lang.String[] args) throws java.lang.IllegalArgumentException
args
- the input arguments.public void usage(java.io.PrintStream out)
public void usage(java.io.PrintWriter out)
public void validateArgs() throws java.lang.IllegalStateException
This does not validate that the arguments are valid. That will be done in initialize().
public void initialize() throws java.sql.SQLException, java.lang.IllegalStateException
This calls setJDBCInfo()
and then creates a
JDBC connection.
setJDBCInfo()
public void go() throws java.sql.SQLException, java.io.IOException
public void shutdown()
public void doLoad() throws java.sql.SQLException
public void doLoad(java.io.File baseDir, java.lang.String path, java.util.Properties dirProps) throws java.sql.SQLException
If path is a directory, all files underneath it that match our patterns will be included. If path is a file, it will be loaded.
baseDir
- the base directory (can be used to get absolute file
paths).path
- the path to the file or directory (this can be multi-part,
not just name).dirProps
- the base md properties for file (this should be a clone
this method can modify as needed).public void printSchema() throws java.io.IOException
public void printSchema(java.io.PrintWriter out, java.lang.String enc) throws java.io.IOException
out
- the output stream.enc
- the file encoding (will go in xml head if not null).public java.util.Properties getMetadataProperties(java.io.File base, java.util.Properties p) throws java.io.IOException
This does not do a META data parse.
base
- the file or directory base path.p
- the properties to load into (null to create new).public java.util.Properties getLoaderFilterProperties(java.io.File f, java.util.Properties p)
f
- the file.p
- the properties object to add to (null to create new one).public void deleteDoc(java.lang.String path) throws java.sql.SQLException
path
- the document pathpublic void insertDoc(java.lang.String path, java.io.File f, java.util.Properties p, java.lang.String mimeType) throws java.sql.SQLException
path
- the document path id.f
- the file of the document (can be null).p
- the implicit properties.mimeType
- the preferred mime type of the document (null for
default).protected void setJDBCInfo() throws java.lang.IllegalStateException
public void commit()
public void rollback()
public boolean shouldInclude(java.lang.String name)
public boolean shouldIgnore(java.lang.String name)
public boolean isHtmlFile(java.lang.String name)
public java.lang.String fixPath(java.lang.String path)
public void debug(java.lang.String mesg)
Subclasses can override this method to change where messages go.
public void warning(java.lang.String mesg, java.lang.Throwable ex)
Subclasses can override this method to change where messages go.
public void warning(java.lang.String mesg)
public void error(java.lang.String mesg, java.lang.Throwable ex)
Subclasses can override this method to change where messages go.
public void error(java.lang.String mesg)
public static boolean isReadableDirectory(java.lang.String name)
public static boolean isHidden(java.io.File f)
Under UNIX, the File.isHidden() reports that "/weblogicCommerce/dmsBase/." is a hidden file, which it is not. So, this fixes that problem by getting canonicals paths for directories before calling isHidden(). That seems to do the trick.
public static java.util.Properties splitToProperties(java.lang.String str, java.lang.String on)
public static java.lang.String fixString(java.lang.String in)
public static void close(java.lang.Object o)
public static int main(BulkLoader loader, java.lang.String[] args)
This will take a BulkLoader through the bulk loading steps. Output
will be sent via the BulkLoader's debug()
,
warning()
, and error()
methods.
This will not call System.exit().
args
- the command-line args.parseArgs(java.lang.String[])
,
validateArgs()
,
initialize()
,
go()
,
commit()
public static void main(java.lang.String[] args)
This will call System.exit() on invalid args or error. To invoke a bulk load from your own code, create and manipulate a BulkLoader object. You can use the other main method, which does not exit.
args
- the command-line args.main(BulkLoader, String[])
|
BEA Systems, Inc. | ||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: INNER | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |