Documaker Source Configuration

You can configure Documaker Server, as discussed previously, to write records to an Archive of Record (AOR) database table and write documents such as PDF files into a disk directory. This source component works with that configuration and performs these tasks:

Reads records from the database table.
Passes the metadata and document data to the Documaker Connector destination you specified.
Posts status updates into the database table to note the progress of the archive process and then later to mark it as complete.

Note:

This component is not used with Documaker Document Factory and the Archiver, which supply their own document source.

The Documaker source is configured to make a Java DataBase Connectivity (JDBC) connection to an SQL database table. The Documaker DAL scripts provided in the banner processing configuration for Documaker write a record into this table for each document. The Documaker source queries the table, as described in Configuring Documaker. Each record’s information is then passed through Documaker Connector to the configured destination and the destination is told to import the document. The status result is returned to the Documaker source, so it can update the database table.

Since you can configure the Connector application to create multiple channels between the source and destination, the Documaker source can create a pool of database connections from which each channel can draw a connection as needed.

A connection is used to query the document records and then later to set the status results after the import is complete. For optimal performance, the number of channels and database connections should be the same. Most of the Documaker source configuration properties set the characteristics and behavior of this database connection pool.

Configure the use of the Documaker source with:

source.name=oracle.documaker.ecmconnector.documakersource.DocumakerSource
periodic.process=oracle.documaker.ecmconnector.documakersource.DocumakerSourceProcess

Table 3-5 Documaker Source Configuration Properties

Property Name	Description	Default
.driver.name	The JDBC driver name to use.
.url	The JDBC database URL.
.username	The JDBC account name used for authentication to the database.
.password	The JDBC password used for authentication to the database. See Handling Passwords in the Property File for more information.
.table	Sets the JDBC database table name.
.timeout.seconds	The JDBC connection timeout interval, in seconds.	0
.connection.property.name	The JDBC custom connection property‘s name. Used for connections to databases which require connection properties beyond those listed above. Each value is passed to JDBC paired with each name listed.	none
Database Connection Performance Tuning Properties
.pool.size	The number of database connections to create in a pool.	25
.max.active.connections	The maximum active connections in the connection pool.	50
.min.idle.connections	The minimum idle connections in the connection pool.	15
.max.idle.connections	The maximum number of database connections left idle in the pool. Additional idle connections are closed.	25
max.wait.seconds	The maximum number of seconds to wait for a connection from the connection pool if one is not immediately available.	30
.max.open.prepared.statements	The maximum number of opened prepared SQL statements to keep for the connection pool.	15
.min.evictable.idle.time.millis	The number of milliseconds a connection must be idle before it is cleaned up.	60000 (10 minutes)
.time.between.eviction.runs.millis	The number of milliseconds between idle connection cleanup.	300000 (5 minutes)
.tests.per.eviction.run	The maximum number of connections tested for idle each iteration.	25
.test.on.borrow	Indicates whether to test connection objects when obtaining a connection.	True
.test.while.idle	Indicates whether to test connection objects while idle.	True
.validation.query	The query string used to test if a connection is alive. This query is made if the source.documaker.test.on.borrow property is True. A default string is created if this property is left empty. The default refers to the source.documaker.table attribute shown in the Database Connection Properties table. If, however, a query string is provided, it is used without modification by the Documaker Source.	select count(1) from tableName
.max.records	The maximum number of records that should be retrieved from the Documaker AOR table as a group to process.	One (1)
.check.length	If True, the source validates the length of the data retrieved for each column before passing that data to the destination. The length of the data retrieved is checked against the length of the destination metadata for that column and if the length of the data exceeds the space available, a LengthCheckException is thrown.	False
.retention.time	The number of days an imported record will stay in the AOR table before it is removed.	none
.excluded.columns	A comma-separated list of columns in the Documaker AOR table that should be hidden from the destination and not be available as document metadata.	none
.contents.columnname	The name of the column in the AOR table that contains the document contents, if the ..include.file.contents property is True. none	none
.row.identifier	The name of the column in the AOR table that contains the row identifier, if the ..include.file.contents property is True.	none

The keys to this configuration are the JDBC settings that link Documaker Connector to the database table written by Documaker. The driver name is the class name of a class in the program's CLASSPATH. The Oracle driver, for example, is distributed in the ojdbc14.jar file and is chosen by configuring: source.documaker.driver.name=oracle.jdbc.driver.OracleDriver

The driver is directed to the database host with a URL:

source.documaker.url=jdbc:oracle:thin:@localhost:1521:xe

The URL always starts with jdbc: indicating the protocol used. This is followed by a DBMS-indicating preface, such as

oracle:
mysql:
microsoft:sqlserver:
db2:
jtds:sqlserver:

The remainder of the URL format is controlled by the specific DBMS driver. For Oracle, the remainder is represented by this format:

driver-type:[username/password]@database_specifier

Parameter Description

driver-type Choose thin, oci, or kprb. This affects the format for the database_specifier property.

username/password

Parameter	Description
driver-type	Choose thin, oci, or kprb. This affects the format for the database_specifier property.
username/password	(Optional) Documaker Connector expects this information as separate configuration items: `source.documaker.username=username source.documaker.password=password` See Handling Passwords in the Property File for more information.
database_specifier	For thin, you can use a string like this one: `@//host_name:port_number/service_name` This connects to an Oracle XE database running on the same computer as Documaker Connector using port 1521. You can find more information on the Oracle JDBC product on the Oracle Technology Network. You can find information for the URL format for your database in your DBMS documentation or on the Internet.

(Optional) Documaker Connector expects this information as separate configuration items:

source.documaker.username=username
source.documaker.password=password

See Handling Passwords in the Property File for more information.

database_specifier

For thin, you can use a string like this one:

@//host_name:port_number/service_name

This connects to an Oracle XE database running on the same computer as Documaker Connector using port 1521. You can find more information on the Oracle JDBC product on the Oracle Technology Network. You can find information for the URL format for your database in your DBMS documentation or on the Internet.

Configure the database table name as shown here: source.documaker.table=tablename

The tablename, as well as column names specified elsewhere, can be case-sensitive or case-blind depending on the database settings. One list of column names is provided in the excluded.columns property. These columns are hidden from the destination and cannot be used as document metadata. Generally, these columns are used to manage the table content by Documaker or the Documaker Source: source.documaker.excluded.columns=column, column, ...

The Documaker Source opens a pool of multiple connections to the database. This pool is shared across all of the instances of the Documaker Source you set up in the Connector parameter:

channel.count=number_of_instances

The size and behavior of this pool are controlled by several properties. The pool can grow to a maximum number of connections given by source.documaker.max.active.connections=max_number_of_connections

pool starts with the number given in this property:

source.documaker.pool.size=initial_number_of_connections

If the pool is at its maximum size, a request for a connection will have to wait. The longest it will wait before failing and causing an error is set here:

source.documaker.max.wait.seconds=wait_seconds

Once the pool is created and connections begin to be used, additional connections are opened and added to the idle pool if the number of idle connections drops below the configured minimum, but only up to the maximum count given above as source.documaker.max.active.connections. Specify the minimum number of idle connections here:

source.documaker.min.idle.connections=desired_idle_connections

As connections become idle, they remain in the pool until the number of idle connections rises above the configured maximum: source.documaker.max.idle.connections=max_idle_connections The pool is managed by a periodic process which checks these numbers. This process is called eviction and runs at a specified interval. At each run, Documaker Connector checks the idle connections to see if they have been idle long enough to be eligible for eviction from the pool — closed. That minimum time is also configurable.

source.documaker.time.between.eviction.runs.millis=eviction_interval_milliseconds
source.documaker.min.evictable.idle.time.millis=evict_min_idle_milliseconds

While connections are sitting idle in the pool, they can become stale if there is a network problem or the DBMS chooses to close the connection due to a time-out setting, for example. The Documaker Connector Source can maintain these connections and avoid timeout intervals by periodically running test transactions during eviction processing. Because this can be time-consuming, especially if the idle pool is large, you can limit it to a few connections per interval, rather than doing all of them every time:

source.documaker.test.while.idle={true | false} 
source.documaker.tests.per.eviction.run=number_of_connections_ to_test

Similarly, before a connection is pulled from the pool to use, it can be tested if this property is set: source.documaker.test.on.borrow={true | false}

The transaction that is run to check the connection is usually named SELECT 1 FROM tablename. You can select a different transaction by setting this property:

source.documaker.validation.query=SQL_query