|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object atg.nucleus.logging.VariableArgumentApplicationLoggingImpl atg.nucleus.GenericService atg.repository.search.indexing.LoaderImpl
public class LoaderImpl
The base class for the bulk loader and the incremental loader. This class implements recursing through the items (calling the various customizer classes when appropriate), gathering property values, generating documents, and sending them to the indexer for indexing.
The loader works primarily from an IndexingOutputConfig
, which
specifies the repository, the repository items of interest, their
properties, and any customizing classes to use.
The basic algorythm for generating the documents is a recursive
breadth-first traversal of the hierarchy of repository items defined in
the XML definition file. As the loader traverses down the hierarchy it
pushes parent repository items onto the Context
object. In each
case, it uses PropertyAccessor#getMetaPropertyValue
to retrieve the child item property.
For a bulk/full load (see BulkLoaderImpl
), the loader performs a
query for top-level repository items, then iterates through any sub-items
specified, until it reaches a document-level item.
For an incremental load (see IncrementalLoaderImpl
), the loader starts
from an item, looks from parents, building Context stack as it traverses
up the hierarchy. Once it has a complete set of parents, it pops off any
sub-document level items, then then generates a document for the
document-level item.
In either case (bulk or incremental), once a document-level repository item is encountered, the loader begins collecting property values. It first collects any property values specified for parents (or other not document branches of the item hierarchy), and then continues the breadth-first traversal, collecting property values as it goes.
The loader then begins generating the XML document, once all the property
values are collected. It uses any specified PropertyFormatter
s and
PropertyValuesFilter
s to convert the collected property values
into string values.
Once the document has been created, the Loader then uses the DocumentSubmitter
to send the output to the indexer for indexing.
Typically the submitter will be an instance of the atg.repository.search.indexing.submitter.BpiDocumentSubmitter
, but
other submitters may be used for debugging.
The order of customizer class invocation is:
VariantProducer
(once per document, will be called again
if returns true)
PropertyAccessor
(once per item property,
once per text and meta property)
PropertyFormatter
(invoked on results of PropertyAccessor property
values). Maybe called before or after a given filter. Invoked
after property values have all been collected.
PropertyValuesFilter
(invoked after all property values are
gather). May be called before or after a give formatter. Invoked
after property values have all been collected.
Created: March 30 2005
Field Summary | |
---|---|
static java.lang.String |
CLASS_VERSION
Class version string |
Fields inherited from class atg.nucleus.GenericService |
---|
SERVICE_INFO_KEY |
Fields inherited from interface atg.nucleus.logging.TraceApplicationLogging |
---|
DEFAULT_LOG_TRACE_STATUS |
Fields inherited from interface atg.nucleus.logging.ApplicationLogging |
---|
DEFAULT_LOG_DEBUG_STATUS, DEFAULT_LOG_ERROR_STATUS, DEFAULT_LOG_INFO_STATUS, DEFAULT_LOG_WARNING_STATUS |
Constructor Summary | |
---|---|
LoaderImpl()
|
Method Summary | |
---|---|
void |
addActiveContext(Context pContext)
Add a Context to the list of active contexts. |
protected java.lang.String |
addUniqueParamsToURL(java.lang.String pBaseURL,
java.util.Map pMapUniqueParams)
Add any unique params to the URL. |
protected void |
afterSessionStart(Context pContext)
This method is invoked immediately after a DocumentSubmitter session is started. |
protected void |
beforeSessionEnd(Context pContext)
This method is invoked immediately before a DocumentSubmitter session is closed. |
boolean |
cancelActiveContext(java.lang.String pConfigurationPath)
Cancel the active context with the specified name. |
protected int |
claimGeneration(java.lang.String pConfigPath,
boolean pIncremental)
Claim a generation for updating. |
protected javax.servlet.Servlet |
createAdminServlet()
Create the admininistration servlet. |
protected Context |
createContext()
Create a new context. |
protected OutputDocumentContent |
createEmptyOutputDocumentContent()
Create a new, empty, OutputDocumentContent object. |
protected void |
deleteDocumentItem(RepositoryItem pItem,
Context pContext)
Remove the specified repository item pItem from the index. |
void |
doStopService()
When doStopService is called, exit out of our update activity thread, if any. |
ContextStatus[] |
getActiveContextStatuses()
Get an array of ContextStatuses... |
protected RepositoryItem[] |
getArrayFromMultiRepoItems(java.lang.Object pItems)
Return an array of RepositoryItems representing the repository items the come from the property pItem. |
ConfigStatePersister |
getConfigStatePersister()
Get the configuration state persister. |
int |
getDocumentsPerTransaction()
Get the number of documents per transaction. |
int |
getLoggingInfoStatusCount()
Get how often to print out count/progress status information. |
long |
getUpdateActivityTimeMillis()
Get the number of milliseconds to wait between updates to lastActivity time. |
boolean |
getUpdateLastActivityTimeInSeparateThread()
Whether to update last activity time in a separate thread. |
protected void |
initContext(Context pContext,
IndexingOutputConfig pConfig,
int pClaimedGeneration)
Initialize the context, using the specified parameter values. |
boolean |
isEchoDocumentsToStdout()
Whether to echo documents to stdout. |
boolean |
isPrettyPrint()
Get whether the XHTML should be pretty printed (that is, indented). |
boolean |
isPropertyLoggingDebug()
Whether to turn on logging debug for invidual property values. |
static boolean |
isSubTypeOfItemDesc(RepositoryItemDescriptor pItemDesc,
RepositoryItemDescriptor pSuper)
Returns whether pItemDesc is a subtype (or equal to) pSuper. |
boolean |
isTransactionPerDocument()
Whether to wrap all repository calls within a document in a transaction. |
atg.repository.search.indexing.LoaderImpl.UpdateActivityInfo |
nextQueuedUpdateActivity()
Return the next queued item for an update activity. |
protected void |
outputAndSubmitDocument(Context pContext)
Create the XHTML file for the Context's OutputDocumentContent and submit it using the DocumentSubmitterSession. |
void |
processItem(Context pContext,
RepositoryItem pItem,
OutputItemSpecifier pSpecifier)
Process the specified item. |
protected void |
processParentProperties(Context pContext)
Process any of the properties of containing items on the the context's stack. |
protected void |
processProperties(Context pContext,
RepositoryItem pItem,
OutputItemSpecifier pSpecifier,
boolean pIsMeta)
Process any properties: that is collect any property values. |
protected void |
processProperties(Context pContext,
RepositoryItem pItem,
OutputItemSpecifier pSpecifier,
OutputProperty[] pProperties,
boolean pIsMeta)
Process any properties: that is collect any property values. |
void |
queueUpdateActivity(atg.repository.search.indexing.LoaderImpl.UpdateActivityInfo pUpdateActivityInfo)
Queue an last activity time update change request. |
protected void |
releaseContext(Context pContext)
Release any resources claimed by the specified context. |
protected void |
releaseGeneration(java.lang.String pConfigPath,
int pGeneration,
boolean pCompletedSuccessfully,
boolean pIncremental)
Release the specified generation. |
void |
removeActiveContext(Context pContext)
Remove a Context from the list of active contexts. |
protected void |
resetContextAfterDocument(Context pContext)
Reset the Context after a generating a document. |
void |
setConfigStatePersister(ConfigStatePersister pConfigStatePersister)
Set the configuration state persister. |
void |
setDocumentsPerTransaction(int pDocumentsPerTransaction)
Set the number of documents per transaction. |
void |
setEchoDocumentsToStdout(boolean pEchoDocumentsToStdout)
Whether to echo documents to stdout. |
void |
setLoggingInfoStatusCount(int pLoggingInfoStatusCount)
Set how often to print out count/progress status information. |
void |
setPrettyPrint(boolean pPrettyPrint)
Set whether the XHTML should be pretty printed (that is, indented). |
void |
setPropertyLoggingDebug(boolean pPropertyLoggingDebug)
Whether to turn on logging debug for invidual property values. |
void |
setTransactionPerDocument(boolean pTransactionPerDocument)
Whether to wrap all repository calls within a document in a transaction. |
protected void |
setupContextForNewDocument(Context pContext,
RepositoryItem pItem,
java.util.Map pMapParams)
Set up our context for a new document representing pItem. |
void |
setUpdateActivityTimeMillis(long pUpdateActivityTimeMillis)
Set the number of milliseconds to wait between updates to lastActivity time. |
void |
setUpdateLastActivityTimeInSeparateThread(boolean pUpdateLastActivityTimeInSeparateThread)
Whether to update last activity time in a separate thread. |
protected java.lang.Object[] |
splitPropertyValue(OutputProperty pOutputProperty,
java.lang.Object pValue)
Split the property value into multiple values, if appropriate. |
void |
updateActivity(Context pContext)
Update our last activity time. |
Methods inherited from class atg.nucleus.logging.VariableArgumentApplicationLoggingImpl |
---|
vlogDebug, vlogDebug, vlogDebug, vlogDebug, vlogError, vlogError, vlogError, vlogError, vlogInfo, vlogInfo, vlogInfo, vlogInfo, vlogTrace, vlogTrace, vlogTrace, vlogTrace, vlogWarning, vlogWarning, vlogWarning, vlogWarning |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static final java.lang.String CLASS_VERSION
Constructor Detail |
---|
public LoaderImpl()
Method Detail |
---|
public void setConfigStatePersister(ConfigStatePersister pConfigStatePersister)
public ConfigStatePersister getConfigStatePersister()
public void setUpdateActivityTimeMillis(long pUpdateActivityTimeMillis)
public long getUpdateActivityTimeMillis()
public void setEchoDocumentsToStdout(boolean pEchoDocumentsToStdout)
public boolean isEchoDocumentsToStdout()
public void setPropertyLoggingDebug(boolean pPropertyLoggingDebug)
public boolean isPropertyLoggingDebug()
public void setTransactionPerDocument(boolean pTransactionPerDocument)
public boolean isTransactionPerDocument()
public void setLoggingInfoStatusCount(int pLoggingInfoStatusCount)
public int getLoggingInfoStatusCount()
public void setPrettyPrint(boolean pPrettyPrint)
public boolean isPrettyPrint()
public void setDocumentsPerTransaction(int pDocumentsPerTransaction)
public int getDocumentsPerTransaction()
public void setUpdateLastActivityTimeInSeparateThread(boolean pUpdateLastActivityTimeInSeparateThread)
public boolean getUpdateLastActivityTimeInSeparateThread()
public void processItem(Context pContext, RepositoryItem pItem, OutputItemSpecifier pSpecifier) throws IndexingException
pContext
- the indexing context object.pRepositoryItem
- the repository item that corresponds to this item.pSpecifier
- the specifier for this item.
IndexingException
public static boolean isSubTypeOfItemDesc(RepositoryItemDescriptor pItemDesc, RepositoryItemDescriptor pSuper)
pItemDesc
- The (presumably child) item descriporpSuper
- The item descriptor to test for parent-hood.
public void updateActivity(Context pContext) throws IndexingException
IndexingException
protected RepositoryItem[] getArrayFromMultiRepoItems(java.lang.Object pItems)
pItems
- the object from which to extract repository
items
protected void outputAndSubmitDocument(Context pContext) throws IndexingException
IndexingException
protected void setupContextForNewDocument(Context pContext, RepositoryItem pItem, java.util.Map pMapParams)
pContext
- the context to be updatedpItem
- the document-level item used to update pContextpMapParams
- the map of distinguising keys and values that
will be appened as query params to the document URL. Both
keys and values must be strings.protected java.lang.String addUniqueParamsToURL(java.lang.String pBaseURL, java.util.Map pMapUniqueParams)
pBaseURL
- the base URL... typically an atgrepo: URL.pMapUniqueParams
- the map of unique paramaters. This
should be an ordered map for consistency sake.protected void resetContextAfterDocument(Context pContext)
protected OutputDocumentContent createEmptyOutputDocumentContent()
protected void processParentProperties(Context pContext) throws IndexingException
pContext
- the context, currently set to the document
level.
IndexingException
protected void processProperties(Context pContext, RepositoryItem pItem, OutputItemSpecifier pSpecifier, boolean pIsMeta)
pItem
. Gets the static set of
properties of pSpecifier
, and possibly additional
property values from the PropertyAccessor, if instance of
AdditionalPropertiesPropertyAccessor.
pContext
- the current contextpItem
- the current item whose property values to collectpSpecifier
- the specifier corresponding to pItem
pIsMeta
- whether these properties are meta propertiesprotected void processProperties(Context pContext, RepositoryItem pItem, OutputItemSpecifier pSpecifier, OutputProperty[] pProperties, boolean pIsMeta)
pItem
.
pContext
- the current contextpItem
- the current item whose property values to collectpSpecifier
- the specifier corresponding to pItem
pProperties
- the array of output properties to collectpIsMeta
- whether these properties are meta propertiesprotected java.lang.Object[] splitPropertyValue(OutputProperty pOutputProperty, java.lang.Object pValue)
pOutputProperty
- the output property the corresponds
to this value.pValue
- the property value
protected Context createContext()
protected void initContext(Context pContext, IndexingOutputConfig pConfig, int pClaimedGeneration)
pContext
- the context to initializepSession
- the document submitter sessionpConfig
- the indexing output configpClaimedGeneration
- the claimed generation for this
update.protected void releaseContext(Context pContext)
pContext
- The context to release.protected int claimGeneration(java.lang.String pConfigPath, boolean pIncremental) throws IndexingException
ConfigStatePersister
to claim a given generation
for updating.
pConfigPath
- the configuration path for for
indexing output config whose index shoudl be udpated.pIncremental
- whether we are claiming this
generation for an incremental update.
IndexingException
protected void releaseGeneration(java.lang.String pConfigPath, int pGeneration, boolean pCompletedSuccessfully, boolean pIncremental) throws IndexingException
ConfigStatePersister
to release the specified generation.
pConfigPath
- The configuration path of the generationpGeneration
- the numeric id of the generation, previously
returned by claimedGeneration.pCompletedSuccessfully
- whether the generation
completed successfully.pIncremental
- whether the generation is an
incremental generation.
IndexingException
public void addActiveContext(Context pContext)
pContext
- the context to add.public void removeActiveContext(Context pContext)
pContext
- the context to remove.public boolean cancelActiveContext(java.lang.String pConfigurationPath)
public ContextStatus[] getActiveContextStatuses()
protected javax.servlet.Servlet createAdminServlet()
createAdminServlet
in class GenericService
protected void deleteDocumentItem(RepositoryItem pItem, Context pContext) throws IndexingException
IndexingException
protected void afterSessionStart(Context pContext) throws IndexingException
pContext
- the Context for the current indexing operation. The
DocumentSubmitterSession
is available as a property, as is
the IndexingOutputConfig
and other state information.
IndexingException
protected void beforeSessionEnd(Context pContext) throws IndexingException
pContext
- the Context for the current indexing operation. The
DocumentSubmitterSession
is available as a property, as is
the IndexingOutputConfig
and other state information.
IndexingException
public void queueUpdateActivity(atg.repository.search.indexing.LoaderImpl.UpdateActivityInfo pUpdateActivityInfo)
public atg.repository.search.indexing.LoaderImpl.UpdateActivityInfo nextQueuedUpdateActivity()
public void doStopService() throws ServiceException
doStopService
in class GenericService
ServiceException
- if the Service had a problem shutting down
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |