|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectatg.nucleus.logging.VariableArgumentApplicationLoggingImpl
atg.nucleus.GenericService
atg.repository.search.indexing.LoaderImpl
public class LoaderImpl
The base class for the bulk loader and the incremental loader. This class implements recursing through the items (calling the various customizer classes when appropriate), gathering property values, generating documents, and sending them to the indexer for indexing.
The loader works primarily from an IndexingOutputConfig
, which
specifies the repository, the repository items of interest, their
properties, and any customizing classes to use.
The basic algorythm for generating the documents is a recursive
breadth-first traversal of the hierarchy of repository items defined in
the XML definition file. As the loader traverses down the hierarchy it
pushes parent repository items onto the Context
object. In each
case, it uses PropertyAccessor.getMetaPropertyValue(atg.repository.search.indexing.Context, atg.repository.RepositoryItem, java.lang.String, atg.repository.search.indexing.specifier.PropertyTypeEnum)
to retrieve the child item property.
For a bulk/full load (see BulkLoaderImpl
), the loader performs a
query for top-level repository items, then iterates through any sub-items
specified, until it reaches a document-level item.
For an incremental load (see IncrementalLoaderImpl
), the loader starts
from an item, looks from parents, building Context stack as it traverses
up the hierarchy. Once it has a complete set of parents, it pops off any
sub-document level items, then then generates a document for the
document-level item.
In either case (bulk or incremental), once a document-level repository item is encountered, the loader begins collecting property values. It first collects any property values specified for parents (or other not document branches of the item hierarchy), and then continues the breadth-first traversal, collecting property values as it goes.
The loader then begins generating the XML document, once all the property
values are collected. It uses any specified PropertyFormatter
s and
PropertyValuesFilter
s to convert the collected property values
into string values.
Once the document has been created, the Loader then uses the DocumentSubmitter
to send the output to the indexer for indexing.
Typically the submitter will be an instance of the atg.repository.search.indexing.submitter.BpiDocumentSubmitter
, but
other submitters may be used for debugging.
The order of customizer class invocation is:
VariantProducer
(once per document, will be called again
if returns true)
PropertyAccessor
(once per item property,
once per text and meta property)
PropertyFormatter
(invoked on results of PropertyAccessor property
values). Maybe called before or after a given filter. Invoked
after property values have all been collected.
PropertyValuesFilter
(invoked after all property values are
gather). May be called before or after a give formatter. Invoked
after property values have all been collected.
Created: March 30 2005
Field Summary | |
---|---|
static java.lang.String |
CLASS_VERSION
Class version string |
protected static java.lang.String |
DELETE_DOCUMENT
For performance monitor |
protected static java.lang.String |
GET_GENERATOR_PROPERTY
For performance monitor |
protected static java.lang.String |
GET_META_PROPERTY
For performance monitor |
protected static java.lang.String |
GET_SUB_PROPERTY_VALUE
For performance monitor |
protected static java.lang.String |
GET_TEXT_PROPERTY
For performance monitor |
protected static java.lang.String |
MY_RESOURCE_NAME
Resource name used for messages. |
protected static java.lang.String |
PROCESS_PROPERTIES
For performance monitor |
protected static java.lang.String |
REGENERATE_FOR_CHANGE
For performance monitor |
protected static java.util.ResourceBundle |
sResourceBundle
|
protected static java.lang.String |
START_QUERY
For performance monitor |
Fields inherited from class atg.nucleus.GenericService |
---|
SERVICE_INFO_KEY |
Fields inherited from interface atg.nucleus.logging.TraceApplicationLogging |
---|
DEFAULT_LOG_TRACE_STATUS |
Fields inherited from interface atg.nucleus.logging.ApplicationLogging |
---|
DEFAULT_LOG_DEBUG_STATUS, DEFAULT_LOG_ERROR_STATUS, DEFAULT_LOG_INFO_STATUS, DEFAULT_LOG_WARNING_STATUS |
Constructor Summary | |
---|---|
LoaderImpl()
|
Method Summary | |
---|---|
void |
addActiveContext(Context pContext)
Add a Context to the list of active contexts. |
protected java.lang.String |
addUniqueParamsToURL(java.lang.String pBaseURL,
java.util.Map pMapUniqueParams)
Add any unique params to the URL. |
void |
adjustIncrementalQueue(IndexingOutputConfig pIndexingOutputConfig,
atg.search.index.IndexInfo pIndexInfo)
Pass-through method to adjust incremental queues. |
protected void |
afterSessionStart(Context pContext)
This method is invoked immediately after a DocumentSubmitter session is started. |
protected void |
beforeSessionEnd(Context pContext)
This method is invoked immediately before a DocumentSubmitter session is closed. |
boolean |
cancelActiveContext(java.lang.String pConfigurationPath)
Cancel the active context with the specified name. |
protected int |
claimGeneration(java.lang.String pConfigPath,
java.lang.String pContentId,
boolean pIncremental,
boolean pUseConfigurationClaims,
IndexingOutputConfig pIOC)
Claim a generation for updating. |
protected javax.servlet.Servlet |
createAdminServlet()
Create the admininistration servlet. |
protected Context |
createContext()
Create a new context. |
protected atg.repository.search.indexing.OutputDocumentContent |
createEmptyOutputDocumentContent()
Create a new, empty, OutputDocumentContent object. |
protected void |
deleteDocumentItem(RepositoryItem pItem,
Context pContext)
Remove the specified repository item pItem from the index. |
void |
doStopService()
When doStopService is called, exit out of our update activity thread, if any. |
atg.repository.search.indexing.ContextStatus[] |
getActiveContextStatuses()
Get an array of ContextStatuses... |
protected RepositoryItem[] |
getArrayFromMultiRepoItems(java.lang.Object pItems)
Return an array of RepositoryItems representing the repository items the come from the property pItem. |
ConfigStatePersister |
getConfigStatePersister()
Get the configuration state persister. |
int |
getDocumentsPerTransaction()
Get the number of documents per transaction. |
atg.repository.search.indexing.IncrementalItemQueue |
getIncrementalItemQueue()
Get the incremental item queue component, used for tracking changes to repository items for the registered configurations. |
int |
getLoggingInfoStatusCount()
Get how often to print out count/progress status information. |
javax.transaction.TransactionManager |
getTransactionManager()
Get the transaction manager from the configStatePersister. |
long |
getUpdateActivityTimeMillis()
Get the number of milliseconds to wait between updates to lastActivity time. |
boolean |
getUpdateLastActivityTimeInSeparateThread()
Whether to update last activity time in a separate thread. |
protected void |
initContext(Context pContext,
IndexingOutputConfig pConfig,
int pClaimedGeneration,
atg.search.index.IndexInfo pIndexInfo)
Initialize the context, using the specified parameter values. |
protected boolean |
isConnectionRelated(java.lang.Throwable pEx)
Return true if the exception is, or was caused by, a java.net.ConnectException. |
boolean |
isEchoDocumentsToStdout()
Whether to echo documents to stdout. |
boolean |
isPrettyPrint()
Get whether the XHTML should be pretty printed (that is, indented). |
boolean |
isPropertyLoggingDebug()
Whether to turn on logging debug for invidual property values. |
static boolean |
isSubTypeOfItemDesc(RepositoryItemDescriptor pItemDesc,
RepositoryItemDescriptor pSuper)
Returns whether pItemDesc is a subtype (or equal to) pSuper. |
boolean |
isTransactionPerDocument()
Whether to wrap all repository calls within a document in a transaction. |
atg.repository.search.indexing.LoaderImpl.UpdateActivityInfo |
nextQueuedUpdateActivity()
Return the next queued item for an update activity. |
protected void |
outputAndSubmitDocument(Context pContext)
Create the XHTML file for the Context's OutputDocumentContent and submit it using the DocumentSubmitterSession. |
void |
popMembershipContexts(atg.search.index.IndexInfo pIndexInfo)
If this is a multisite indexing operation, pop site membership context from into the MembershipContext. |
void |
processItem(Context pContext,
RepositoryItem pItem,
atg.repository.search.indexing.specifier.OutputItemSpecifier pSpecifier)
Process the specified item. |
protected void |
processMetaProperty(Context pContext,
RepositoryItem pItem,
atg.repository.search.indexing.specifier.OutputProperty pOutputProperty,
atg.repository.search.indexing.PropertyAccessor pPropAccessor,
java.lang.Object pPropertyValue)
Process a single meta property |
protected void |
processParentProperties(Context pContext)
Process any of the properties of containing items on the the context's stack. |
protected void |
processProperties(Context pContext,
RepositoryItem pItem,
atg.repository.search.indexing.specifier.OutputItemSpecifier pSpecifier,
boolean pIsMeta)
Process any properties: that is collect any property values. |
protected void |
processProperties(Context pContext,
RepositoryItem pItem,
atg.repository.search.indexing.specifier.OutputItemSpecifier pSpecifier,
atg.repository.search.indexing.specifier.OutputProperty[] pProperties,
boolean pIsMeta,
java.util.Map<atg.repository.search.indexing.specifier.OutputProperty,java.lang.Object> pGeneratorMap)
Process any properties: that is collect any property values. |
protected void |
processTextProperty(Context pContext,
RepositoryItem pItem,
atg.repository.search.indexing.specifier.OutputProperty pOutputProperty,
atg.repository.search.indexing.PropertyAccessor pPropAccessor,
java.lang.Object pPropertyValue)
Process a single text property |
boolean |
pushMembershipContexts(atg.search.index.IndexInfo pIndexInfo)
If this is a multisite indexing operation, push site membership context values into the MembershipContext. |
void |
queueUpdateActivity(atg.repository.search.indexing.LoaderImpl.UpdateActivityInfo pUpdateActivityInfo)
Queue an last activity time update change request. |
protected void |
releaseContext(Context pContext)
Release any resources claimed by the specified context. |
protected void |
releaseGeneration(java.lang.String pConfigPath,
java.lang.String pContentId,
int pGeneration,
boolean pCompletedSuccessfully,
boolean pIncremental,
boolean pUseConfigurationClaims)
Release the specified generation. |
void |
removeActiveContext(Context pContext)
Remove a Context from the list of active contexts. |
protected void |
resetContextAfterDocument(Context pContext)
Reset the Context after a generating a document. |
void |
setConfigStatePersister(ConfigStatePersister pConfigStatePersister)
Set the configuration state persister. |
void |
setDocumentsPerTransaction(int pDocumentsPerTransaction)
Set the number of documents per transaction. |
void |
setEchoDocumentsToStdout(boolean pEchoDocumentsToStdout)
Whether to echo documents to stdout. |
void |
setIncrementalItemQueue(atg.repository.search.indexing.IncrementalItemQueue pIncrementalItemQueue)
Set the incremental item queue component, used for tracking changes to repository items for the registered configurations. |
void |
setLoggingInfoStatusCount(int pLoggingInfoStatusCount)
Set how often to print out count/progress status information. |
protected void |
setNextIncrementalGeneration(java.lang.String pConfigPath,
java.lang.String pContentId,
int pGeneration)
Release the specified generation. |
void |
setPrettyPrint(boolean pPrettyPrint)
Set whether the XHTML should be pretty printed (that is, indented). |
void |
setPropertyLoggingDebug(boolean pPropertyLoggingDebug)
Whether to turn on logging debug for invidual property values. |
void |
setTransactionPerDocument(boolean pTransactionPerDocument)
Whether to wrap all repository calls within a document in a transaction. |
protected void |
setupContextForNewDocument(Context pContext,
RepositoryItem pItem,
java.util.Map pMapParams)
Set up our context for a new document representing pItem. |
void |
setUpdateActivityTimeMillis(long pUpdateActivityTimeMillis)
Set the number of milliseconds to wait between updates to lastActivity time. |
void |
setUpdateLastActivityTimeInSeparateThread(boolean pUpdateLastActivityTimeInSeparateThread)
Whether to update last activity time in a separate thread. |
protected java.lang.Object[] |
splitPropertyValue(atg.repository.search.indexing.specifier.OutputProperty pOutputProperty,
java.lang.Object pValue)
Split the property value into multiple values, if appropriate. |
void |
updateActivity(Context pContext)
Update our last activity time. |
Methods inherited from class atg.nucleus.logging.VariableArgumentApplicationLoggingImpl |
---|
vlogDebug, vlogDebug, vlogDebug, vlogDebug, vlogError, vlogError, vlogError, vlogError, vlogInfo, vlogInfo, vlogInfo, vlogInfo, vlogTrace, vlogTrace, vlogTrace, vlogTrace, vlogWarning, vlogWarning, vlogWarning, vlogWarning |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static java.lang.String CLASS_VERSION
protected static final java.lang.String MY_RESOURCE_NAME
protected static final java.lang.String START_QUERY
protected static final java.lang.String PROCESS_PROPERTIES
protected static final java.lang.String GET_SUB_PROPERTY_VALUE
protected static final java.lang.String GET_META_PROPERTY
protected static final java.lang.String GET_TEXT_PROPERTY
protected static final java.lang.String DELETE_DOCUMENT
protected static final java.lang.String GET_GENERATOR_PROPERTY
protected static final java.lang.String REGENERATE_FOR_CHANGE
protected static java.util.ResourceBundle sResourceBundle
Constructor Detail |
---|
public LoaderImpl()
Method Detail |
---|
public void setIncrementalItemQueue(atg.repository.search.indexing.IncrementalItemQueue pIncrementalItemQueue)
public atg.repository.search.indexing.IncrementalItemQueue getIncrementalItemQueue()
public void setConfigStatePersister(ConfigStatePersister pConfigStatePersister)
public ConfigStatePersister getConfigStatePersister()
public void setUpdateActivityTimeMillis(long pUpdateActivityTimeMillis)
public long getUpdateActivityTimeMillis()
public void setEchoDocumentsToStdout(boolean pEchoDocumentsToStdout)
public boolean isEchoDocumentsToStdout()
public void setPropertyLoggingDebug(boolean pPropertyLoggingDebug)
public boolean isPropertyLoggingDebug()
public void setTransactionPerDocument(boolean pTransactionPerDocument)
public boolean isTransactionPerDocument()
public void setLoggingInfoStatusCount(int pLoggingInfoStatusCount)
public int getLoggingInfoStatusCount()
public void setPrettyPrint(boolean pPrettyPrint)
public boolean isPrettyPrint()
public void setDocumentsPerTransaction(int pDocumentsPerTransaction)
public int getDocumentsPerTransaction()
public void setUpdateLastActivityTimeInSeparateThread(boolean pUpdateLastActivityTimeInSeparateThread)
public boolean getUpdateLastActivityTimeInSeparateThread()
public void processItem(Context pContext, RepositoryItem pItem, atg.repository.search.indexing.specifier.OutputItemSpecifier pSpecifier) throws IndexingException
pContext
- the indexing context object.pRepositoryItem
- the repository item that corresponds to this item.pSpecifier
- the specifier for this item.
IndexingException
public static boolean isSubTypeOfItemDesc(RepositoryItemDescriptor pItemDesc, RepositoryItemDescriptor pSuper)
pItemDesc
- The (presumably child) item descriporpSuper
- The item descriptor to test for parent-hood.
public void updateActivity(Context pContext) throws IndexingException
IndexingException
protected RepositoryItem[] getArrayFromMultiRepoItems(java.lang.Object pItems)
pItems
- the object from which to extract repository
items
protected boolean isConnectionRelated(java.lang.Throwable pEx)
protected void outputAndSubmitDocument(Context pContext) throws IndexingException
IndexingException
protected void setupContextForNewDocument(Context pContext, RepositoryItem pItem, java.util.Map pMapParams)
pContext
- the context to be updatedpItem
- the document-level item used to update pContextpMapParams
- the map of distinguising keys and values that
will be appened as query params to the document URL. Both
keys and values must be strings.protected java.lang.String addUniqueParamsToURL(java.lang.String pBaseURL, java.util.Map pMapUniqueParams)
pBaseURL
- the base URL... typically an atgrepo: URL.pMapUniqueParams
- the map of unique paramaters. This
should be an ordered map for consistency sake.protected void resetContextAfterDocument(Context pContext)
protected atg.repository.search.indexing.OutputDocumentContent createEmptyOutputDocumentContent()
protected void processParentProperties(Context pContext) throws IndexingException
pContext
- the context, currently set to the document
level.
IndexingException
protected void processProperties(Context pContext, RepositoryItem pItem, atg.repository.search.indexing.specifier.OutputItemSpecifier pSpecifier, boolean pIsMeta)
pItem
. Gets the static set of
properties of pSpecifier
, and possibly additional
property values from the PropertyAccessor, if instance of
AdditionalPropertiesPropertyAccessor.
pContext
- the current contextpItem
- the current item whose property values to collectpSpecifier
- the specifier corresponding to pItem
pIsMeta
- whether these properties are meta propertiesprotected void processMetaProperty(Context pContext, RepositoryItem pItem, atg.repository.search.indexing.specifier.OutputProperty pOutputProperty, atg.repository.search.indexing.PropertyAccessor pPropAccessor, java.lang.Object pPropertyValue)
protected void processTextProperty(Context pContext, RepositoryItem pItem, atg.repository.search.indexing.specifier.OutputProperty pOutputProperty, atg.repository.search.indexing.PropertyAccessor pPropAccessor, java.lang.Object pPropertyValue)
protected void processProperties(Context pContext, RepositoryItem pItem, atg.repository.search.indexing.specifier.OutputItemSpecifier pSpecifier, atg.repository.search.indexing.specifier.OutputProperty[] pProperties, boolean pIsMeta, java.util.Map<atg.repository.search.indexing.specifier.OutputProperty,java.lang.Object> pGeneratorMap)
pItem
.
pContext
- the current contextpItem
- the current item whose property values to collectpSpecifier
- the specifier corresponding to pItem
pProperties
- the array of output properties to collectpIsMeta
- whether these properties are meta propertiesprotected java.lang.Object[] splitPropertyValue(atg.repository.search.indexing.specifier.OutputProperty pOutputProperty, java.lang.Object pValue)
pOutputProperty
- the output property the corresponds
to this value.pValue
- the property value
protected Context createContext()
protected void initContext(Context pContext, IndexingOutputConfig pConfig, int pClaimedGeneration, atg.search.index.IndexInfo pIndexInfo)
pContext
- the context to initializepSession
- the document submitter sessionpConfig
- the indexing output configpClaimedGeneration
- the claimed generation for this
update.protected void releaseContext(Context pContext)
pContext
- The context to release.protected int claimGeneration(java.lang.String pConfigPath, java.lang.String pContentId, boolean pIncremental, boolean pUseConfigurationClaims, IndexingOutputConfig pIOC) throws IndexingException
ConfigStatePersister
to claim a given generation
for updating.
pConfigPath
- the configuration path for for
indexing output config whose index shoudl be udpated.pContentId
- The content ID that uniquely identifies the
configuration.pIncremental
- whether we are claiming this
generation for an incremental update.pUseConfigurationClaims
- If true, use configuration claims
and enforce exclusive access to a configuration + content ID by
throwing exceptions if calims cannot be obtaind.
IndexingException
protected void releaseGeneration(java.lang.String pConfigPath, java.lang.String pContentId, int pGeneration, boolean pCompletedSuccessfully, boolean pIncremental, boolean pUseConfigurationClaims) throws IndexingException
ConfigStatePersister
to release the specified generation.
pConfigPath
- The configuration path of the generationpGeneration
- the numeric id of the generation, previously
returned by claimedGeneration.pCompletedSuccessfully
- whether the generation
completed successfully.pIncremental
- whether the generation is an
incremental generation.pUseConfigurationClaims
- If true, use configuration claims
and enforce exclusive access to a configuration + content ID by
throwing exceptions if calims cannot be obtaind.
IndexingException
protected void setNextIncrementalGeneration(java.lang.String pConfigPath, java.lang.String pContentId, int pGeneration) throws IndexingException
ConfigStatePersister
to release the specified generation.
pConfigPath
- The configuration path of the generationpGeneration
- the numeric id of the generation, previously
returned by claimedGeneration.pCompletedSuccessfully
- whether the generation
completed successfully.pIncremental
- whether the generation is an
incremental generation.
IndexingException
public void addActiveContext(Context pContext)
pContext
- the context to add.public void removeActiveContext(Context pContext)
pContext
- the context to remove.public boolean cancelActiveContext(java.lang.String pConfigurationPath)
public atg.repository.search.indexing.ContextStatus[] getActiveContextStatuses()
protected javax.servlet.Servlet createAdminServlet()
createAdminServlet
in class GenericService
public javax.transaction.TransactionManager getTransactionManager()
protected void deleteDocumentItem(RepositoryItem pItem, Context pContext) throws IndexingException
IndexingException
protected void afterSessionStart(Context pContext) throws IndexingException
pContext
- the Context for the current indexing operation. The
DocumentSubmitterSession
is available as a property, as is
the IndexingOutputConfig
and other state information.
IndexingException
protected void beforeSessionEnd(Context pContext) throws IndexingException
pContext
- the Context for the current indexing operation. The
DocumentSubmitterSession
is available as a property, as is
the IndexingOutputConfig
and other state information.
IndexingException
public void queueUpdateActivity(atg.repository.search.indexing.LoaderImpl.UpdateActivityInfo pUpdateActivityInfo)
public atg.repository.search.indexing.LoaderImpl.UpdateActivityInfo nextQueuedUpdateActivity()
public void doStopService() throws ServiceException
doStopService
in class GenericService
ServiceException
- if the Service had a problem shutting downpublic void adjustIncrementalQueue(IndexingOutputConfig pIndexingOutputConfig, atg.search.index.IndexInfo pIndexInfo) throws IndexingException
IndexingException
public boolean pushMembershipContexts(atg.search.index.IndexInfo pIndexInfo)
pIndexInfo
- the object containing the context information
public void popMembershipContexts(atg.search.index.IndexInfo pIndexInfo)
pIndexInfo
- the object containing the context information
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |