atg.endeca.index.schema

Class SchemaExporter

java.lang.Object

atg.nucleus.logging.VariableArgumentApplicationLoggingImpl

atg.nucleus.GenericService

atg.endeca.index.PerApplicationExporter

atg.endeca.index.schema.SchemaExporter

All Implemented Interfaces:

atg.core.logging.Logging, atg.core.logging.TraceLogging, atg.naming.AbsoluteNameable, NameContextBindingListener, NameContextElement, atg.naming.NameContextParentable, NameResolver, AdminableService, atg.nucleus.Configured, ApplicationLogging, atg.nucleus.logging.ApplicationLoggingSender, atg.nucleus.logging.TraceApplicationLogging, VariableArgumentApplicationLogging, ComponentNameResolver, Service, ServiceListener, java.util.EventListener

public class SchemaExporter
extends PerApplicationExporter

This class uploads an Endeca schema data into a RecordStore using a RecordStoreDocumentSubmitter. It introspects the specified IndexingOutputConfig to generate its base schema, and then layers on any additional changes form the XML file specfied by the supplementalSchemaRecords property.

The schema is represented by a Records object that holds a Record-based representation of the schema, which is then uploaded via a RecordStoreDocumentSubmitter.

This class supports merging in supplemental/override information via the supplementalSchemaRecords property.

Created: November 16 2011

Nested Class Summary

Nested classes/interfaces inherited from class atg.endeca.index.PerApplicationExporter

PerApplicationExporter.MinimalBulkLoaderResults, PerApplicationExporter.MinimalIncrementalLoaderResults

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	ATTRIBUTE_DIMENSION_AUTOGEN CAS Record property name for dimension autogen.
static java.lang.String	ATTRIBUTE_DIMENSION_MULTISELECT_TYPE CAS Record property name to set for multiselect type.
static java.lang.String	ATTRIBUTE_DIMENSION_RANGE_COMPARISON_TYPE CAS Record property name to set for multiselect type.
static java.lang.String	ATTRIBUTE_DISPLAY_NAME CAS Record property name for attribute diplay name.
static java.lang.String	ATTRIBUTE_NAME CAS Record property name that names the Records we use to represent the schema.
static java.lang.String	ATTRIBUTE_PROPERTY_DATA_TYPE CAS Record property name for property data type.
static java.lang.String	ATTRIBUTE_ROLLUP_KEY CAS Record property name property name to set for a text searchable attribute.
static java.lang.String	ATTRIBUTE_SEARCH_SEARCHABLE CAS Record property name property name to set for a text searchable attribute.
static java.lang.String	ATTRIBUTE_SEARCH_WILDCARD CAS Record property name property name to set for a wildcard searchable attribute.
static java.lang.String	ATTRIBUTE_SOURCE_NAME CAS Record property name for attribute source name.
static java.lang.String	ATTRIBUTE_TYPE CAS Record property name for attribute type (that is, dimension or property).
static java.lang.String	CLASS_VERSION Class version string

Fields inherited from class atg.endeca.index.PerApplicationExporter

TEST_GENERATION_CONTEXT_ATTRIBUTE

Fields inherited from class atg.nucleus.GenericService

SERVICE_INFO_KEY

Fields inherited from interface atg.nucleus.logging.TraceApplicationLogging

DEFAULT_LOG_TRACE_STATUS

Fields inherited from interface atg.nucleus.logging.ApplicationLogging

DEFAULT_LOG_DEBUG_STATUS, DEFAULT_LOG_ERROR_STATUS, DEFAULT_LOG_INFO_STATUS, DEFAULT_LOG_WARNING_STATUS

Constructor Summary

Constructors

Constructor and Description
SchemaExporter()

Method Summary

Methods

Modifier and Type	Method and Description
protected void	addBaselineSchemaRecords(Records pRecords, java.util.Locale pLocale, boolean pAddMeta) Add the baseline schema records to the specified records object.
void	addSchemaRecordsForProperties(java.util.List<atg.repository.search.indexing.specifier.OutputProperty> pOutputProperties, Records pRecords, java.util.Locale pLocale, boolean pAddMeta) Add the records that represent the passed in output properties the specified Records object.
void	addSchemaRecordsForProperties(java.util.List<atg.repository.search.indexing.specifier.OutputProperty> pOutputProperties, Records pRecords, java.util.Locale pLocale, boolean pAddMeta, boolean pAddGenerative) Add the records that represent the passed in output properties the specified Records object.
Records	createBaselineSchemaRecords(java.util.Locale pLocale) Create a baseline set of schema records for the specified locale.
Records	createEmptyRecords() Create an empty Records object with ATTRIBUTE_NAME already set as the name property.
protected Records	createExportRecords(EndecaContext pContext, java.util.Locale pLocale) Create the export records for the specified locale.
Records	createMergedSchemaRecords(java.util.Locale pLocale) Return the complete, merged schema records.
java.lang.String	generateDebugBaselineSchemaAsXML() Return the baseline (pre-supplemental merge) schema generated from the IndexingOutputConfig as XML.
java.lang.String	generateDebugMergedSchemaAsXML() Return an XML represetation of the merged schema XML.
java.lang.String	generateDebugSupplementalSchemaAsXML() Return an XML represetation of the supplemental schema XML.
protected java.lang.String	getDimensionNameForProperty(atg.repository.search.indexing.specifier.OutputProperty pProperty) Return the dimension name for the specified property.
DimensionNameProvider[]	getDimensionNameProviders() Returns property dimensionNameProviders.
protected java.lang.String	getEndecaDataTypeForProperty(atg.repository.search.indexing.specifier.OutputProperty pProperty) Return the endeca data type for the specified property.
MetaPropertiesHelper	getMetaPropertiesHelper() Return MetaPropertisHelper
java.lang.String[]	getNonAutogenDimensionPropertyNames() Returns property nonAutogenDimensionPropertyNames.
java.lang.String[]	getPropertiesNamesToIgnore() Returns property propertiesNamesToIgnore.
java.util.Map<java.lang.String,java.lang.String>	getPropertyNameToDimensionName() Returns property propertyNameToDimensionName.
java.lang.String	getRecordIdName()
java.util.List<SchemaExporter>	getSchemaExporters()
atg.xml.XMLFile	getSupplementalSchemaRecords() Returns property supplementalSchemaRecords.
protected java.lang.String	getXmlOutputTitle() Return the text string used as the title for the XML output in the HTML admin.
boolean	isAddNewSupplementalRecords() Returns property addNewSupplementalRecords.
protected boolean	isEndecaDimensionProperty(atg.repository.search.indexing.specifier.OutputProperty pProperty) Return whether the specified property is an Endeca dimension property.
protected boolean	isEndecaRollUpProperty(atg.repository.search.indexing.specifier.OutputProperty pProperty) Return whether the specified property is an Endeca roll-up poperty.
boolean	isOutputPropertyCollisionCheckEnabled()
boolean	isSkipPropertiesWithGenerativePropertyAccessors() Returns property skipPropertiesWithGenerativePropertyAccessors.
protected boolean	isTextSearchable(atg.repository.search.indexing.specifier.OutputProperty pProperty) Return whether the specified property should be text searchable (full text indexed).
void	mergeSupplementalIntoSchemaRecords(Records pRecords, boolean pAddNewRecords) Merge any supplement schema information from the supplementalSchemaRecords property into the specified records.
Records	parseSupplementalRecords() Parse supplemental records as specified by the supplementalSchemaRecords property.
void	setAddNewSupplementalRecords(boolean pAddNewSupplementalRecords) Sets property addNewSupplementalRecords.
void	setDimensionNameProviders(DimensionNameProvider[] pDimensionNameProviders) Sets property dimensionNameProviders.
void	setIndexables(Indexable[] pIndexables) This method takes an array of Indexable and allows this Indexable to have reference to other Indexable's
void	setMetaPropertiesHelper(MetaPropertiesHelper pMetaPropertiesHelper) Sets MetaPropertiesHelper
void	setNonAutogenDimensionPropertyNames(java.lang.String[] pNonAutogenDimensionPropertyNames) Sets property nonAutogenDimensionPropertyNames.
void	setOutputPropertyCollisionCheckEnabled(boolean pOutputPropertyCollisionCheckEnabled)
void	setPropertiesNamesToIgnore(java.lang.String[] pPropertiesNamesToIgnore) Sets property propertiesNamesToIgnore.
void	setPropertyNameToDimensionName(java.util.Map<java.lang.String,java.lang.String> pPropertyNameToDimensionName) Sets property propertyNameToDimensionName.
void	setRecordIdName(java.lang.String pRecordIdName)
void	setSchemaExporters(java.util.List<SchemaExporter> pSchemaExporters)
void	setSkipPropertiesWithGenerativePropertyAccessors(boolean pSkipPropertiesWithGenerativePropertyAccessors) Sets property skipPropertiesWithGenerativePropertyAccessors.
void	setSupplementalSchemaRecords(atg.xml.XMLFile pSupplementalSchemaRecords) Sets property supplementalSchemaRecords.
boolean	shouldIgnoreProperty(java.lang.String pPropertyName) Whether we should ignore the specified property name when generating records.

Methods inherited from class atg.endeca.index.PerApplicationExporter

cancel, createAdminServlet, createContext, createExportRecordsAsXML, doStartService, export, exportRecords, getDefaultLanguageForRecordStores, getDocumentSubmitter, getIndexingApplicationConfiguration, getIndexingOutputConfig, getIndexingOutputConfigPaths, getLocales, initContext, isEchoRecordsToStdout, isForceToBaseline, isNeededForIncremental, isSingleLanguageOverride, isSupportsStatusCounts, mayNeedCleanup, performBaselineUpdate, performPartialUpdate, postIndexingCleanup, restoreDynamoRequestLocale, setDefaultLanguageForRecordStores, setDocumentSubmitter, setDynamoRequestLocale, setEchoRecordsToStdout, setIndexingApplicationConfiguration, setIndexingOutputConfig

Methods inherited from class atg.nucleus.GenericService

addLogListener, doStopService, getAbsoluteName, getAdminServlet, getAdminServletOutputStreamEncoding, getLoggingForVlogging, getLogListenerCount, getLogListeners, getName, getNameContext, getNucleus, getRoot, getServiceConfiguration, getServiceInfo, isAdminServletUseServletOutputStream, isLoggingDebug, isLoggingError, isLoggingInfo, isLoggingTrace, isLoggingWarning, isRunning, logDebug, logDebug, logDebug, logError, logError, logError, logInfo, logInfo, logInfo, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, nameContextElementBound, nameContextElementUnbound, removeLogListener, reResolveThis, resolveName, resolveName, resolveName, resolveName, sendLogEvent, setAdminServletOutputStreamEncoding, setAdminServletUseServletOutputStream, setLoggingDebug, setLoggingError, setLoggingInfo, setLoggingTrace, setLoggingWarning, setNucleus, setServiceInfo, startService, stopService

Methods inherited from class atg.nucleus.logging.VariableArgumentApplicationLoggingImpl

vlogDebug, vlogDebug, vlogDebug, vlogDebug, vlogDebugTrace, vlogError, vlogError, vlogError, vlogError, vlogInfo, vlogInfo, vlogInfo, vlogInfo, vlogTrace, vlogTrace, vlogTrace, vlogTrace, vlogWarning, vlogWarning, vlogWarning, vlogWarning

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CLASS_VERSION

public static java.lang.String CLASS_VERSION

Class version string

ATTRIBUTE_NAME

public static final java.lang.String ATTRIBUTE_NAME

CAS Record property name that names the Records we use to represent the schema.

See Also:

Constant Field Values

ATTRIBUTE_SOURCE_NAME

public static final java.lang.String ATTRIBUTE_SOURCE_NAME

CAS Record property name for attribute source name.

See Also:

Constant Field Values

ATTRIBUTE_DISPLAY_NAME

public static final java.lang.String ATTRIBUTE_DISPLAY_NAME

CAS Record property name for attribute diplay name.

See Also:

Constant Field Values

ATTRIBUTE_TYPE

public static final java.lang.String ATTRIBUTE_TYPE

CAS Record property name for attribute type (that is, dimension or property).

See Also:

Constant Field Values

ATTRIBUTE_PROPERTY_DATA_TYPE

public static final java.lang.String ATTRIBUTE_PROPERTY_DATA_TYPE

CAS Record property name for property data type.

See Also:

Constant Field Values

ATTRIBUTE_DIMENSION_AUTOGEN

public static final java.lang.String ATTRIBUTE_DIMENSION_AUTOGEN

CAS Record property name for dimension autogen.

See Also:

Constant Field Values

ATTRIBUTE_DIMENSION_MULTISELECT_TYPE

public static final java.lang.String ATTRIBUTE_DIMENSION_MULTISELECT_TYPE

CAS Record property name to set for multiselect type.

See Also:

Constant Field Values

ATTRIBUTE_DIMENSION_RANGE_COMPARISON_TYPE

public static final java.lang.String ATTRIBUTE_DIMENSION_RANGE_COMPARISON_TYPE

CAS Record property name to set for multiselect type. This is not actually supported by CAS, but is is used when using the ConfigImportDocumentSubmitter to set rangeComparisonType on a Dimension objet.

See Also:

Constant Field Values

ATTRIBUTE_SEARCH_SEARCHABLE

public static final java.lang.String ATTRIBUTE_SEARCH_SEARCHABLE

CAS Record property name property name to set for a text searchable attribute.

See Also:

Constant Field Values

ATTRIBUTE_ROLLUP_KEY

public static final java.lang.String ATTRIBUTE_ROLLUP_KEY

CAS Record property name property name to set for a text searchable attribute.

See Also:

Constant Field Values

ATTRIBUTE_SEARCH_WILDCARD

public static final java.lang.String ATTRIBUTE_SEARCH_WILDCARD

CAS Record property name property name to set for a wildcard searchable attribute.

See Also:

Constant Field Values

Constructor Detail

SchemaExporter

public SchemaExporter()

Method Detail

setSupplementalSchemaRecords

public void setSupplementalSchemaRecords(atg.xml.XMLFile pSupplementalSchemaRecords)

Sets property supplementalSchemaRecords. An XMLFile that allows one to supplement the auto-generated information.

getSupplementalSchemaRecords

public atg.xml.XMLFile getSupplementalSchemaRecords()

Returns property supplementalSchemaRecords. An XMLFile that allows one to supplement the auto-generated information.

setDimensionNameProviders

public void setDimensionNameProviders(DimensionNameProvider[] pDimensionNameProviders)

Sets property dimensionNameProviders. A set of DimensionNameProviders who can override the default dimension name for a dimension property.

getDimensionNameProviders

public DimensionNameProvider[] getDimensionNameProviders()

Returns property dimensionNameProviders. A set of DimensionNameProviders who can override the default dimension name for a dimension property.

getMetaPropertiesHelper

public MetaPropertiesHelper getMetaPropertiesHelper()

Return MetaPropertisHelper

Returns:

MetaPropertiesHelper

setMetaPropertiesHelper

public void setMetaPropertiesHelper(MetaPropertiesHelper pMetaPropertiesHelper)

Sets MetaPropertiesHelper

Parameters:

pMetaPropertiesHelper - MetaPropertiesHelper

setPropertyNameToDimensionName

public void setPropertyNameToDimensionName(java.util.Map<java.lang.String,java.lang.String> pPropertyNameToDimensionName)

Sets property propertyNameToDimensionName. Map from an output property name to a dimension name. Consulted if none of the DimensionNameProviders provide a mapping.

getPropertyNameToDimensionName

public java.util.Map<java.lang.String,java.lang.String> getPropertyNameToDimensionName()

Returns property propertyNameToDimensionName. Map from an output property name to a source dimension name. Consulted if none of the DimensionNameProviders provide a mapping.

setAddNewSupplementalRecords

public void setAddNewSupplementalRecords(boolean pAddNewSupplementalRecords)

Sets property addNewSupplementalRecords. Whether to add records that only exist in the supplemental data.

isAddNewSupplementalRecords

public boolean isAddNewSupplementalRecords()

Returns property addNewSupplementalRecords. Whether to add records that only exist in the supplemental data.

setSkipPropertiesWithGenerativePropertyAccessors

public void setSkipPropertiesWithGenerativePropertyAccessors(boolean pSkipPropertiesWithGenerativePropertyAccessors)

Sets property skipPropertiesWithGenerativePropertyAccessors. Whether to ignore properties that specify generative property accessor, since those typically generate properties with dynamic names.

isSkipPropertiesWithGenerativePropertyAccessors

public boolean isSkipPropertiesWithGenerativePropertyAccessors()

Returns property skipPropertiesWithGenerativePropertyAccessors. Whether to ignore properties that specify generative property accessor, since those typically generate properties with dynamic names.

setPropertiesNamesToIgnore

public void setPropertiesNamesToIgnore(java.lang.String[] pPropertiesNamesToIgnore)

Sets property propertiesNamesToIgnore. Property names that we should ignore when generating.

getPropertiesNamesToIgnore

public java.lang.String[] getPropertiesNamesToIgnore()

Returns property propertiesNamesToIgnore. Property names that we should ignore.

setNonAutogenDimensionPropertyNames

public void setNonAutogenDimensionPropertyNames(java.lang.String[] pNonAutogenDimensionPropertyNames)

Sets property nonAutogenDimensionPropertyNames. These are dimension output property that should not be autogenerated.

getNonAutogenDimensionPropertyNames

public java.lang.String[] getNonAutogenDimensionPropertyNames()

Returns property nonAutogenDimensionPropertyNames. Property names that we should ignore.

getRecordIdName

public java.lang.String getRecordIdName()

Returns:

the mIndexingOutputConfigs

setRecordIdName

public void setRecordIdName(java.lang.String pRecordIdName)

Parameters:

mIndexingOutputConfigs - the mIndexingOutputConfigs to set

getSchemaExporters

public java.util.List<SchemaExporter> getSchemaExporters()

Returns:

the mIndexingOutputConfigs

setSchemaExporters

public void setSchemaExporters(java.util.List<SchemaExporter> pSchemaExporters)

Parameters:

mIndexingOutputConfigs - the mIndexingOutputConfigs to set

isOutputPropertyCollisionCheckEnabled

public boolean isOutputPropertyCollisionCheckEnabled()

Returns:

the outputPropertyCollisionCheckEnabled

setOutputPropertyCollisionCheckEnabled

public void setOutputPropertyCollisionCheckEnabled(boolean pOutputPropertyCollisionCheckEnabled)

Parameters:

pOutputPropertyCollisionCheckEnabled - the outputPropertyCollisionCheckEnabled to set

createExportRecords

protected Records createExportRecords(EndecaContext pContext,
                          java.util.Locale pLocale)

Create the export records for the specified locale.

Specified by:

createExportRecords in class PerApplicationExporter

Parameters:

pContext - the current indexing context.

pLocale - to locale for which to generate records.

Returns:

the generated set of records.

generateDebugBaselineSchemaAsXML

public java.lang.String generateDebugBaselineSchemaAsXML()
                                                  throws java.io.IOException,
                                                         atg.repository.search.indexing.XMLWriterException

Return the baseline (pre-supplemental merge) schema generated from the IndexingOutputConfig as XML.

Returns:

the XML representation.

Throws:

java.io.IOException

atg.repository.search.indexing.XMLWriterException

createEmptyRecords

public Records createEmptyRecords()

Create an empty Records object with ATTRIBUTE_NAME already set as the name property.

createMergedSchemaRecords

public Records createMergedSchemaRecords(java.util.Locale pLocale)

Return the complete, merged schema records.

createBaselineSchemaRecords

public Records createBaselineSchemaRecords(java.util.Locale pLocale)

Create a baseline set of schema records for the specified locale.

Parameters:

pLocale - the locale in which to export the schema records.

mergeSupplementalIntoSchemaRecords

public void mergeSupplementalIntoSchemaRecords(Records pRecords,
                                      boolean pAddNewRecords)

Merge any supplement schema information from the supplementalSchemaRecords property into the specified records.

Parameters:

pRecords - the records to merge the supplemental records into.

pAddNewRecords - whether to add new records if no corresponding supplemental record exists.

addBaselineSchemaRecords

protected void addBaselineSchemaRecords(Records pRecords,
                            java.util.Locale pLocale,
                            boolean pAddMeta)

Add the baseline schema records to the specified records object.

Parameters:

pRecords - the Records object to add records to.

pLocale - the current locale.

pAddMeta - true specifies that records for meta properties should be added, and false specifies that records for text properties should be added.

addSchemaRecordsForProperties

public void addSchemaRecordsForProperties(java.util.List<atg.repository.search.indexing.specifier.OutputProperty> pOutputProperties,
                                 Records pRecords,
                                 java.util.Locale pLocale,
                                 boolean pAddMeta)

Add the records that represent the passed in output properties the specified Records object.

Parameters:

pOutputProperties - the output properties to add records for.

pRecords - the Records object to add records to.

pLocale - the current Locale

pAddMeta - true specifies that records for meta properties should be added, and false specifies that records for text properties should be added.

addSchemaRecordsForProperties

public void addSchemaRecordsForProperties(java.util.List<atg.repository.search.indexing.specifier.OutputProperty> pOutputProperties,
                                 Records pRecords,
                                 java.util.Locale pLocale,
                                 boolean pAddMeta,
                                 boolean pAddGenerative)

Add the records that represent the passed in output properties the specified Records object.

Parameters:

pOutputProperties - the output properties to add records for.

pRecords - the Records object to add records to.

pLocale - the current Locale

pAddMeta - true specifies that records for meta properties should be added, and false specifies that records for text properties should be added.

pAddGenerative - whether to force adding of generative properties. if false, than isSkipPropertiesWithGenerativePropertyAccessors() will be honored.

getDimensionNameForProperty

protected java.lang.String getDimensionNameForProperty(atg.repository.search.indexing.specifier.OutputProperty pProperty)

Return the dimension name for the specified property. Consults dimensionNameProviders and the propertyNameToDimensionName map. Defaults to output name of the property if not overridden.

Parameters:

pProperty - the property whose dimension name to return.

isEndecaDimensionProperty

protected boolean isEndecaDimensionProperty(atg.repository.search.indexing.specifier.OutputProperty pProperty)

Return whether the specified property is an Endeca dimension property. If no explicit 'store-as-dimension' attribute is defined, return true if the property is "storeAsDocset", "storeAsCollection", or "storeAsMetaIndex."

Returns:

true if the property is an Endeca dimension property.

isEndecaRollUpProperty

protected boolean isEndecaRollUpProperty(atg.repository.search.indexing.specifier.OutputProperty pProperty)

Return whether the specified property is an Endeca roll-up poperty. Depends on the "is-rollup-key in the output config XML.

Returns:

true if the property is an Endeca dimension property.

isTextSearchable

protected boolean isTextSearchable(atg.repository.search.indexing.specifier.OutputProperty pProperty)

Return whether the specified property should be text searchable (full text indexed). Returns true if the property is explicitly marked as text searchable, the property is a text property, or has had a text property merged into it. If a property is explicitly set as not text searchable, returns false despite it is a text property or has a text property merged into it.

Parameters:

pProperty - the property to example to determine whether it should be text searchable.

Returns:

true if pProperty should be text searchable, false otherwise.

shouldIgnoreProperty

public boolean shouldIgnoreProperty(java.lang.String pPropertyName)

Whether we should ignore the specified property name when generating records.

getEndecaDataTypeForProperty

protected java.lang.String getEndecaDataTypeForProperty(atg.repository.search.indexing.specifier.OutputProperty pProperty)

Return the endeca data type for the specified property.

Parameters:

pProperty - the output property whose endeca data type should be returned.

generateDebugSupplementalSchemaAsXML

public java.lang.String generateDebugSupplementalSchemaAsXML()

Return an XML represetation of the supplemental schema XML. This does not include any baseline data from the IndexingOutputConfig.

generateDebugMergedSchemaAsXML

public java.lang.String generateDebugMergedSchemaAsXML()

Return an XML represetation of the merged schema XML.

parseSupplementalRecords

public Records parseSupplementalRecords()

Parse supplemental records as specified by the supplementalSchemaRecords property.

Returns:

a Records object containing supplemental records, null if supplementalSchemaRecords is null.

setIndexables

public void setIndexables(Indexable[] pIndexables)

This method takes an array of Indexable and allows this Indexable to have reference to other Indexable's

Overrides:

setIndexables in class PerApplicationExporter

Parameters:

pIndexables - - array of Indexable

getXmlOutputTitle

protected java.lang.String getXmlOutputTitle()

Description copied from class: PerApplicationExporter

Return the text string used as the title for the XML output in the HTML admin.

Overrides:

getXmlOutputTitle in class PerApplicationExporter

Returns:

the XML output title.