When ENABLE is set to TRUE, language analysis (in addition to tokenization) is enabled; the analysis includes not only stemming, but any other functions that the analysis can perform, such as the use of a thesaurus, of stop words, and so on.

When ENABLE is set to FALSE, the language analysis performs only tokenization. A warning is displayed if ENABLE is set to FALSE and OLT language analysis is selected. The warning informs you that the setting of ENABLE in this case will be ignored, because OLT performs stemming unconditionally.

Note that the only stemming that Latin-1 analysis performs on English is to treat singular and plural wordforms as matches for each other; for example, to make "house" and "houses" match each other. This follows necessarily from the largely uninflected nature of English.

If USE_STATIC_WORDFORMS is set to TRUE, the stemming feature uses the static wordform dictionary files shipped with the MDEX Engine package. A static wordform dictionary file defines sets of inflected forms that are treated as matches for each other by the Guided Search feature; for example, the German word that means table, "Tisch", in its different grammatical cases, is specified as follows in the German wordforms file:

<WORD_FORMS>
  <WORD_FORM>tisch</WORD_FORM>
  <WORD_FORM>tisches</WORD_FORM>
  <WORD_FORM>tische</WORD_FORM>
  <WORD_FORM>tischen</WORD_FORM>
 </WORD_FORMS>

Default static wordform dictionary files are stored in Endeca\MDEX\version\conf\stemming (on Windows) and usr/local/endeca/MDEX/version/conf/stemming (on UNIX). You can update the default static wordform dictionary files, or create custom static wordform dictionary files to use in place of the default files. For information about how to do this, refer to the MDEX Developer's Guide.

If USE_STATIC_WORDFORMS is set to FALSE, dgidx generates the wordforms file from the source data dynamically.

Selecting a Language Analysis through USE_STATIC_WORDFORMS

Static wordform files are always used for stemming by Latin-1 analysis and are never used by OLT analysis. Thus, if USE_STATIC_WORDFORMS is set to TRUE, Latin-1 is selected; if set to FALSE, OLT is selected.

Setting USE_STATIC_WORDFORMS to TRUE forces Latin-1 analysis to be selected for a language even if Latin-1 is not the better analysis for that language. For example, if you set USE_STATIC_WORDFORMS to TRUE for traditional Chinese, Latin-1 is applied, with the result that the Chinese text is not properly tokenized.

USE_COMPOUND_MATCHING

If USE_COMPOUND_MATCHING is set to TRUE, the stemming feature matches compound words not only with themselves, but also with any of their elements taken individually.

Compound matching is supported only when OLT is used for language analysis. A word is recognized as a compound if it is known as such by OLT, or if you have added it to your auxiliary dictionary using the COMPOUND command. For information about the COMPOUND command, see Configuring decompounding in an auxiliary dictionary.

For example, when compound matching is enabled, the GERMAN word "Bananenstecker" (banana plug) can be matched either by "Banane" (banana) or by "Stecker" (plug). When compound matching is disabled, "Bananenstecker" is not matched by "Banane" or "Stecker", although it can be matched by inflected forms of "Bananenstecker" such as "Bananensteckers" (genitive singular).

Similarly, when compound matching is enabled for Dutch, then "tomaten" matches "tomaten", "tomaat", "tomatensaus", "tomatensauzen", "cherrytomaten", and "cherrytomaat". However, if compound matching is disabled, then "tomaten" (tomatoes) matches any of its inflected forms, such as "tomaten" and "tomaat", but does not match compounds containing "tomaat" or "tomaten", such as "tomatensaus", "tomatensauzen", "cherrytomaten", or "cherrytomaat".

To change default values in stemming.xml, you can edit it manually – that is, you can edit it directly using a text editor, rather than editing it through Developer Studio. Manual edits that you make to stemming.xml can be affected when you save your Developer Studio project and when you upgrade Developer Studio to the current version.