AUTO_LEXER Language-Dependent Attributes

These are the language-dependent attributes available in the AUTO_LEXER. The <language> variable in the attribute name refers to any of the supported language names.

Note: Attribute names must not exceed 30 characters. Therefore, where the <language> variable is specified, the language name may need to be abbreviated in certain instances. For example, traditional_chinese should be abbreviated to trad_chinese and simplified_chinese should be abbreviated to simp_chinese.

Table 16 AUTO_LEXER Language-Dependent Attributes

Attribute	Attribute Value	Description
<language>_prefix_morphemes	characters (space-delimited string)	Specifies the list of inflectional prefixes that, when enclosed by parentheses, are kept together with the base word. For example, (re) analyze.
<language>_suffix_morphemes	characters (space-delimited string)	Specifies the list of inflectional suffixes that, when enclosed by parentheses are kept together with the base word. For example, file(s).
<language>_punctuations	characters (space-delimited string)	Specifies punctuation that breaks sentences.
<language>_non_sent_end_abbr	characters (space-delimited string)	Specifies abbreviations that do not end sentences.

Table 17 Default Values for AUTO_LEXER Language-Dependent Attributes

Attribute	Language	Default Value
<language>_prefix_morphemes	All languages	None
<language>_suffix_morphemes	English	s es er
<language>_suffix_morphemes	Spanish	ba n s es
<language>_suffix_morphemes	Portuguese	s es
<language>_suffix_morphemes	German	in innen
<language>_suffix_morphemes	French	ne e
<language>_suffix_morphemes	All other languages	None
<language>_punctuations	English	. ? !
<language>_punctuations	Catalan, Czech, Dutch, Greek, Hungarian, Polish, Romanian, Russian, Turkish	. ? ! - –
<language>_punctuations	French, German, Italian, Korean, Portuguese, Spanish, Swedish	, ? !
<language>_punctuations	Japanese
<language>_punctuations	Simplified Chinese Abbreviate to: simp_chinese
<language>_punctuations	Traditional Chinese Abbreviate to: trad_chinese
<language>_non_sent_end_abbr	Polish, Romanian, Russian, Turkish	e.g. i.e. viz. a.k.a.
<language>_non_sent_end_abbr	Catalan	R.D. pp.
<language>_non_sent_end_abbr	Czech, Greek, Hungarian	e.g. i.e. viz. a.k.a.
<language>_non_sent_end_abbr	Dutch	f.eks. f. eks. inkl. sr. skuesp. sekr. prof. mus. lrs. logr. kgl. insp. hr. hrs. gdr. frk. fr. forst. forf. fm. fmd. esq. d.æ d.æ. d.y. dr. dir. dept.chef civiling. bibl. ass. admn. adj. Skt. H.K.H.
<language>_non_sent_end_abbr	English, Japanese, Simplified Chinese (abbreviate to simp_chinese), Traditional Chinese (abbreviate to trad_chinese)	e.g. i.e. viz. a.k.a. Adm. Br. Capt. Cdr. Cmdr. Col. Comdr. Comdt. Dr. Drs. Fr. Gen. Gov. Hon. Ins. Lieut. Lt. Maj. Messrs. Mdm. Mlle. Mlles. Mme. Mmes. Mr. Mrs. Ms. Pres. Prof. Profs. Pvt. Rep. Rev. Revd. Secy. Sen. Sgt. Sra. Srta. St. Ste.
<language>_non_sent_end_abbr	French	c.-à-d. cf. e.g. ex. i.e. Pr. Prof. M. Mr. Mrs. Mme Mmes Mlle Mlles Mgr. MM. Lieut. Gén. Dr. Col.
<language>_non_sent_end_abbr	German	ca. bzw. e.g. i.e. inkl. Fr. Frl. Mme. Mile. Mag. Stud. Tel. Hr. Hrn. apl.Prof. Prof.
<language>_non_sent_end_abbr	Italian	e.g. i.e. pag. pagg. tel. T.V. N.H. N.D. comm. col. cav. cap. geom. gen. ing. jr. mr. mons. mar. magg. prof. prof.ssa prof.sse proff. pres. perito ind. p. p.i. sr. s.ten. sottoten. sig. serg. sen. segr. sac. ten. uff. vicepres. vesc. S.S. S.E. avv. app. amm. arch. on. dir. dott. dott.ssa dr. rag.
<language>_non_sent_end_abbr	Korean	e.g. i.e. a.k.a. Dr. Mr. Mrs. Ms. Prof.
<language>_non_sent_end_abbr	Portuguese	cf. Cf. e.g. E.g. i.é. I.é. p.ex. P.ex. pág. pag. Pág. Pag. tel. telef. Tel. Telef. sr. srs. sra. mr. eng. dr. dra. Dr. Dra. V.Ex. V.Exa. S. N. S. Mrs. Eng. Ex. Exa.
<language>_non_sent_end_abbr	Spanish	e.g. i.e. ej. p.ej. pág. págs. tel. tfno. Fr. Ldo. Lda. Lic. Pbro. D. Dña. Dr. Dres. Dra. Dras. Dn. Mons. Rvdo. Sto. Sta. Sr. Srs. Srta. Srtas. Sres. Sra. Sras. Excmo. Excma. Ilmo. Ilma. Sto. Sta.
<language>_non_sent_end_abbr	Swedish	inkl. prof. hrr. hr. Hrr. Hr. dr. Dr.

Examples for AUTO_LEXER Language-Dependent Attributes

Example 2-1 ctx_ddl.create_preference to associate a dictionary with an index

exec CTX_DDL.CREATE_PREFERENCE('A_LEX', 'AUTO_LEXER');
exec CTX_ANL. ADD_DICTIONARY('MY_ENGLISH', 'ENGLISH', lobloc);
select * from CTX_USR_ANL_DICTS;
exec CTX_DDL.SET_ATTRIBUTE('A_LEX', 'english_dictionary', 'MY_ENGLISH'
);

Example 2-2 <language>_prefix_morphemes

ctx_ddl.set_attribute(
      'a_lex', 'english_prefix_morphemes', 're'
);

Example 2-3 <language>_suffix_morphemes

ctx_ddl.set_attribute(
      'a_lex', 'english_suffix_morphemes', 's es'
);

Example 2-4 <language>_punctuations

ctx_ddl.set_attribute(
      'a_lex', 'english_punctuations', '. ? !'
);

Example 2-5 <language>_non_sentence_ending_abbrev

ctx_ddl.set_attribute(
      'a_lex', 'english_non_sentence_ending_abbrev', 'e.g. a.k.a. Dr.'
);