MULTI_LEXER

Requires a LANGUAGE column in the table that identifies the language for each document. Each language has an associated sub-lexer, defined by the user. This lexer has no attributes.

Use MULTI_LEXER to index text columns that contain documents of different languages. For example, use this lexer to index a text column that stores English, German, and Japanese documents.

You must have a LANGUAGE column in your base table. To index multi-language tables, specify the LANGUAGE column when you create the index. You must also specify the language at query time (through Session settings or a Language settings in a query template), and the queries only look for documents that are indexed using the current language.

Create a multi-lexer preference with CTX_DDL.CREATE_PREFERENCE. Add language-specific lexers to the multi-lexer preference with the CTX_DDL.ADD_SUB_LEXER procedure.

During indexing, the MULTI_LEXER examines each row’s language column value and switches in the language-specific lexer to process the document.

Note: If you drop the language column from a multi-lexer indexed table, you must also drop the index and rebuild it.

The WORLD_LEXER lexer also performs multi-language indexing, but without the need for separate LANGUAGE columns (that is, it has automatic language detection). For more on WORLD_LEXER, see “WORLD_LEXER”.

This section contains the following topics.

• MULTI_LEXER Restriction

• MULTI_LEXER Multi-language Stoplists

• MULTI_LEXER Example

• MULTI_LEXER and Querying Multi-Language Tables