JAPANESE_LEXER
Identifies tokens in Japanese for creating Oracle Text indexes. Offers advantages over JAPANESE_VGRAM_LEXER, such as generates a smaller index, has a better query response time, and generates real word tokens resulting in better query precision.
The JAPANESE_LEXER type supports the stem ($) operator. Because the JAPANESE_LEXER uses a new algorithm to generate tokens, indexing time is longer than with JAPANESE_VGRAM_LEXER.
You can modify the existing lexicon (dictionary) used by the Japanese lexer, or create your own Japanese lexicon, with the ctxlc command.
See Also: “Lexical Compiler (ctxlc)” in Oracle Text Utilities
This lexer has the following attributes:
Table 25 JAPANESE_LEXER Attributes
| Attribute | Attribute Value |
|---|---|
delimiter |
Specify NONE or ALL to ignore certain Japanese blank characters, such as a full-width forward slash or a full-width middle dot. Default is NONE. |
mixed_case_ASCII7 |
Enable mixed-case (upper- and lower-case) searches of ASCII7 text (for example, cat and Cat). Allowable values are YES and NO (default). |
The JAPANESE_LEXER supports the following character sets:
When you specify JAPANESE_LEXER for creating text index, the JAPANESE_LEXER resolves a sentence into words.
For example, the following compound word (natural language institute)
Description of the illustration nihongo1.gif
is indexed as three tokens:
Description of the illustration nihongo2.gif
To resolve a sentence into words, the internal dictionary is referenced. When a word cannot be found in the internal dictionary, Oracle Text uses the JAPANESE_VGRAM_LEXER to resolve it.