JAPANESE_VGRAM_LEXER

Identifies tokens in Japanese for creating Oracle Text indexes. This lexer supports the stem ($) operator.

Table 24 JAPANESE_VGRAM_LEXER Attributes

Attribute	Attribute Value
delimiter	Specify whether to consider certain Japanese blank characters, such as a full-width forward slash or a full-width middle dot, as part of the indexed token. ALL considers these characters as part of the token while NONE ignores them. The default is NONE.
mixed_case_ASCII7	Enable mixed-case (upper- and lower-case) searches of ASCII7 text (for example, cat and Cat). Allowable values are YES and NO (default).
bigram	Specify TRUE to enable the bigram mode for the Japanese VGRAM lexer. In the bigram mode, the Japanese queries run faster because only 2-gram tokens are generated, thus avoiding the internal wildcard search. But, in the bigram mode, the index size needs to be increased to accommodate the large number of tokens. Enable the bigram mode, if the performance of queries is of higher importance to you than the disk space. Default is FALSE.
printjoins	Specify the non alphanumeric characters that, when they appear anywhere in a word (beginning, middle, or end), are processed as alphanumeric and included with the token in the Text index. This includes printjoins that occur consecutively. See Basic Lexer ““printjoins””.
skipjoins	Specify the non-alphanumeric characters that, when they appear within a word, identify the word as a single token; however, the characters are not stored with the token in the Text index. See Basic Lexer ““skipjoins””.

You can use this lexer if your database uses one of the following character sets:

• JA16SJIS

• JA16EUC

• UTF8

• AL32UTF8

• JA16EUCTILDE

• JA16EUCYEN

• JA16SJISTILDE

• JA16SJISYEN

Rules for PRINTJOIN and SKIPJOIN Characters

• Only non-alphanumeric ASCII characters that**do not include any Chinese, Japanese, or Korean characters orany full-width non-alphanumeric characters are accepted.

• You can specify a single non-alphanumeric character or multiple non-alphanumeric characters at a time.

• The printjoin/skipjoin will be ignored if you enter any characters that are not allowed. This includes alphanumeric characters, CJK – Chinese, Japanese, Korean – characters or full-width non-alphanumeric characters.

• In case of duplicate non-alphanumeric characters, duplicate entries will be ignored.

Examples

Example 2-6 Using Printjoins with JAPANESE_VGRAM_LEXER

This example defines the hyphen and underscore characters as printjoins thereby indicating that these characters must be included with the token in the Text index. Therefore, words such as web-site or web_site as indexed as web-site and web_site. Queries that search for website will not return documents containing web-site or web_site.

ctx_ddl.create_preference('mylex', 'JAPANESE_VGRAM_LEXER');
ctx_ddl.set_attribute('mylex', 'printjoins', '_-');

Example 2-7 Using Skipjoins with JAPANESE_VGRAM_LEXER

This example defines the hyphen and underscore characters as skipjoins thereby indicating that these characters must not be included with the token in the Text index. Therefore, words such as web-site or web_site as indexed as website. Queries that search for website will return documents containing web-site or web_site.

ctx_ddl.create_preference('mylex', 'JAPANESE_VGRAM_LEXER');
ctx_ddl.set_attribute('mylex', 'skipjoins', '_-');