SHOW_TOKENS

The CTX_REPORT.SHOW_TOKENS function creates a report showing all the tokens (that is, words) in the main index ($I and $G index tables).

You can call this operation as a function that returns the report as a CLOB. This API displays all the tokens in the index, ordered by token and token type. By default, all the tokens are sorted alphabetically, without any header and with each token appearing on a new line.

For example:

CAT
DOG
FROG
HORSE

Syntax

function CTX_REPORT.SHOW_TOKENS (
  index_name     IN VARCHAR2,
  part_name      IN VARCHAR2 DEFAULT NULL,
  report_format  IN VARCHAR2 DEFAULT FMT_TEXT,
  types          IN BOOLEAN DEFAULT FALSE,
  docid_counts   IN BOOLEAN DEFAULT FALSE,
  docid_ranges   IN BOOLEAN DEFAULT FALSE,
  frag_counts    IN BOOLEAN DEFAULT FALSE)
return clob;

index_name

Specify the name of the index.

part_name

Specify the name of the index partition.

If the index is a local partitioned index, then part_name must be provided. SHOW_TOKENS will apply to that index partition.

report_format

Specify whether the report should be generated in a plain text or JSON format:

types

Specify TRUE to print the token_type number and the corresponding type_name of section:

For example:

CAT   0  TEXT

DOG 101  SORTABLE SDATA

FROG 101 SORTABLE SDATA

HORSE 400 MDATA

JSON Format:
[
   {
      "token":"CAT",
      "token_type":0,
      "type_name":"TEXT"
   },
   {
      "token":"DOG",
      "token_type":101,
      "type_name":"TEXT"
   },
   {
      "token":"FROG",
      "token_type":101,
      "type_name":"SORTABLE_SDATA"
   },
   {
      "token":"HORSE",
      "token_type":400,
      "type_name":"MDATA"
   }
]

docid_counts

Specify TRUE to include the docid (document ID) count, that is, the total number of documents associated with the token.

For example:

JSON Format:
[
   {
      "token":"CAT",
      "docid_count":2
   },
   {
      "token":"DOG",
      "docid_count":3
   },
   {
      "token":"FROG",
      "docid_count":10
   },
   {
      "token":"HORSE",
      "docid_count":4
   }
]

docid_ranges

Specify TRUE to include the docid_first and docid_last details, that is, the number of tokens in the first and last document (within a range of documents associated with the token).

For example:

JSON Format:
[
   {
      "token":"CAT",
      "docid_first":2,
      "docid_last":3
   },
   {
      "token":"DOG",
      "docid_first":4,
      "docid_last":7
   },
   {
      "token":"FROG",
      "docid_first":10,
      "docid_last":25
   },
   {
      "token":"HORSE",
      "docid_first":1,
      "docid_last":25
   }
]

frag_counts

Specify TRUE to include the token_count and fragment_count. Here, fragment_count refers to the number of rows in the index tables ($I table, or $G table, or both) used to store the specified token. Note that the fragment_count is not a direct fragmentation measure, but can be used indirectly to assess a “low” or “high” fragmentation for the token.

For example, when the token_count is 3 and the fragment_count is 3, this implies that each instance of the token got stored as a separate row (instead of one compacted row). This indicates fragmentation, which can be resolved by running index optimization.

For example:

JSON Format:
[
   {
      "token":"CAT",
      "token_type":101,
      "token_count":3,
      "fragment_count":1,
      "docid_first":2,
      "docid_last":3,
      "docid_count":2
   },
   {
      "token":"DOG",
      "token_type":101,
      "token_count":5,
      "fragment_count":2,
      "docid_first":4,
      "docid_last":7,
      "docid_count":2
   },
   {
      "token":"FROG",
      "token_type":201,
      "token_count":12,
      "fragment_count":2,
      "docid_first":10,
      "docid_last":25,
      "docid_count":12
   },
   {
      "token":"HORSE",
      "token_type":400,
      "token_count":10,
      "fragment_count":2,
      "docid_first":1,
      "docid_last":25,
      "docid_count":8
   }
]

Related Topics