| Interface | Description | 
|---|---|
| Tokenizer.TokenFactory | 
 Token factory which will create subclass of Token if required 
 | 
| Class | Description | 
|---|---|
| Token<T extends Token> | 
 A Token represents a lexical unit, either an alphanumeric identifier, a
 string or numeric literal, a comment or whitespace etc. 
 | 
| Tokenizer | 
 A utility class for the purpose of splitting PL/SQL source into a list of
  
tokens that enable simple lexical analysis. | 
| Tokenizer.Config | 
 Configuration object that drives the tokenization 
 | 
| TokenPattern<T extends Token> | 
 TokenPattern works in collaboration with the  
Tokenizerto provide a
 search mechanism for Tokens (typically of SQL or PL/SQL). | 
| Enum | Description | 
|---|---|
| Token.Type | 
 The type of Token 
 |