public interface SyntaxToken
SyntaxToken interface encapsulates a single syntax
 token retrieved by a call to
 SyntaxTranslator.translate(). This simply provides a
 convenient way for the translater to return multiple pieces of
 information (translate code, start offset, end offset) from the
 SyntaxTranslator to its clients. It is defined as an interface to allow language-specific Lexer implementations to add additional information pieces to the token object.
 The SyntaxToken is allocated using a call to
 SyntaxRecognizer.createSyntaxToken(), and is passed into
 SyntaxRecognizer.next() to be filled in. 
SyntaxRecognizer| Modifier and Type | Field and Description | 
|---|---|
static int | 
SYNTAX_UNINITIALIZED  | 
| Modifier and Type | Method and Description | 
|---|---|
int | 
getCode()
Fetches the (language-specific) token represented by this
  
SyntaxToken. | 
java.lang.String | 
getContextString()
If non-null, this String provides information about the parsing
 context of this token. 
 | 
int | 
getDataValue()
There is often some numeric integer value that can be associated
 with specific syntax tokens. 
 | 
int | 
getEndOffset()
Fetches the end offset of the token (offset relative to the original
  
TextBuffer being lexed. | 
int | 
getStartOffset()
Fetches the start offset of the token (offset relative to the
 original  
TextBuffer being lexed. | 
static final int SYNTAX_UNINITIALIZED
int getCode()
SyntaxToken. You must refer to the language-specific
 SyntaxTranslator that was responsible for filling this
 SyntaxToken instance to interpret the token.int getStartOffset()
TextBuffer being lexed.  Note that the
 start offset is inclusive (compare to end offset which is
 exclusive), to match the behavior in the String
 class.int getEndOffset()
TextBuffer being lexed.  Note that the end offset is
 exclusive (similar to end offsets in the String class.)int getDataValue()
SyntaxToken instance to interpret the
 token.java.lang.String getContextString()
getCode() == SyntaxRecognizer.SYNTAX_ERROR, this
 will return the syntax error message.