public interface LexerToken
LexerTokeninterface encapsulates a single token retrieved by a call to
Lexer.lex(). This simply provides a convenient way for the lexer to return multiple pieces of information (token class, start offset, end offset) from the Lexer to its clients.
It is defined as an interface to allow language-specific Lexer implementations to add additional information pieces to the token object.
LexerToken is allocated using a call to
Lexer.createLexerToken(), and is passed into the
Lexer.lex() to be filled in.
|Modifier and Type||Method and Description|
Fetches the end offset of the token (offset relative to the original
Fetches the start offset of the token (offset relative to the original
Fetches the (language-specific) token represented by this
LexerToken. You must refer to the language-specific Lexer that was responsible for filling this
LexerTokeninstance to interpret the token.
ReadTextBufferbeing lexed. Note that the start offset is inclusive (compare to end offset which is exclusive), to match the behavior in the
ReadTextBufferbeing lexed. Note that the end offset is exclusive (similar to end offsets in the String class.)