public abstract class AbstractLexer extends java.lang.Object implements Lexer
AbstractLexer
is a basic abstract implementation which takes care of common functionality across all Lexers, such as the buffer and offset issues, and for providing a basic LexerToken
object which just stores token and offset information.Lexer
, LexerToken
Modifier and Type | Class and Description |
---|---|
protected static class |
AbstractLexer.DefaultLexerToken
The
DefaultLexerToken just implements a basic LexerToken which stores a token and start/end offsets. |
Modifier and Type | Field and Description |
---|---|
protected int |
currentPos
The current position in the buffer where the next lex operation will occur (assuming that
backup() has not been called.) |
protected ReadTextBuffer |
textBuffer
The underlying text buffer we are lexing.
|
TK_EOF, TK_NOT_FOUND
Constructor and Description |
---|
AbstractLexer() |
Modifier and Type | Method and Description |
---|---|
abstract void |
backup()
Unlexes the last found token.
|
LexerToken |
createLexerToken()
Creates an empty
LexerToken for a specific Lexer implementation to be passed in to the lex() call to be filled in. |
ReadTextBuffer |
getTextBuffer()
Gets the text buffer that is being used by this Lexer.
|
abstract int |
lex(LexerToken lexedToken)
Scans the text buffer at the current position and returns the token that was found.
|
void |
setPosition(int offset)
Sets the current lex (read) position to the given offset in the buffer.
|
void |
setTextBuffer(ReadTextBuffer buffer)
Sets the text buffer that is to be used by this Lexer.
|
protected ReadTextBuffer textBuffer
protected int currentPos
backup()
has not been called.)public void setTextBuffer(ReadTextBuffer buffer)
setTextBuffer
in interface Lexer
buffer
- the text buffer to usepublic ReadTextBuffer getTextBuffer()
getTextBuffer
in interface Lexer
public void setPosition(int offset)
setPosition
in interface Lexer
offset
- the offset for the next lex()
operation.public LexerToken createLexerToken()
LexerToken
for a specific Lexer
implementation to be passed in to the lex()
call to be filled in.createLexerToken
in interface Lexer
public abstract int lex(LexerToken lexedToken)
lexedToken
instance passed in to the call.