public abstract class AbstractLexer extends java.lang.Object implements Lexer
AbstractLexer
is a basic abstract implementation
which takes care of common functionality across all Lexers, such
as the buffer and offset issues, and for providing a basic
LexerToken
object which just stores token and offset
information. Lexer
,
LexerToken
Modifier and Type | Class and Description |
---|---|
protected static class |
AbstractLexer.DefaultLexerToken
The
DefaultLexerToken just implements a basic
LexerToken which stores a token and start/end offsets. |
Modifier and Type | Field and Description |
---|---|
protected int |
currentPos
The current position in the buffer where the next lex operation
will occur (assuming that
backup() has not been
called.) |
protected ReadTextBuffer |
textBuffer
The underlying text buffer we are lexing.
|
TK_EOF, TK_NOT_FOUND
Constructor and Description |
---|
AbstractLexer() |
Modifier and Type | Method and Description |
---|---|
abstract void |
backup()
Unlexes the last found token.
|
LexerToken |
createLexerToken()
Creates an empty
LexerToken for a specific
Lexer implementation to be passed in to the
lex() call to be filled in. |
ReadTextBuffer |
getTextBuffer()
Gets the text buffer that is being used by this Lexer.
|
abstract int |
lex(LexerToken lexedToken)
Scans the text buffer at the current position and returns the
token that was found.
|
void |
setPosition(int offset)
Sets the current lex (read) position to the given offset in the
buffer.
|
void |
setTextBuffer(ReadTextBuffer buffer)
Sets the text buffer that is to be used by this Lexer.
|
protected ReadTextBuffer textBuffer
protected int currentPos
backup()
has not been
called.)public void setTextBuffer(ReadTextBuffer buffer)
setTextBuffer
in interface Lexer
buffer
- the text buffer to usepublic ReadTextBuffer getTextBuffer()
getTextBuffer
in interface Lexer
public void setPosition(int offset)
setPosition
in interface Lexer
offset
- the offset for the next lex()
operation.public LexerToken createLexerToken()
LexerToken
for a specific
Lexer
implementation to be passed in to the
lex()
call to be filled in.createLexerToken
in interface Lexer
public abstract int lex(LexerToken lexedToken)
lexedToken
instance passed in to the
call.