oracle.javatools.parser.java.v2.scanner

Class ArrayedLexer

java.lang.Object

oracle.javatools.parser.AbstractLexer

oracle.javatools.parser.java.v2.scanner.FastLexer

oracle.javatools.parser.java.v2.scanner.ArrayedLexer

All Implemented Interfaces:

Lexer, LexerToken

public final class ArrayedLexer
extends FastLexer

Lexer based on a precalculated array of tokens.

Nested Class Summary

Nested classes/interfaces inherited from class oracle.javatools.parser.AbstractLexer

AbstractLexer.DefaultLexerToken

Field Summary

Fields inherited from class oracle.javatools.parser.java.v2.scanner.FastLexer

skipComments

Fields inherited from class oracle.javatools.parser.AbstractLexer

currentPos, textBuffer

Fields inherited from interface oracle.javatools.parser.Lexer

TK_EOF, TK_NOT_FOUND

Constructor Summary

Constructors

Constructor and Description
ArrayedLexer(TokenArray tokens, FastLexer baseLexer)

Method Summary

Modifier and Type	Method and Description
int	getEndOffset() Fetches the end offset of the token (offset relative to the original ReadTextBuffer being lexed.
int	getStartOffset() Fetches the start offset of the token (offset relative to the original ReadTextBuffer being lexed.
int	getToken() Fetches the (language-specific) token represented by this LexerToken.
int	getTokenIndex() Calling getTokenIndex() before the initial lex() will produce an unpredictable result.
protected boolean	isComment(int tk) Used by the ArrayedLexer.
int	lex() Calls lex( LexerToken ) but does not call fillLexerToken.
int	lex(LexerToken lexerToken) Scans the text buffer at the current position and returns the token that was found.
void	setPosition(int offset) Sets the current lex (read) position to the given offset in the buffer.
void	setSkipComments(boolean skipComments) Sets whether the FastLexer should generate tokens for Source comments.
void	setTextBuffer(ReadTextBuffer textBuffer) Sets the text buffer that is to be used by this Lexer.
void	setTokenIndex(int index)

Methods inherited from class oracle.javatools.parser.java.v2.scanner.FastLexer

backup

Methods inherited from class oracle.javatools.parser.AbstractLexer

createLexerToken, getTextBuffer

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ArrayedLexer

public ArrayedLexer(TokenArray tokens,
                    FastLexer baseLexer)

Method Detail

setTextBuffer

public void setTextBuffer(ReadTextBuffer textBuffer)

Description copied from class: AbstractLexer

Sets the text buffer that is to be used by this Lexer.

Specified by:

setTextBuffer in interface Lexer

Overrides:

setTextBuffer in class AbstractLexer

Parameters:

textBuffer - the text buffer to use

setPosition

public void setPosition(int offset)

Description copied from class: AbstractLexer

Sets the current lex (read) position to the given offset in the buffer. It is the clients responsibility to ensure that this offset corresponds to the start of a token, otherwise unexpected (and incorrect) results may occur.

Specified by:

setPosition in interface Lexer

Overrides:

setPosition in class AbstractLexer

Parameters:

offset - The start offset of (or in the whitespace previous to) the next token to be returned by lex().

setTokenIndex

public void setTokenIndex(int index)

Parameters:

index - Token index of the next token to be returned by lex().

setSkipComments

public void setSkipComments(boolean skipComments)

Description copied from class: FastLexer

Sets whether the FastLexer should generate tokens for Source comments.

Overrides:

setSkipComments in class FastLexer

Parameters:

skipComments - true to ignore comments in token generation

isComment

protected boolean isComment(int tk)

Description copied from class: FastLexer

Used by the ArrayedLexer.

Overrides:

isComment in class FastLexer

Returns:

True if the indicated token value is considered to be a comment.

getToken

public int getToken()

Description copied from interface: LexerToken

Fetches the (language-specific) token represented by this LexerToken. You must refer to the language-specific Lexer that was responsible for filling this LexerToken instance to interpret the token.

Specified by:

getToken in interface LexerToken

Specified by:

getToken in class FastLexer

Returns:

Token value of the last token returned by lex().

getStartOffset

public int getStartOffset()

Description copied from interface: LexerToken

Fetches the start offset of the token (offset relative to the original ReadTextBuffer being lexed. Note that the start offset is inclusive (compare to end offset which is exclusive), to match the behavior in the String class.

Specified by:

getStartOffset in interface LexerToken

Specified by:

getStartOffset in class FastLexer

Returns:

Start offset of the last token returned by lex().

getEndOffset

public int getEndOffset()

Description copied from interface: LexerToken

Fetches the end offset of the token (offset relative to the original ReadTextBuffer being lexed. Note that the end offset is exclusive (similar to end offsets in the String class.)

Specified by:

getEndOffset in interface LexerToken

Specified by:

getEndOffset in class FastLexer

Returns:

End offset of the last token returned by lex().

getTokenIndex

public int getTokenIndex()

Calling getTokenIndex() before the initial lex() will produce an unpredictable result.

Returns:

The index into the token array of the last token this lexer returned. If the last token was not on a token boundary, -1 is returned to signify an invalid token index.

lex

public int lex()

Description copied from class: FastLexer

Calls lex( LexerToken ) but does not call fillLexerToken.

Specified by:

lex in class FastLexer

lex

public int lex(LexerToken lexerToken)

Description copied from class: AbstractLexer

Scans the text buffer at the current position and returns the token that was found. The token and offset information is also stored in the lexedToken instance passed in to the call.

Specified by:

lex in interface Lexer

Specified by:

lex in class FastLexer

Parameters:

lexerToken - the instance passed in where token info is stored

Returns:

the token that was found, same as calling lexedToken.getToken() (for convenience)