oracle.javatools.parser.java.v2.scanner

Class FastLexer

java.lang.Object

oracle.javatools.parser.AbstractLexer

oracle.javatools.parser.java.v2.scanner.FastLexer

All Implemented Interfaces:

Lexer, LexerToken

Direct Known Subclasses:

ArrayedLexer, CSSLexer, JavadocLexer, JavaLexer

public abstract class FastLexer
extends AbstractLexer
implements LexerToken

The FastLexer features two optimizations that make it faster than having just a Lexer.

1. The Lexer interface methods are implemented by abstract methods. This means that the VM calls invokevirtual instead of invokeinterfacey for a performance saving (on the order of factor-4).

2. The FastLexer has a lex() method that takes no LexerToken. This saves on the field copying done by a lexer( LexerToken ).

See Also:

Lexer

Nested Class Summary

Nested classes/interfaces inherited from class oracle.javatools.parser.AbstractLexer

AbstractLexer.DefaultLexerToken

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	skipComments Whether to skip comments or not.

Fields inherited from class oracle.javatools.parser.AbstractLexer

currentPos, textBuffer

Fields inherited from interface oracle.javatools.parser.Lexer

TK_EOF, TK_NOT_FOUND

Constructor Summary

Constructors

Constructor and Description
FastLexer()

Method Summary

Modifier and Type	Method and Description
void	backup() Unlexes the last found token.
abstract int	getEndOffset() Fetches the end offset of the token (offset relative to the original ReadTextBuffer being lexed.
abstract int	getStartOffset() Fetches the start offset of the token (offset relative to the original ReadTextBuffer being lexed.
abstract int	getToken() Fetches the (language-specific) token represented by this LexerToken.
protected boolean	isComment(int tk) Used by the ArrayedLexer.
abstract int	lex() Calls lex( LexerToken ) but does not call fillLexerToken.
abstract int	lex(LexerToken lexedToken) Scans the text buffer at the current position and returns the token that was found.
void	setSkipComments(boolean skipComments) Sets whether the FastLexer should generate tokens for Source comments.

Methods inherited from class oracle.javatools.parser.AbstractLexer

createLexerToken, getTextBuffer, setPosition, setTextBuffer

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

skipComments

protected boolean skipComments

Whether to skip comments or not.

Constructor Detail

FastLexer

public FastLexer()

Method Detail

setSkipComments

public void setSkipComments(boolean skipComments)

Sets whether the FastLexer should generate tokens for Source comments.

Parameters:

skipComments - true to ignore comments in token generation

isComment

protected boolean isComment(int tk)

Used by the ArrayedLexer.

Returns:

True if the indicated token value is considered to be a comment.

getToken

public abstract int getToken()

Description copied from interface: LexerToken

Fetches the (language-specific) token represented by this LexerToken. You must refer to the language-specific Lexer that was responsible for filling this LexerToken instance to interpret the token.

Specified by:

getToken in interface LexerToken

Returns:

the lexer token

getStartOffset

public abstract int getStartOffset()

Description copied from interface: LexerToken

Fetches the start offset of the token (offset relative to the original ReadTextBuffer being lexed. Note that the start offset is inclusive (compare to end offset which is exclusive), to match the behavior in the String class.

Specified by:

getStartOffset in interface LexerToken

Returns:

the starting offset

getEndOffset

public abstract int getEndOffset()

Description copied from interface: LexerToken

Fetches the end offset of the token (offset relative to the original ReadTextBuffer being lexed. Note that the end offset is exclusive (similar to end offsets in the String class.)

Specified by:

getEndOffset in interface LexerToken

Returns:

the ending offset

lex

public abstract int lex()

Calls lex( LexerToken ) but does not call fillLexerToken.

lex

public abstract int lex(LexerToken lexedToken)

Description copied from class: AbstractLexer

Scans the text buffer at the current position and returns the token that was found. The token and offset information is also stored in the lexedToken instance passed in to the call.

Specified by:

lex in interface Lexer

Specified by:

lex in class AbstractLexer

Parameters:

lexedToken - the instance passed in where token info is stored

Returns:

the token that was found, same as calling lexedToken.getToken() (for convenience)

backup

public void backup()

Unlexes the last found token. The next call to lex() will return the last token and offset information found.

Specified by:

backup in interface Lexer

Specified by:

backup in class AbstractLexer