public class JavaLexer extends FastLexer implements JavaTokens, LexerToken
JavaLexer is an implementation of the Lexer
interface for the Source language. Details of the Source Language taken from
Source Language SpecificationLexer,
JavaTokensAbstractLexer.DefaultLexerToken| Modifier and Type | Field and Description |
|---|---|
static KeywordTable |
keyword13Table
The keyword table for JDK 1.3.
|
static KeywordTable |
keyword14Table
The keyword table for JDK 1.4.
|
static KeywordTable |
keyword15Table
The keyword table for JDK 1.5.
|
skipCommentscurrentPos, textBufferDW_values, DW_words, J2SE_13, J2SE_14, J2SE_15, J2SE_16, J2SE_17, J2SE_18, J2SE_MAX, J2SE_MIN, KW_ABSTRACT, KW_ASSERT, KW_base, KW_BOOLEAN, KW_BREAK, KW_BYTE, KW_CASE, KW_CATCH, KW_CHAR, KW_CLASS, KW_CONST, KW_CONTINUE, KW_DEFAULT, KW_DO, KW_DOUBLE, KW_ELSE, KW_ENUM, KW_EXTENDS, KW_FINAL, KW_FINALLY, KW_FLOAT, KW_FOR, KW_GOTO, KW_IF, KW_IMPLEMENTS, KW_IMPORT, KW_INSTANCEOF, KW_INT, KW_INTERFACE, KW_j2se13_max, KW_j2se14_max, KW_j2se15_max, KW_j2se16_max, KW_j2se17_max, KW_j2se18_max, KW_j2se19_max, KW_LONG, KW_max, KW_NATIVE, KW_NEW, KW_PACKAGE, KW_PRIVATE, KW_PROTECTED, KW_PUBLIC, KW_RETURN, KW_SHORT, KW_STATIC, KW_STRICTFP, KW_SUPER, KW_SWITCH, KW_SYNCHRONIZED, KW_THIS, KW_THROW, KW_THROWS, KW_TRANSIENT, KW_TRY, KW_VOID, KW_VOLATILE, KW_WHILE, KW_words, OP_AND_ASG, OP_ASG, OP_AT, OP_base, OP_BINAND, OP_binary, OP_BINNOT, OP_BINOR, OP_BINXOR, OP_COLON, OP_COLONCOLON, OP_COMMA, OP_DEC, OP_DIV, OP_DIV_ASG, OP_DOT, OP_ELLIPSIS, OP_EQ, OP_GE, OP_GT, OP_HASH, OP_INC, OP_LBRACE, OP_LBRACKET, OP_LE, OP_LOGAND, OP_LOGNOT, OP_LOGOR, OP_LPAREN, OP_LSH, OP_LSH_ASG, OP_LT, OP_max, OP_MINUS, OP_MINUS_ASG, OP_MOD, OP_MOD_ASG, OP_MUL, OP_MUL_ASG, OP_NE, OP_OR_ASG, OP_PLUS, OP_PLUS_ASG, OP_prefix, OP_QUES, OP_RBRACE, OP_RBRACKET, OP_RIGHT_ARROW, OP_RPAREN, OP_RSH, OP_RSH_ASG, OP_SEMI, OP_URSH, OP_URSH_ASG, OP_words, OP_XOR_ASG, TK_base, TK_BOOLEAN_LITERAL, TK_CHAR_LITERAL, TK_DOC_COMMENT, TK_DOUBLE_LITERAL, TK_FLOAT_LITERAL, TK_IDENTIFIER, TK_INT_LITERAL, TK_LITERAL_base, TK_LITERAL_max, TK_LONG_LITERAL, TK_max, TK_MULTI_COMMENT, TK_NULL_LITERAL, TK_SINGLE_COMMENT, TK_SQLJ, TK_STRING_LITERAL, TK_WHITESPACE, TK_wordsTK_NOT_FOUND| Constructor and Description |
|---|
JavaLexer()
Constructs a default
JavaLexer with a starting
position of 0. |
JavaLexer(byte j2se)
Deprecated.
Use JavaLexer( JdkVersion ) instead
|
JavaLexer(JdkVersion jdkVersion)
Constructs a default
JavaLexer with a starting
position of 0. |
| Modifier and Type | Method and Description |
|---|---|
void |
backup()
Unlexes the last found token.
|
int |
getEndOffset()
Fetches the end offset of the token (offset relative to the original
ReadTextBuffer being lexed. |
int |
getStartOffset()
Fetches the start offset of the token (offset relative to the
original
ReadTextBuffer being lexed. |
int |
getToken()
Fetches the (language-specific) token represented by this
LexerToken. |
protected boolean |
isComment(int tk)
Used by the ArrayedLexer.
|
static boolean |
isDecimalDigit(char digit)
Utility routine to check whether the given digit is a decimal
digit.
|
static boolean |
isHexDigit(char digit)
Utility routine to check whether the given digit is a hexadecimal
digit.
|
static boolean |
isJavaIdentifierPart(char c)
The lexer maintain a fast lookup table for ascii characters,
and calling this method is faster than calling
Character.isJavaIdentifierPart.
|
static boolean |
isJavaIdentifierStart(char c)
The lexer maintain a fast lookup table for ascii characters,
and calling this method is faster than calling
Character.isJavaIdentifierStart.
|
static boolean |
isOctalDigit(char digit)
Utility routine to check whether the given digit is an octal
digit.
|
int |
lex()
Calls lex( LexerToken ) but does not call fillLexerToken.
|
int |
lex(LexerToken lexedToken)
Scans the text buffer at the current position and returns the
token that was found.
|
void |
setPosition(int offset)
Sets the current lex (read) position to the given offset in the
buffer.
|
void |
setRecognizeSQLJ(boolean recognizeSQLJ)
Deprecated.
SQLJ statements (#sql) are not supported anymore
|
setSkipCommentscreateLexerToken, getTextBuffer, setTextBufferpublic static final KeywordTable keyword13Table
public static final KeywordTable keyword14Table
public static final KeywordTable keyword15Table
public JavaLexer()
JavaLexer with a starting
position of 0. Clients must call setTextBuffer() to
initialize the text buffer used for the Lexer. To start lexing
from an offset other than 0, call setPosition().
NOTE: this lexer will use the maximum supported JDK version
for syntax rules. Use lex( byte ) to do a lex targeting
a specific JDK version.@Deprecated public JavaLexer(byte j2se)
JavaLexer with a starting
position of 0. Clients must call setTextBuffer() to
initialize the text buffer used for the Lexer. To start lexing
from an offset other than 0, call setPosition().j2se - Indicates the JDK version this lexer will use for
syntax rules, use one of JavaConstants.J2SE_XXX.public JavaLexer(JdkVersion jdkVersion)
JavaLexer with a starting
position of 0. Clients must call setTextBuffer() to
initialize the text buffer used for the Lexer. To start lexing
from an offset other than 0, call setPosition().jdkVersion - Indicates the JDK version this lexer will use for
syntax rules, use one of JavaConstants.J2SE_XXX.public void setPosition(int offset)
AbstractLexersetPosition in interface LexersetPosition in class AbstractLexeroffset - the offset for the next lex() operation.@Deprecated public void setRecognizeSQLJ(boolean recognizeSQLJ)
JavaLexer should recognize SQLJ
statements and return a single token for them (instead of trying
to lex it as Source).recognizeSQLJ - true to recognize SQLJ statements as a tokenprotected boolean isComment(int tk)
FastLexerpublic int getToken()
LexerTokenLexerToken. You must refer to the language-specific
Lexer that was responsible for filling this
LexerToken instance to interpret the token.getToken in interface LexerTokengetToken in class FastLexerpublic int getStartOffset()
LexerTokenReadTextBuffer being lexed. Note that the
start offset is inclusive (compare to end offset which is
exclusive), to match the behavior in the String
class.getStartOffset in interface LexerTokengetStartOffset in class FastLexerpublic int getEndOffset()
LexerTokenReadTextBuffer being lexed. Note that the end offset is
exclusive (similar to end offsets in the String class.)getEndOffset in interface LexerTokengetEndOffset in class FastLexerpublic int lex()
public int lex(LexerToken lexedToken)
lexedToken instance passed in to the
call.public void backup()
FastLexerlex()
will return the last token and offset information found.public static boolean isDecimalDigit(char digit)
digit - the digit to checkpublic static boolean isOctalDigit(char digit)
digit - the digit to checkpublic static boolean isHexDigit(char digit)
digit - the digit to checkpublic static boolean isJavaIdentifierStart(char c)
c - A characterpublic static boolean isJavaIdentifierPart(char c)
c - A character