public class JavaLexer extends FastLexer implements JavaTokens, LexerToken
JavaLexer
is an implementation of the Lexer
interface for the Source language. Details of the Source Language taken from Source Language SpecificationLexer
, JavaTokens
AbstractLexer.DefaultLexerToken
Modifier and Type | Field and Description |
---|---|
static KeywordTable |
keyword13Table
The keyword table for JDK 1.3.
|
static KeywordTable |
keyword14Table
The keyword table for JDK 1.4.
|
static KeywordTable |
keyword15Table
The keyword table for JDK 1.5.
|
skipComments
currentPos, textBuffer
DW_values, DW_words, J2SE_13, J2SE_14, J2SE_15, J2SE_16, J2SE_17, J2SE_18, J2SE_MAX, J2SE_MIN, KW_ABSTRACT, KW_ASSERT, KW_base, KW_BOOLEAN, KW_BREAK, KW_BYTE, KW_CASE, KW_CATCH, KW_CHAR, KW_CLASS, KW_CONST, KW_CONTINUE, KW_DEFAULT, KW_DO, KW_DOUBLE, KW_ELSE, KW_ENUM, KW_EXTENDS, KW_FINAL, KW_FINALLY, KW_FLOAT, KW_FOR, KW_GOTO, KW_IF, KW_IMPLEMENTS, KW_IMPORT, KW_INSTANCEOF, KW_INT, KW_INTERFACE, KW_j2se13_max, KW_j2se14_max, KW_j2se15_max, KW_j2se16_max, KW_j2se17_max, KW_j2se18_max, KW_LONG, KW_max, KW_NATIVE, KW_NEW, KW_PACKAGE, KW_PRIVATE, KW_PROTECTED, KW_PUBLIC, KW_RETURN, KW_SHORT, KW_STATIC, KW_STRICTFP, KW_SUPER, KW_SWITCH, KW_SYNCHRONIZED, KW_THIS, KW_THROW, KW_THROWS, KW_TRANSIENT, KW_TRY, KW_VOID, KW_VOLATILE, KW_WHILE, KW_words, OP_AND_ASG, OP_ASG, OP_AT, OP_base, OP_BINAND, OP_binary, OP_BINNOT, OP_BINOR, OP_BINXOR, OP_COLON, OP_COLONCOLON, OP_COMMA, OP_DEC, OP_DIV, OP_DIV_ASG, OP_DOT, OP_ELLIPSIS, OP_EQ, OP_GE, OP_GT, OP_HASH, OP_INC, OP_LBRACE, OP_LBRACKET, OP_LE, OP_LOGAND, OP_LOGNOT, OP_LOGOR, OP_LPAREN, OP_LSH, OP_LSH_ASG, OP_LT, OP_max, OP_MINUS, OP_MINUS_ASG, OP_MOD, OP_MOD_ASG, OP_MUL, OP_MUL_ASG, OP_NE, OP_OR_ASG, OP_PLUS, OP_PLUS_ASG, OP_prefix, OP_QUES, OP_RBRACE, OP_RBRACKET, OP_RIGHT_ARROW, OP_RPAREN, OP_RSH, OP_RSH_ASG, OP_SEMI, OP_URSH, OP_URSH_ASG, OP_words, OP_XOR_ASG, TK_base, TK_BOOLEAN_LITERAL, TK_CHAR_LITERAL, TK_DOC_COMMENT, TK_DOUBLE_LITERAL, TK_FLOAT_LITERAL, TK_IDENTIFIER, TK_INT_LITERAL, TK_LITERAL_base, TK_LITERAL_max, TK_LONG_LITERAL, TK_max, TK_MULTI_COMMENT, TK_NULL_LITERAL, TK_SINGLE_COMMENT, TK_SQLJ, TK_STRING_LITERAL, TK_WHITESPACE, TK_words
TK_NOT_FOUND
Constructor and Description |
---|
JavaLexer()
Constructs a default
JavaLexer with a starting position of 0. |
JavaLexer(byte j2se)
Deprecated.
Use JavaLexer( JdkVersion ) instead
|
JavaLexer(JdkVersion jdkVersion)
Constructs a default
JavaLexer with a starting position of 0. |
Modifier and Type | Method and Description |
---|---|
void |
backup()
Unlexes the last found token.
|
int |
getEndOffset()
Fetches the end offset of the token (offset relative to the original
ReadTextBuffer being lexed. |
int |
getStartOffset()
Fetches the start offset of the token (offset relative to the original
ReadTextBuffer being lexed. |
int |
getToken()
Fetches the (language-specific) token represented by this
LexerToken . |
protected boolean |
isComment(int tk)
Used by the ArrayedLexer.
|
static boolean |
isDecimalDigit(char digit)
Utility routine to check whether the given digit is a decimal digit.
|
static boolean |
isHexDigit(char digit)
Utility routine to check whether the given digit is a hexadecimal digit.
|
static boolean |
isJavaIdentifierPart(char c)
The lexer maintain a fast lookup table for ascii characters, and calling this method is faster than calling Character.isJavaIdentifierPart
|
static boolean |
isJavaIdentifierStart(char c)
The lexer maintain a fast lookup table for ascii characters, and calling this method is faster than calling Character.isJavaIdentifierStart
|
static boolean |
isOctalDigit(char digit)
Utility routine to check whether the given digit is an octal digit.
|
int |
lex()
Calls lex( LexerToken ) but does not call fillLexerToken.
|
int |
lex(LexerToken lexedToken)
Scans the text buffer at the current position and returns the token that was found.
|
void |
setPosition(int offset)
Sets the current lex (read) position to the given offset in the buffer.
|
void |
setRecognizeSQLJ(boolean recognizeSQLJ)
Sets whether the
JavaLexer should recognize SQLJ statements and return a single token for them (instead of trying to lex it as Source.) |
setSkipComments
createLexerToken, getTextBuffer, setTextBuffer
public static final KeywordTable keyword13Table
public static final KeywordTable keyword14Table
public static final KeywordTable keyword15Table
public JavaLexer()
JavaLexer
with a starting position of 0. Clients must call setTextBuffer()
to initialize the text buffer used for the Lexer. To start lexing from an offset other than 0, call setPosition()
. NOTE: this lexer will use the maximum supported JDK version for syntax rules. Use lex( byte ) to do a lex targeting a specific JDK version.@Deprecated public JavaLexer(byte j2se)
JavaLexer
with a starting position of 0. Clients must call setTextBuffer()
to initialize the text buffer used for the Lexer. To start lexing from an offset other than 0, call setPosition()
.j2se
- Indicates the JDK version this lexer will use for syntax rules, use one of JavaConstants.J2SE_XXX.public JavaLexer(JdkVersion jdkVersion)
JavaLexer
with a starting position of 0. Clients must call setTextBuffer()
to initialize the text buffer used for the Lexer. To start lexing from an offset other than 0, call setPosition()
.jdkVersion
- Indicates the JDK version this lexer will use for syntax rules, use one of JavaConstants.J2SE_XXX.public void setPosition(int offset)
AbstractLexer
setPosition
in interface Lexer
setPosition
in class AbstractLexer
offset
- the offset for the next lex()
operation.public void setRecognizeSQLJ(boolean recognizeSQLJ)
JavaLexer
should recognize SQLJ statements and return a single token for them (instead of trying to lex it as Source.)recognizeSQLJ
- true to recognize SQLJ statements as a tokenprotected boolean isComment(int tk)
FastLexer
public int getToken()
LexerToken
LexerToken
. You must refer to the language-specific Lexer that was responsible for filling this LexerToken
instance to interpret the token.getToken
in interface LexerToken
getToken
in class FastLexer
public int getStartOffset()
LexerToken
ReadTextBuffer
being lexed. Note that the start offset is inclusive (compare to end offset which is exclusive), to match the behavior in the String
class.getStartOffset
in interface LexerToken
getStartOffset
in class FastLexer
public int getEndOffset()
LexerToken
ReadTextBuffer
being lexed. Note that the end offset is exclusive (similar to end offsets in the String class.)getEndOffset
in interface LexerToken
getEndOffset
in class FastLexer
public int lex()
public int lex(LexerToken lexedToken)
lexedToken
instance passed in to the call.public void backup()
FastLexer
lex()
will return the last token and offset information found.public static boolean isDecimalDigit(char digit)
digit
- the digit to checkpublic static boolean isOctalDigit(char digit)
digit
- the digit to checkpublic static boolean isHexDigit(char digit)
digit
- the digit to checkpublic static boolean isJavaIdentifierStart(char c)
c
- A characterpublic static boolean isJavaIdentifierPart(char c)
c
- A character