public static interface Tokenizer.TokenFactory
| Modifier and Type | Method and Description |
|---|---|
Token |
createToken(Token.Type type,
int start,
Token prev)
Create the new Token (or typically subclass)
|
void |
setEnd(Token tk,
int end)
Set the end offset of the token
|
Token createToken(Token.Type type, int start, Token prev)
type - The type of tokenstart - The start offset in the source of the tokenprev - The previous tokenvoid setEnd(Token tk, int end)
tk - The token on which to set the end offsetend - The end offset of the token