public static interface Tokenizer.TokenFactory
| Modifier and Type | Method and Description | 
|---|---|
Token | 
createToken(Token.Type type,
           int start,
           Token prev)
Create the new Token (or typically subclass) 
 | 
void | 
setEnd(Token tk,
      int end)
Set the end offset of the token 
 | 
Token createToken(Token.Type type, int start, Token prev)
type - The type of tokenstart - The start offset in the source of the tokenprev - The previous tokenvoid setEnd(Token tk, int end)
tk - The token on which to set the end offsetend - The end offset of the token