T
- The class of Token, which will ensure next/previous tokens in the
list will be of the expected class.public class Token<T extends Token>
extends java.lang.Object
Modifier and Type | Class and Description |
---|---|
static class |
Token.Type
The type of Token
|
Modifier | Constructor and Description |
---|---|
protected |
Token(Token.Type type,
int start,
Token prev)
Constructor
|
Modifier and Type | Method and Description |
---|---|
boolean |
firstCodeTokenOnLine()
Deprecated.
|
int |
getEnd() |
protected java.lang.String |
getFullSource() |
T |
getNextCodeToken() |
T |
getNextCodeToken(int numAhead) |
T |
getNextToken() |
T |
getPrevCodeToken() |
T |
getPrevToken() |
java.lang.String |
getSource() |
java.lang.String |
getSource(boolean format)
Return the source represented by this token
|
java.lang.String |
getSource(boolean format,
Token toToken)
Return the source represented from this token to the given end token
|
int |
getStart() |
T |
getTokenAt(int pos)
Return the token at a given offset
|
Token.Type |
getType() |
boolean |
isCode()
Test if the token represents "code".
|
boolean |
isCode(boolean ignorePuncAndLiterals)
Test if the token represents "code".
|
boolean |
isComment()
Test if the token represents some form of comment.
|
boolean |
isEndMarker()
Test if the token is one of the two end markers of the linked list.
|
boolean |
isNumber()
Test if the token represents a number.
|
boolean |
isPunctuation()
Test if the token represents some form of punctuation symbol.
|
boolean |
isWhiteSpace()
Test if the token represents white space.
|
boolean |
isWord()
Test if the token represents a word".
|
boolean |
matches(java.lang.String str)
Tests if the source of the token is equivalent to the given string
|
void |
setEnd(int end)
Set the end offset of the token
|
protected void |
setHeaderSourceImpl(java.lang.String source)
Records the source that is being tokenized against this token, which
must be the fist in the list.
|
void |
setNextToken(Token nextToken) |
void |
setPrevToken(Token prevToken)
Set the previous token in the list
|
protected void |
setType(Token.Type type)
Set the type of the token.
|
protected Token(Token.Type type, int start, Token prev)
type
- The type of tokenstart
- The start offset of the token to constructprev
- The token before it in the listprotected void setHeaderSourceImpl(java.lang.String source)
source
- The source being tokenized.public Token.Type getType()
protected final void setType(Token.Type type)
type
- The token type.public int getStart()
public int getEnd()
public void setEnd(int end)
the
- start offset of the token.public T getNextToken()
public void setNextToken(Token nextToken)
public T getPrevToken()
public void setPrevToken(Token prevToken)
prevToken
- public boolean isCode()
public boolean isWord()
public boolean isPunctuation()
public boolean isWhiteSpace()
public boolean isComment()
public boolean isEndMarker()
public boolean isCode(boolean ignorePuncAndLiterals)
isWord()
.ignorePuncAndLiterals
- set to true if the value of this method for
tokens representing string and numeric literals should be false.public boolean isNumber()
public T getPrevCodeToken()
public T getNextCodeToken()
public T getNextCodeToken(int numAhead)
public java.lang.String getSource()
public java.lang.String getSource(boolean format)
format
- Applies formatting if truepublic java.lang.String getSource(boolean format, Token toToken)
format
- toToken
- protected java.lang.String getFullSource()
public boolean matches(java.lang.String str)
@Deprecated public boolean firstCodeTokenOnLine()
public T getTokenAt(int pos)