oracle.javatools.db.token

Class Token<T extends Token>

java.lang.Object

oracle.javatools.db.token.Token<T>

Type Parameters:

T - The class of Token, which will ensure next/previous tokens in the list will be of the expected class.

Direct Known Subclasses:

PlSqlToken

public class Token<T extends Token>
extends java.lang.Object

A Token represents a lexical unit, either an alphanumeric identifier, a string or numeric literal, a comment or whitespace etc. Tokens are produced by the Tokenizer and held in a double linked list for ease of navigation.

Since:

12.2.1

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Token.Type The type of Token

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	Token(Token.Type type, int start, Token prev) Constructor

Method Summary

Modifier and Type	Method and Description
boolean	firstCodeTokenOnLine() Deprecated.
int	getEnd()
protected java.lang.String	getFullSource()
T	getNextCodeToken()
T	getNextCodeToken(int numAhead)
T	getNextToken()
T	getPrevCodeToken()
T	getPrevToken()
java.lang.String	getSource()
java.lang.String	getSource(boolean format) Return the source represented by this token
java.lang.String	getSource(boolean format, Token toToken) Return the source represented from this token to the given end token
int	getStart()
T	getTokenAt(int pos) Return the token at a given offset
Token.Type	getType()
boolean	isCode() Test if the token represents "code".
boolean	isCode(boolean ignorePuncAndLiterals) Test if the token represents "code".
boolean	isComment() Test if the token represents some form of comment.
boolean	isEndMarker() Test if the token is one of the two end markers of the linked list.
boolean	isNumber() Test if the token represents a number.
boolean	isPunctuation() Test if the token represents some form of punctuation symbol.
boolean	isWhiteSpace() Test if the token represents white space.
boolean	isWord() Test if the token represents a word".
boolean	matches(java.lang.String str) Tests if the source of the token is equivalent to the given string
void	setEnd(int end) Set the end offset of the token
protected void	setHeaderSourceImpl(java.lang.String source) Records the source that is being tokenized against this token, which must be the fist in the list.
void	setNextToken(Token nextToken)
void	setPrevToken(Token prevToken) Set the previous token in the list
protected void	setType(Token.Type type) Set the type of the token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Token

protected Token(Token.Type type,
                int start,
                Token prev)

Constructor

Parameters:

type - The type of token

start - The start offset of the token to construct

prev - The token before it in the list

Method Detail

setHeaderSourceImpl

protected void setHeaderSourceImpl(java.lang.String source)

Records the source that is being tokenized against this token, which must be the fist in the list.

Parameters:

source - The source being tokenized.

getType

public Token.Type getType()

Returns:

The type of token.

setType

protected final void setType(Token.Type type)

Set the type of the token.

Parameters:

type - The token type.

getStart

public int getStart()

Returns:

the start offset of the token.

getEnd

public int getEnd()

Returns:

the end offset of the token.

setEnd

public void setEnd(int end)

Set the end offset of the token

Parameters:

the - start offset of the token.

getNextToken

public T getNextToken()

Returns:

the next Token in the list

setNextToken

public void setNextToken(Token nextToken)

getPrevToken

public T getPrevToken()

Returns:

the previous Token in the list

setPrevToken

public void setPrevToken(Token prevToken)

Set the previous token in the list

Parameters:

prevToken -

isCode

public boolean isCode()

Test if the token represents "code". That is something other than a comment or whitespace.

isWord

public boolean isWord()

Test if the token represents a word". That is something other than a comment, whitespace, or numeric string or literal.

isPunctuation

public boolean isPunctuation()

Test if the token represents some form of punctuation symbol.

isWhiteSpace

public boolean isWhiteSpace()

Test if the token represents white space.

isComment

public boolean isComment()

Test if the token represents some form of comment.

isEndMarker

public boolean isEndMarker()

Test if the token is one of the two end markers of the linked list.

isCode

public boolean isCode(boolean ignorePuncAndLiterals)

Test if the token represents "code". That is something other than a comment or whitespace. If specified, punctuation and literals can be ignored. This is equivalent to isWord().

Parameters:

ignorePuncAndLiterals - set to true if the value of this method for tokens representing string and numeric literals should be false.

isNumber

public boolean isNumber()

Test if the token represents a number.

getPrevCodeToken

public T getPrevCodeToken()

Returns:

the previous Code Token (ignoring whitespace and comments) in the list

getNextCodeToken

public T getNextCodeToken()

Returns:

the next Code Token (ignoring whitespace and comments) in the list

getNextCodeToken

public T getNextCodeToken(int numAhead)

Returns:

the skips forward the given number of Code Tokens (ignoring whitespace and comments) in the list

getSource

public java.lang.String getSource()

Returns:

return the source represented by this token

getSource

public java.lang.String getSource(boolean format)

Return the source represented by this token

Parameters:

format - Applies formatting if true

Returns:

getSource

public java.lang.String getSource(boolean format,
                                  Token toToken)

Return the source represented from this token to the given end token

Parameters:

format -

toToken -

Returns:

getFullSource

protected java.lang.String getFullSource()

matches

public boolean matches(java.lang.String str)

Tests if the source of the token is equivalent to the given string

firstCodeTokenOnLine

@Deprecated
public boolean firstCodeTokenOnLine()

Deprecated.

getTokenAt

public T getTokenAt(int pos)

Return the token at a given offset