Package org.codehaus.groovy.syntax

Class Token

java.lang.Object
org.codehaus.groovy.syntax.CSTNode
org.codehaus.groovy.syntax.Token
public class Token extends CSTNode
A CSTNode produced by the Lexer during lexical analysis. Represents a single token in the token stream, including its type, text content, and position in the source.
See Also:

  • Field Details

    • EOF

      public static final Token EOF
      Sentinel token indicating end-of-file.

    • NULL

      public static final Token NULL
      Sentinel token for null/unknown positions.

  • Constructor Details

    • Token

      public Token(int type, String text, int startLine, int startColumn)
      Constructs a Token with the specified type, text, and source position.
      Parameters:
      type - the token type from Types
      text - the token's text content
      startLine - the source line number (1-based, or -1 if unknown)
      startColumn - the source column number (1-based, or -1 if unknown)

  • Method Details

    • dup

      public Token dup()
      Creates a shallow copy of this Token, preserving type, text, position, and meaning.
      Returns:
      a new Token with the same attributes as this one

    • getMeaning

      public int getMeaning()
      Returns the current meaning (interpretation) of this token. May differ from getType() if the meaning has been reassigned.
      Overrides:
      getMeaning in class CSTNode
      Returns:
      the token's current meaning type

    • setMeaning

      public CSTNode setMeaning(int meaning)
      Sets the meaning (interpretation) for this token. Has no effect on the sentinel tokens (EOF and NULL).
      Overrides:
      setMeaning in class CSTNode
      Parameters:
      meaning - the new meaning type from Types
      Returns:
      this token, for convenience chaining

    • getType

      public int getType()
      Returns the actual type of this token as determined by the lexer.
      Overrides:
      getType in class CSTNode
      Returns:
      the token type from Types

    • size

      public int size()
      Returns the number of elements in this node, which is always 1 (the token itself).
      Specified by:
      size in class CSTNode
      Returns:
      1

    • get

      public CSTNode get(int index)
      Returns the element at the specified index. Only index 0 (the token itself) is valid.
      Specified by:
      get in class CSTNode
      Parameters:
      index - the element index
      Returns:
      this token if index is 0
      Throws:
      GroovyBugError - if index is greater than 0

    • getRoot

      public Token getRoot()
      Returns the root token of this node, which is always this token itself.
      Specified by:
      getRoot in class CSTNode
      Returns:
      this token

    • getRootText

      public String getRootText()
      Returns the text content of this token. Equivalent to getText().
      Overrides:
      getRootText in class CSTNode
      Returns:
      the token's text content

    • getText

      public String getText()
      Returns the text content of this token.
      Returns:
      the token's text content

    • setText

      public void setText(String text)
      Sets the text content of this token. Has no effect on the sentinel tokens (EOF and NULL).
      Parameters:
      text - the new text content

    • getStartLine

      public int getStartLine()
      Returns the starting line number of this token in the source.
      Overrides:
      getStartLine in class CSTNode
      Returns:
      the line number (1-based), or -1 if not known

    • getStartColumn

      public int getStartColumn()
      Returns the starting column number of this token in the source.
      Overrides:
      getStartColumn in class CSTNode
      Returns:
      the column number (1-based), or -1 if not known

    • asReduction

      public Reduction asReduction()
      Converts this token to a Reduction with this token as the root.
      Specified by:
      asReduction in class CSTNode
      Returns:
      a new Reduction containing this token as its root element

    • asReduction

      public Reduction asReduction(CSTNode second)
      Converts this token to a Reduction with this token as the root and the specified node as the second element.
      Parameters:
      second - the second element to add to the reduction
      Returns:
      a new Reduction containing this token and the second element

    • asReduction

      public Reduction asReduction(CSTNode second, CSTNode third)
      Converts this token to a Reduction with this token as the root and the specified nodes as the second and third elements.
      Parameters:
      second - the second element to add to the reduction
      third - the third element to add to the reduction
      Returns:
      a new Reduction containing this token and the specified elements

    • asReduction

      public Reduction asReduction(CSTNode second, CSTNode third, CSTNode fourth)
      Converts this token to a Reduction with this token as the root and the specified nodes as the second, third, and fourth elements.
      Parameters:
      second - the second element to add to the reduction
      third - the third element to add to the reduction
      fourth - the fourth element to add to the reduction
      Returns:
      a new Reduction containing this token and the specified elements

    • newKeyword

      public static Token newKeyword(String text, int startLine, int startColumn)
      Factory method to create a keyword token if the given text represents a known keyword.
      Parameters:
      text - the text to check as a keyword
      startLine - the source line number (1-based)
      startColumn - the source column number (1-based)
      Returns:
      a Token with the appropriate keyword type, or null if the text is not a keyword

    • newString

      public static Token newString(String text, int startLine, int startColumn)
      Factory method to create a string literal token.
      Parameters:
      text - the string content
      startLine - the source line number (1-based)
      startColumn - the source column number (1-based)
      Returns:
      a Token representing a string literal

    • newIdentifier

      public static Token newIdentifier(String text, int startLine, int startColumn)
      Factory method to create an identifier token.
      Parameters:
      text - the identifier name
      startLine - the source line number (1-based)
      startColumn - the source column number (1-based)
      Returns:
      a Token representing an identifier

    • newInteger

      public static Token newInteger(String text, int startLine, int startColumn)
      Factory method to create an integer literal token.
      Parameters:
      text - the integer literal text (may include prefix and suffix)
      startLine - the source line number (1-based)
      startColumn - the source column number (1-based)
      Returns:
      a Token representing an integer literal

    • newDecimal

      public static Token newDecimal(String text, int startLine, int startColumn)
      Factory method to create a decimal number literal token.
      Parameters:
      text - the decimal literal text
      startLine - the source line number (1-based)
      startColumn - the source column number (1-based)
      Returns:
      a Token representing a decimal number literal

    • newSymbol

      public static Token newSymbol(int type, int startLine, int startColumn)
      Factory method to create a symbol token from a type constant. The symbol text is looked up from the Types registry.
      Parameters:
      type - the symbol type from Types
      startLine - the source line number (1-based)
      startColumn - the source column number (1-based)
      Returns:
      a Token representing a symbol

    • newSymbol

      public static Token newSymbol(String text, int startLine, int startColumn)
      Factory method to create a symbol token from text. The symbol type is looked up from the Types registry.
      Parameters:
      text - the symbol text (e.g., "+", "-", "{")
      startLine - the source line number (1-based)
      startColumn - the source column number (1-based)
      Returns:
      a Token representing a symbol

    • newPlaceholder

      public static Token newPlaceholder(int meaning)
      Factory method to create a placeholder token with a specific meaning but unknown text. Used internally during parsing to hold semantic information.
      Parameters:
      meaning - the token's meaning type from Types
      Returns:
      a placeholder Token with empty text and unknown type