Package jadex.commons

Class ExpressionTokenizer

  • public class ExpressionTokenizer
extends java.lang.Object
    An expression tokenizer is able to parse a string with respect to different nesting-levels (eg. brackets, quotes).

    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected java.lang.String allseps
      All separators.
      protected java.lang.String close
      The nesting level delimiters.
      static java.lang.String ESCAPE_CHARACTERS
      The escape characters allow to include separators in the tokens.
      protected java.lang.String open
      The nesting level delimiters.
      protected int pos
      The current tokenization position.
      protected boolean retsep
      Return the separators.
      protected java.lang.String separators
      The top level separators.
      protected java.lang.String source
      The source string.

    • Constructor Summary

      Constructors 
      Constructor Description
      ExpressionTokenizer​(java.lang.String string, java.lang.String separators, java.lang.String[] metas)
      Create a new tokenizer.
      ExpressionTokenizer​(java.lang.String string, java.lang.String separators, java.lang.String[] metas, boolean retsep)
      Create a new tokenizer.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      int countTokens()
      Count the number of tokens.
      boolean hasMoreTokens()
      Test if there are more tokens available.
      static void main​(java.lang.String[] args)
      Main for testing.
      protected boolean nestingSeparatorAt​(int pos)
      Check if there is a nesting level separator character at the specified position.
      java.lang.String nextToken()
      Get the next token from the string.
      java.lang.String remainingTokens()
      Get the remaining tokens as single string.
      protected boolean separatorAt​(int pos)
      Check if there is a separator character at the specified position.
      void setPosition​(int pos)
      Set the parse position manually.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • ESCAPE_CHARACTERS

        public static final java.lang.String ESCAPE_CHARACTERS
        The escape characters allow to include separators in the tokens. E.g. with escape character '\' it is possible to parse "(\))".
        See Also:
        Constant Field Values

      • source

        protected java.lang.String source
        The source string.

      • separators

        protected java.lang.String separators
        The top level separators.

      • open

        protected java.lang.String open
        The nesting level delimiters.

      • close

        protected java.lang.String close
        The nesting level delimiters.

      • retsep

        protected boolean retsep
        Return the separators.

      • allseps

        protected java.lang.String allseps
        All separators.

      • pos

        protected int pos
        The current tokenization position.

    • Constructor Detail

      • ExpressionTokenizer

        public ExpressionTokenizer​(java.lang.String string,
                           java.lang.String separators,
                           java.lang.String[] metas)
        Create a new tokenizer.
        Parameters:
        string - The string.
        separators - The separator chars.
        metas - The nesting level delimiters.

      • ExpressionTokenizer

        public ExpressionTokenizer​(java.lang.String string,
                           java.lang.String separators,
                           java.lang.String[] metas,
                           boolean retsep)
        Create a new tokenizer.
        Parameters:
        string - The string.
        separators - The separator chars.
        metas - The nesting level delimiters.
        retsep - Return the separator chars.

    • Method Detail

      • nextToken

        public java.lang.String nextToken()
        Get the next token from the string.
        Returns:
        The next token.

      • remainingTokens

        public java.lang.String remainingTokens()
        Get the remaining tokens as single string. When separators are to be returned, the remaining string will start with the next separator. Otherwise, the separators at the current position are skipped.
        Returns:
        The remaining tokens token.

      • hasMoreTokens

        public boolean hasMoreTokens()
        Test if there are more tokens available.

      • countTokens

        public int countTokens()
        Count the number of tokens.
        Returns:
        The number of tokens.

      • setPosition

        public void setPosition​(int pos)
        Set the parse position manually. Be careful when using this method!.
        Parameters:
        pos - The new position.

      • separatorAt

        protected boolean separatorAt​(int pos)
        Check if there is a separator character at the specified position.
        Parameters:
        pos - The position to check.
        Returns:
        true, if the position contains a separator.

      • nestingSeparatorAt

        protected boolean nestingSeparatorAt​(int pos)
        Check if there is a nesting level separator character at the specified position.
        Parameters:
        pos - The position to check.
        Returns:
        true, if the position contains a separator.

      • main

        public static void main​(java.lang.String[] args)
        Main for testing.