Package jadex.common
Class ExpressionTokenizer
java.lang.Object
jadex.common.ExpressionTokenizer
An expression tokenizer is able to parse a string with respect
to different nesting-levels (eg. brackets, quotes).
-
Field Summary
FieldsModifier and TypeFieldDescriptionprotected String
All separators.protected String
The nesting level delimiters.static final String
The escape characters allow to include separators in the tokens.protected String
The nesting level delimiters.protected int
The current tokenization position.protected boolean
Return the separators.protected String
The top level separators.protected String
The source string. -
Constructor Summary
ConstructorsConstructorDescriptionExpressionTokenizer
(String string, String separators, String[] metas) Create a new tokenizer.ExpressionTokenizer
(String string, String separators, String[] metas, boolean retsep) Create a new tokenizer. -
Method Summary
Modifier and TypeMethodDescriptionint
Count the number of tokens.boolean
Test if there are more tokens available.static void
Main for testing.protected boolean
nestingSeparatorAt
(int pos) Check if there is a nesting level separator character at the specified position.Get the next token from the string.Get the remaining tokens as single string.protected boolean
separatorAt
(int pos) Check if there is a separator character at the specified position.void
setPosition
(int pos) Set the parse position manually.
-
Field Details
-
ESCAPE_CHARACTERS
The escape characters allow to include separators in the tokens. E.g. with escape character '\' it is possible to parse "(\))".- See Also:
-
source
The source string. -
separators
The top level separators. -
open
The nesting level delimiters. -
close
The nesting level delimiters. -
retsep
protected boolean retsepReturn the separators. -
allseps
All separators. -
pos
protected int posThe current tokenization position.
-
-
Constructor Details
-
ExpressionTokenizer
Create a new tokenizer.- Parameters:
string
- The string.separators
- The separator chars.metas
- The nesting level delimiters.
-
ExpressionTokenizer
Create a new tokenizer.- Parameters:
string
- The string.separators
- The separator chars.metas
- The nesting level delimiters.retsep
- Return the separator chars.
-
-
Method Details
-
nextToken
Get the next token from the string.- Returns:
- The next token.
-
remainingTokens
Get the remaining tokens as single string. When separators are to be returned, the remaining string will start with the next separator. Otherwise, the separators at the current position are skipped.- Returns:
- The remaining tokens token.
-
hasMoreTokens
public boolean hasMoreTokens()Test if there are more tokens available. -
countTokens
public int countTokens()Count the number of tokens.- Returns:
- The number of tokens.
-
setPosition
public void setPosition(int pos) Set the parse position manually. Be careful when using this method!.- Parameters:
pos
- The new position.
-
separatorAt
protected boolean separatorAt(int pos) Check if there is a separator character at the specified position.- Parameters:
pos
- The position to check.- Returns:
- true, if the position contains a separator.
-
nestingSeparatorAt
protected boolean nestingSeparatorAt(int pos) Check if there is a nesting level separator character at the specified position.- Parameters:
pos
- The position to check.- Returns:
- true, if the position contains a separator.
-
main
Main for testing.
-