jadex.commons

Class ExpressionTokenizer

java.lang.Object

jadex.commons.ExpressionTokenizer

public class ExpressionTokenizer
extends java.lang.Object

An expression tokenizer is able to parse a string with respect to different nesting-levels (eg. brackets, quotes).

Field Summary

Fields

Modifier and Type	Field and Description
protected java.lang.String	allseps All separators.
protected java.lang.String	close The nesting level delimiters.
static java.lang.String	ESCAPE_CHARACTERS The escape characters allow to include separators in the tokens.
protected java.lang.String	open The nesting level delimiters.
protected int	pos The current tokenization position.
protected boolean	retsep Return the separators.
protected java.lang.String	separators The top level separators.
protected java.lang.String	source The source string.

Constructor Summary

Constructors

Constructor and Description
ExpressionTokenizer(java.lang.String string, java.lang.String separators, java.lang.String[] metas) Create a new tokenizer.
ExpressionTokenizer(java.lang.String string, java.lang.String separators, java.lang.String[] metas, boolean retsep) Create a new tokenizer.

Method Summary

Modifier and Type	Method and Description
int	countTokens() Count the number of tokens.
boolean	hasMoreTokens() Test if there are more tokens available.
static void	main(java.lang.String[] args) Main for testing.
protected boolean	nestingSeparatorAt(int pos) Check if there is a nesting level separator character at the specified position.
java.lang.String	nextToken() Get the next token from the string.
java.lang.String	remainingTokens() Get the remaining tokens as single string.
protected boolean	separatorAt(int pos) Check if there is a separator character at the specified position.
void	setPosition(int pos) Set the parse position manually.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ESCAPE_CHARACTERS

public static final java.lang.String ESCAPE_CHARACTERS

The escape characters allow to include separators in the tokens. E.g. with escape character '\' it is possible to parse "(\))".

See Also:

Constant Field Values

source

protected java.lang.String source

The source string.

separators

protected java.lang.String separators

The top level separators.

open

protected java.lang.String open

The nesting level delimiters.

close

protected java.lang.String close

The nesting level delimiters.

retsep

protected boolean retsep

Return the separators.

allseps

protected java.lang.String allseps

All separators.

pos

protected int pos

The current tokenization position.

Constructor Detail

ExpressionTokenizer

public ExpressionTokenizer(java.lang.String string,
                           java.lang.String separators,
                           java.lang.String[] metas)

Create a new tokenizer.

Parameters:

string - The string.

separators - The separator chars.

metas - The nesting level delimiters.

ExpressionTokenizer

public ExpressionTokenizer(java.lang.String string,
                           java.lang.String separators,
                           java.lang.String[] metas,
                           boolean retsep)

Create a new tokenizer.

Parameters:

string - The string.

separators - The separator chars.

metas - The nesting level delimiters.

retsep - Return the separator chars.

Method Detail

nextToken

public java.lang.String nextToken()

Get the next token from the string.

Returns:

The next token.

remainingTokens

public java.lang.String remainingTokens()

Get the remaining tokens as single string. When separators are to be returned, the remaining string will start with the next separator. Otherwise, the separators at the current position are skipped.

Returns:

The remaining tokens token.

hasMoreTokens

public boolean hasMoreTokens()

Test if there are more tokens available.

countTokens

public int countTokens()

Count the number of tokens.

Returns:

The number of tokens.

setPosition

public void setPosition(int pos)

Set the parse position manually. Be careful when using this method!.

Parameters:

pos - The new position.

separatorAt

protected boolean separatorAt(int pos)

Check if there is a separator character at the specified position.

Parameters:

pos - The position to check.

Returns:

true, if the position contains a separator.

nestingSeparatorAt

protected boolean nestingSeparatorAt(int pos)

Check if there is a nesting level separator character at the specified position.

Parameters:

pos - The position to check.

Returns:

true, if the position contains a separator.

main

public static void main(java.lang.String[] args)

Main for testing.