|
|||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||||
java.lang.Object | +--net.percederberg.grammatica.parser.Tokenizer
A character stream tokenizer. This class groups the characters read from the stream together into tokens ("words"). The grouping is controlled by token patterns that contain either a fixed string to search for, or a regular expression. If the stream of characters don't match any of the token patterns, a parse exception is thrown.
| Constructor Summary | |
Tokenizer(java.io.Reader input)
Creates a new tokenizer for the specified input stream. |
|
| Method Summary | |
void |
addPattern(TokenPattern pattern)
Adds a new token pattern to the tokenizer. |
int |
getCurrentColumn()
Returns the current column number. |
int |
getCurrentLine()
Returns the current line number. |
java.lang.String |
getPatternDescription(int id)
Returns a description of the token pattern with the specified id. |
Token |
next()
Finds the next token on the stream. |
| Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
public Tokenizer(java.io.Reader input)
input - the input stream to read| Method Detail |
public java.lang.String getPatternDescription(int id)
id - the token pattern id
public int getCurrentLine()
public int getCurrentColumn()
public void addPattern(TokenPattern pattern)
throws ParserCreationException
pattern - the pattern to add
ParserCreationException - if the pattern couldn't be
added to the tokenizer
public Token next()
throws ParseException
ParseException - if the input stream couldn't be read or
parsed correctly
|
|||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||||