Package | Description |
---|---|
parsii.tokenizer |
Modifier and Type | Method and Description |
---|---|
Token |
Token.addToContent(char ch)
Adds the given character to the content (and the source) but not to the trigger
|
Token |
Token.addToContent(Char ch)
Adds the given Char to the content (and the source) but not to the trigger
|
Token |
Token.addToSource(Char ch)
Adds the given Char to the source of this token, but neither to the trigger nor to the content.
|
Token |
Token.addToTrigger(Char ch)
Adds the given Char to the trigger (and the source) but not to the content
|
static Token |
Token.create(Token.TokenType type,
Position pos)
Creates a new token with the given type, using the given position as location info.
|
static Token |
Token.createAndFill(Token.TokenType type,
Char ch)
Creates a new token with the given type, using the Char a initial trigger and content.
|
protected Token |
Tokenizer.endOfInput() |
protected Token |
Tokenizer.fetch() |
protected Token |
Tokenizer.fetchId()
Reads and returns an identifier
|
protected Token |
Tokenizer.fetchNumber()
Reads and returns a number.
|
protected Token |
Tokenizer.fetchSpecialId()
Reads and returns a special id.
|
protected Token |
Tokenizer.fetchString()
Reads and returns a string constant.
|
protected Token |
Tokenizer.fetchSymbol()
Reads and returns a symbol.
|
protected Token |
Tokenizer.handleKeywords(Token idToken)
Checks if the given identifier is a keyword and returns an appropriate Token
|
Token |
Token.silentAddToContent(char ch)
Adds a character to the content without adding it to the source.
|
Modifier and Type | Method and Description |
---|---|
protected Token |
Tokenizer.handleKeywords(Token idToken)
Checks if the given identifier is a keyword and returns an appropriate Token
|
protected boolean |
Tokenizer.handleStringEscape(char separator,
char escapeChar,
Token stringToken)
Evaluates an string escape like \n
|
Copyright © 2020. All rights reserved.