STLexer

java.lang.Object
- org.stringtemplate.v4.compiler.STLexer

All Implemented Interfaces:

org.antlr.runtime.TokenSource
```
public class STLexer
extends Object
implements org.antlr.runtime.TokenSource
```
This class represents the tokenizer for templates. It operates in two modes: inside and outside of expressions. It implements the TokenSource interface so it can be used with ANTLR parsers. Outside of expressions, we can return these token types: TEXT, INDENT, LDELIM (start of expression), RCURLY (end of subtemplate), and NEWLINE. Inside of an expression, this lexer returns all of the tokens needed by STParser. From the parser's point of view, it can treat a template as a simple stream of elements.
This class defines the token types and communicates these values to STParser.g via STLexer.tokens file (which must remain consistent).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
`static class`	`STLexer.STToken` We build `STToken` tokens instead of relying on `CommonToken` so we can override `STLexer.STToken.toString()`.

Field Summary

Fields

Modifier and Type	Field and Description
`static int`	`AND`
`static int`	`AT`
`static int`	`BANG`
`static int`	`COLON`
`static int`	`COMMA`
`static int`	`COMMENT`
`static int`	`DOT`
`static int`	`ELLIPSIS`
`static int`	`ELSE`
`static int`	`ELSEIF`
`static int`	`ENDIF`
`static char`	`EOF`
`static int`	`EOF_TYPE`
`static int`	`EQUALS`
`static int`	`FALSE`
`static int`	`ID`
`static int`	`IF`
`static int`	`INDENT`
`static int`	`LBRACK`
`static int`	`LCURLY`
`static int`	`LDELIM`
`static int`	`LPAREN`
`static int`	`NEWLINE`
`static int`	`OR`
`static int`	`PIPE`
`static int`	`RBRACK`
`static int`	`RCURLY`
`static int`	`RDELIM`
`static int`	`REGION_END`
`static int`	`RPAREN`
`static int`	`SEMI`
`static org.antlr.runtime.Token`	`SKIP`
`static int`	`STRING`
`int`	`subtemplateDepth` To be able to properly track the inside/outside mode, we need to track how deeply nested we are in some templates.
`static int`	`SUPER`
`static int`	`TEXT`
`static int`	`TRUE`

Constructor Summary

Constructors

Constructor and Description
`STLexer(org.antlr.runtime.CharStream input)`
`STLexer(ErrorManager errMgr, org.antlr.runtime.CharStream input, org.antlr.runtime.Token templateToken)`
`STLexer(ErrorManager errMgr, org.antlr.runtime.CharStream input, org.antlr.runtime.Token templateToken, char delimiterStartChar, char delimiterStopChar)`

Method Summary

All Methods

Static Methods

Instance Methods

Concrete Methods
Modifier and Type	Method and Description
`org.antlr.runtime.Token`	`_nextToken()`
`protected void`	`consume()`
`void`	`emit(org.antlr.runtime.Token token)`
`String`	`getSourceName()`
`protected org.antlr.runtime.Token`	`inside()`
`static boolean`	`isIDLetter(char c)`
`static boolean`	`isIDStartLetter(char c)`
`static boolean`	`isUnicodeLetter(char c)`
`static boolean`	`isWS(char c)`
`void`	`match(char x)` Consume if `x` is next character on the input stream.
`org.antlr.runtime.Token`	`newToken(int ttype)`
`org.antlr.runtime.Token`	`newToken(int ttype, String text)`
`org.antlr.runtime.Token`	`newToken(int ttype, String text, int pos)`
`org.antlr.runtime.Token`	`newTokenFromPreviousChar(int ttype)`
`org.antlr.runtime.Token`	`nextToken()`
`protected org.antlr.runtime.Token`	`outside()`
`static String`	`str(int c)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail
- EOF
```
public static final char EOF
```
  See Also:
  
  Constant Field Values
- EOF_TYPE
```
public static final int EOF_TYPE
```
  See Also:
  
  Constant Field Values
- SKIP
```
public static final org.antlr.runtime.Token SKIP
```
- RBRACK
```
public static final int RBRACK
```
  See Also:
  
  Constant Field Values
- LBRACK
```
public static final int LBRACK
```
  See Also:
  
  Constant Field Values
- ELSE
```
public static final int ELSE
```
  See Also:
  
  Constant Field Values
- ELLIPSIS
```
public static final int ELLIPSIS
```
  See Also:
  
  Constant Field Values
- LCURLY
```
public static final int LCURLY
```
  See Also:
  
  Constant Field Values
- BANG
```
public static final int BANG
```
  See Also:
  
  Constant Field Values
- EQUALS
```
public static final int EQUALS
```
  See Also:
  
  Constant Field Values
- TEXT
```
public static final int TEXT
```
  See Also:
  
  Constant Field Values
- ID
```
public static final int ID
```
  See Also:
  
  Constant Field Values
- SEMI
```
public static final int SEMI
```
  See Also:
  
  Constant Field Values
- LPAREN
```
public static final int LPAREN
```
  See Also:
  
  Constant Field Values
- IF
```
public static final int IF
```
  See Also:
  
  Constant Field Values
- ELSEIF
```
public static final int ELSEIF
```
  See Also:
  
  Constant Field Values
- COLON
```
public static final int COLON
```
  See Also:
  
  Constant Field Values
- RPAREN
```
public static final int RPAREN
```
  See Also:
  
  Constant Field Values
- COMMA
```
public static final int COMMA
```
  See Also:
  
  Constant Field Values
- RCURLY
```
public static final int RCURLY
```
  See Also:
  
  Constant Field Values
- ENDIF
```
public static final int ENDIF
```
  See Also:
  
  Constant Field Values
- RDELIM
```
public static final int RDELIM
```
  See Also:
  
  Constant Field Values
- SUPER
```
public static final int SUPER
```
  See Also:
  
  Constant Field Values
- DOT
```
public static final int DOT
```
  See Also:
  
  Constant Field Values
- LDELIM
```
public static final int LDELIM
```
  See Also:
  
  Constant Field Values
- STRING
```
public static final int STRING
```
  See Also:
  
  Constant Field Values
- PIPE
```
public static final int PIPE
```
  See Also:
  
  Constant Field Values
- OR
```
public static final int OR
```
  See Also:
  
  Constant Field Values
- AND
```
public static final int AND
```
  See Also:
  
  Constant Field Values
- INDENT
```
public static final int INDENT
```
  See Also:
  
  Constant Field Values
- NEWLINE
```
public static final int NEWLINE
```
  See Also:
  
  Constant Field Values
- AT
```
public static final int AT
```
  See Also:
  
  Constant Field Values
- REGION_END
```
public static final int REGION_END
```
  See Also:
  
  Constant Field Values
- TRUE
```
public static final int TRUE
```
  See Also:
  
  Constant Field Values
- FALSE
```
public static final int FALSE
```
  See Also:
  
  Constant Field Values
- COMMENT
```
public static final int COMMENT
```
  See Also:
  
  Constant Field Values
- subtemplateDepth
```
public int subtemplateDepth
```
  To be able to properly track the inside/outside mode, we need to track how deeply nested we are in some templates. Otherwise, we know whether a '}' and the outermost subtemplate to send this back to outside mode.

Constructor Detail

STLexer

public STLexer(org.antlr.runtime.CharStream input)

STLexer

public STLexer(ErrorManager errMgr,
               org.antlr.runtime.CharStream input,
               org.antlr.runtime.Token templateToken)

STLexer

public STLexer(ErrorManager errMgr,
               org.antlr.runtime.CharStream input,
               org.antlr.runtime.Token templateToken,
               char delimiterStartChar,
               char delimiterStopChar)

Method Detail

nextToken

public org.antlr.runtime.Token nextToken()

match
```
public void match(char x)
```
Consume if x is next character on the input stream.

consume
```
protected void consume()
```

emit

public void emit(org.antlr.runtime.Token token)

_nextToken

public org.antlr.runtime.Token _nextToken()

outside

protected org.antlr.runtime.Token outside()

inside

protected org.antlr.runtime.Token inside()

isIDStartLetter

public static boolean isIDStartLetter(char c)

isIDLetter

public static boolean isIDLetter(char c)

isWS
```
public static boolean isWS(char c)
```

isUnicodeLetter

public static boolean isUnicodeLetter(char c)

newToken

public org.antlr.runtime.Token newToken(int ttype)

newTokenFromPreviousChar

public org.antlr.runtime.Token newTokenFromPreviousChar(int ttype)

newToken

public org.antlr.runtime.Token newToken(int ttype,
                                        String text,
                                        int pos)

newToken

public org.antlr.runtime.Token newToken(int ttype,
                                        String text)

getSourceName
```
public String getSourceName()
```

str
```
public static String str(int c)
```

Class STLexer

Nested Class Summary

Nested Classes

Field Summary

Fields

Constructor Summary

Constructors

Method Summary

Methods inherited from class java.lang.Object

Field Detail

EOF

EOF_TYPE

SKIP

RBRACK

LBRACK

ELSE

ELLIPSIS

LCURLY

BANG

EQUALS

TEXT

ID

SEMI

LPAREN

IF

ELSEIF

COLON

RPAREN

COMMA

RCURLY

ENDIF

RDELIM

SUPER

DOT

LDELIM

STRING

PIPE

OR

AND

INDENT

NEWLINE

AT

REGION_END

TRUE

FALSE

COMMENT

subtemplateDepth

Constructor Detail

STLexer

STLexer

STLexer

Method Detail

nextToken

match

consume

emit

_nextToken

outside

inside

isIDStartLetter

isIDLetter

isWS

isUnicodeLetter

newToken

newTokenFromPreviousChar

newToken

newToken

getSourceName

str