Lexer splits the input onto tokens.
The input chunks scanned by lexer one at a time. Each token found is emitted by calling provided emitter. On completion, the input has to by flushed in order to process the remaining input.
Flushes the input emitting all pending tokens.
Scans the input chunk for tokens.
chunk
Chunk of input to scan.
Generated using TypeDoc
Lexer splits the input onto tokens.
The input chunks scanned by lexer one at a time. Each token found is emitted by calling provided emitter. On completion, the input has to by flushed in order to process the remaining input.