... a tool to generate lexical analyzers. A lexical analyzer is a program
that transforms a stream of characters into a stream of 'atomic chunks of meaning', so called tokens.
It is licenced under LGPL Version 2.1.
- generate directly coded lexical analyzers, rather than table based engines.
- respond to queries on Unicode properties and regular expressions on the command line.
- generate state transition graphs of the generated engines.
- Many examples.
- Modes that can be inherited and mode transitions that can be controlled.
- Line and column number counting as part of the generated engine.
- Include stacks for include file management.
- Support for indentation based analysis (INDENT, DEDENT, NODENT tokens).
- Path and Template compression for minimal code size.
- Two token passing strategies: 'queue' and 'single token'.
- Support for customized token types.
- Event handlers for fine grained control over analyzer actions.
- Buffer management providing tell&seek based on
character indices even with dynamic size character codings (e.g. UTF-8, UTF-16).