SnippetsTest for Antlr4

SnippetsTest is a TestNG-based unit testing framework that simplifies the debugging and consistency testing of Antlr4 grammars. The test inputs are minimal viable source texts, aka snippets, for the grammar system under test.

Features

• No coding requried to add a test
  • just add a new snippet to a snippet test source directory
• No 'expected' result management required
  • result subdirectories are automatically added to mirror the source directory structure
  • tentative 'expected' result files are automatically generated when appropriate
  • existing 'expected' result files are never overwritten
• Discrete characteriaztion of each primary error source
  • separate test assertions against lexer, parser, and tree-walker behaviors
• Snippets are are run as individual tests within a standard TestNG test package
  • compatible with standard IDE/TestNG launchers, CI builders, and reporters

Operation

Each snippet test generates three 'found' results:

• a full lexer token stream dump;
• a Lisp-style representation of the generated parse-tree; and
• a system result string containing whatever is generated by the parser and applicable tree-walkers.

These 'found' results are separately asserted equal to their prior stored 'expected' results.

Options allow for the 'found' results to be echoed to the system console (stdio) for ease of inspection.

If, and only if, files containing the prior stored 'expected' results do not exist, the 'found' results are stored as tentative 'expected' result files. If an 'expected' result is not -- or is no longer -- proper, remove the 'expected' result file, fix the grammar system, and re-run the test.

Use

1. Copy the SnippetsTest package files into a testing package within your project.
2. Customize the marked blocks within SnippetsTestNG.java. Three key methods require customization (where possible, use entry points within your existing project to ensure these test methods reflect the actual operation of your project code):
   • createLexerStream -- returns the CommonTokenStream for the lexer being tested
   • createParseTree -- returns the ParseTree for the parser being tested
     • note the required annotation of the parse-tree with its recognizer
   • createResults -- returns a String result for the full system being tested
3. Create a snippet test source directory structure
   • add snippets that are as minimal as possible -- to keep the results from becoming unwieldy.
   • a mirroring result directory structure will be automatically generated/updated on each run
4. Run, launch, or otherwise execute SnippetsTestNG.java as a standard TestNG test, however appropriate for your IDE, build system, or platform (see the TestNG doc for details).
   • add new snippets frequently
   • run the tests more frequently

Directory Structures

Type
Source:	<baseDir>/test.snippets/<category>/<snippets>
Result:	<baseDir>/test.expected/<category>/<expected>

where
<baseDir>	a path to a filesystem directory -- typically the project root
<category>	a simple subdirectory name -- sibling and child categories encouraged
<snippets>	any number of separate minimal viable source texts
<expected>	the sets of 'expected' results texts

License

Standard 3-clause BSD

Dependencies

• TestNg v6.9.8+
• Antlr4 v4.5+