Output Data

Current output

In random order, with indication of the debug level (Success [S], Warning [W], Error [E], Debug [D], Info [I], Fatal [F])

Some outputs can be truncated if they exceed a (configurable) number of lines.

• General information (for each test)

  • test name [I]
  • command line [I]
  • execution errors [F]
    • errors running the executable (e.g. the file does not exist)
    • process terminating with an error code
    • segmentation fault
    • timeout
  • input to the process [I]
  • content of the temporary file used for input [I]
  • output of the process [D]

• Summary of the results

  • total number of warnings, errors and successes [I]
  • number of warnings, errors and successes reported each section [I]
  • number of warnings, errors and successes reported by valgrind [I]

• Result of the comparison of the unordered contents of sections

  • missing/extra lines [E]
  • mismatching lines [E]

• Result of the comparison of the ordered contents of sections

  • missing/extra lines [E]
  • mismatching lines [E]

• Missing section [W]

• Unexpected section [D]

• Empty section (when not expected) [E]

• Side-by-side comparison of expected and actual output [D]

• Successful comparison of a section [S]

• Info from Valgrind

  • allocated memory not freed (two messages, bytes and alloc/free counts) [W]
  • correctly freed memory (two messages, bytes and alloc/free counts) [D]
  • valgrind errors [E]
  • valgrind is fine [D]