Status of standard tests in GAP packages

For the full list of all publicly available tests from the GAP test suite see https://github.com/gap-system/gap-distribution/blob/master/README.md

Please read the following information to ensure that you interpret the build logs properly.

By the standard test of a GAP package we denote the test which is specified in the TestFile component of the PackageInfo.g file. This usually provides a basic test of the package (for example, to check that it works as expected and/or that the manual examples are correct) and should take not more than several minutes.

The standard test file can either consist of calls of TestDirectory or Test (in this case, it is common to call it testall.g), or be itself a test file, having an extension .tst and supposed to be read via Test. It is assumed that the latter case occurs if and only if the file contains the substring

"gap> START_TEST("

(with exactly one space after the GAP prompt).

For packages redistributed with GAP, their standard tests are run by the GAP Group regularly, as a part of the standard GAP test suite. For the efficient testing it is recommended that the test specified in the PackageInfo.g file does not produce excessive output.

Packages may contain other test files as well, but those will be not a part of the automated testing. Such tests, which may produce excessive output and/or require substantial runtime, may be placed in some subdirectory (e.g. extra) in the tst directory of the package, with further instructions on how to run them.

Because of different approaches to testing, used by different packages, it is not always easy to identify whether an automated test passed or failed. Presently, automated detection works fine if a package uses a single .tst file or uses TestDirectory to run a collection of tests. Otherwise, one should follow the convention to output a single string of a fixed format to indicate that the test had been finished, along the following lines:

if testresult then
  Print("#I  No errors detected while testing"\n");
else
  Print("#I  Errors detected while testing\n");
fi;

If these rules are not followed, for packages with the test output having neither ########> Diff markers nor the "#I No errors detected while testing" line, it's not clear whether the test passed or crashed (for example, it could happen that its compilation failed or it crashed in the middle of the test).