Test Suite

The MUNGE test suite uses Sharness for higher-level tests of the executables, and libtap for lower-level tests of the C internals. Sharness tests reside in the t/ directory, whereas libtap tests are located in the directories under src/.

Both Sharness and libtap are TAP Producers, outputting test results according to the Test Anything Protocol. This TAP output can be fed into a TAP Consumer for further processing.

Running the tests

• From the check target of the Makefile. Specify options by appending shell variables to the make command.
  make check root=/tmp/munge-$$ verbose=t

• From the prove command. Specify --help, or read the prove(1) manpage or Perl docs.
  prove -v 0010-basic.t

• From the test itself. Specify options by prepending shell variables to the command, or appending command-line options.
  ./0010-basic.t --root=/tmp/munge-$$ -v

Variables of Note

• chain_lint is a sharness variable for checking whether each test properly "&&-chains" all commands in order to ensure a failure of any command in the chain will cause the test to fail. If chain_lint=t, sharness will perform this check in addition to running the tests. This is primarily intended for development.

• debug is a sharness variable primarily intended for development. If debug=t, execution of test_debug() commands will be performed, and removal of "trash" directories (used to store all temporary data during the test) will be prevented in order to allow their contents to be inspected afterwards.

• root is a sharness variable for specifying a directory to be prepended to the "trash" directories (used to store all temporary data during the test). If this variable is not specified, these directories are created in the current working directory. They are removed after the test finishes unless a test fails or debug=t is specified.

• TEST_LONG is a sharness variable for running tests marked as being "expensive". If this variable is not set, sharness tests requiring sudo (expensive in privilege) or valgrind (expensive in memory and time) will be skipped. This is primarily intended for development.

• verbose is a sharness variable for making test output more verbose. By default, standard output and standard error streams are discarded, and only a result of "ok" or "not ok" is reported. But if verbose=t, the test commands being run and their resulting output will be logged to the corresponding .log files as well as test-suite.log.

• VERBOSE is an automake test harness variable. If set, the output from failed tests collected in test-suite.log will be displayed after all tests have completed.

Tests utilizing sudo

• Sharness tests checking the behavior of executables running as root use sudo to elevate privileges. They require sudo-1.7.0 or later. These tests will be skipped unless the TEST_LONG variable is set and sudo is configured to run commands without prompting for a password (see the NOPASSWD: tag in sudoers(5)).

Tests utilizing valgrind

• Sharness tests using valgrind to check for memory errors require valgrind-3.9.0 (2013-10-31) or later. These tests will be skipped unless the TEST_LONG variable is set and valgrind is installed.

Troubleshooting

• If tests fail, try running the test suite with verbose=t and examining t/test-suite.log afterwards for more information.

• If your current working directory (or one of its parent directories) has overly-permissive permissions, you may see a multitude of errors noting various files and directories are insecure. Either correct the directory permissions, or try moving the "trash" directories to another filesystem such as /tmp (which should have the sticky-bit set) by running the test suite with root=/tmp/munge-$$.

• If your current working directory is inside an NFS mount, you may encounter race conditions with NFS locks; for example, rm: cannot remove '/path/to/some/.nfs000000000123456789abcdef': Device or resource busy. Try moving the "trash" directories to a non-NFS local filesystem by running the test suite with root=/tmp/munge-$$.