TestAllSandia

The test_all_sandia script helps automate the testing of the combinatorial explosion of compilers, backends, and TPLs supported by Kokkos and on a variety of testing platforms that are available to KokkosKernels developers at Sandia Nat. Labs.

Prior to submission of a new pull request to the kokkos-kernels repository, the code changes must be tested and pass a minimum set of tests run with the --spot-check option of the test_all_sandia script on one of our testing platforms; results should be posted as a message to the imminent pull request as documentation. The script is also used for autmoated nightly testing, nightly testing covers any gaps of compiler + backend that are skipped during a spot-check.

Basic Usage

Create a testing directory

mkdir testing
cd testing

Call the test_all_sandia script with desired options - at minimum the paths to kokkos and kokkos-kernels should be provided if the testing directory is not a sub-directory of kokkos-kernels and if kokkos-kernels and kokkos are not located in the same base directory.

<PATH_TO_KOKKOSKERNELS>/scripts/test_all_sandia --kokkoskernels-path=<PATH_TO_KOKKOSKERNELS> --kokkos-path=<PATH_TO_KOKKOS>

To see available testing options, run the script with the --help flag

<PATH_TO_KOKKOSKERNELS>/scripts/test_all_sandia --help

Note that a default architecture is assumed on each supported platform; however, on some testbeds there are queues available with hardware that differs from default architecture options. In this case, the user must supply the correct architecture information explicitly via the --arch flag. See this link for a list of architecture variable names recognized by Kokkos.

Examples

Full testing

The command below will run the full set of tests on the given architecture. If kokkos and kokkos-kernels are in the same base directory, and the testing directory is nested within kokkos-kernels, then the path argumants do not need to be passed.

<PATH_TO_KOKKOSKERNELS>/scripts/test_all_sandia --kokkoskernels-path=<PATH_TO_KOKKOSKERNELS> --kokkos-path=<PATH_TO_KOKKOS>

kokkos-dev testbed

Note: This testbed does not have multi-queue support. It has access to a limited subset of sems modules.

Test only Cuda and OpenMP backend builds

<PATH_TO_KOKKOSKERNELS>/scripts/test_all_sandia --kokkoskernels-path=<PATH_TO_KOKKOSKERNELS> --kokkos-path=<PATH_TO_KOKKOS> --build-list=Cuda_OpenMP

Spot-check with only cuda compilers

<PATH_TO_KOKKOSKERNELS>/scripts/test_all_sandia --kokkoskernels-path=<PATH_TO_KOKKOSKERNELS> --kokkos-path=<PATH_TO_KOKKOS> cuda --spot-check

Test Serial backend with gcc/7.2.0 compiler

<PATH_TO_KOKKOSKERNELS>/scripts/test_all_sandia --kokkoskernels-path=<PATH_TO_KOKKOSKERNELS> --kokkos-path=<PATH_TO_KOKKOS> sems-gcc/7.2.0 --build-list=Serial

White testbed

Note: This testbed has separate queues supporting Power8 with NVIDIA Kepler and Pascal GPUs; the test_all_sandia script defaults to the Power8+Kepler37 architecture.
LSF is used for job scheduling

Power8 + NVIDIA Pascal hardware queue, xlC compiler, Serial backend

First, allocate an interactive node on the appropriate queue (alternatively write a batch script to submit to the queue)

bsub -Is -n1 -q rhel7G bash
<PATH_TO_KOKKOSKERNELS>/scripts/test_all_sandia --kokkoskernels-path=<PATH_TO_KOKKOSKERNELS> --kokkos-path=<PATH_TO_KOKKOS> xl --arch=Power8,Pascal60 --build-list=Serial