Post-CMake merge and 3.0 items

[ndellingwood]

Reminder list of items following merge of #471 (previous) now #491

• Re-enable -Werror for the clang builds on kokkos-dev-2 when we resume testing off of Kokkos' develop branch
• Add more testing configurations to kokkos-dev-2 (cover items in spot-check)
• Convert testing to use test_all_sandia replacement script patterned from kokkos' cm_test_all_sandia script

[ndellingwood]

Some open issues to resolve for 3.0 testing and release:

• spgemm shmem alignment issues: Issue spgemm cuda build with complex_double: error( cudaErrorMisalignedAddress) with Kokkos develop branch #500 PR spgemm: shmem alignment updates for numeric routines #503 @ndellingwood

• spmv unit testing tolerance settings: Issue SpMV unit-test tolerance is incorrectly set #506 PR SpMV tests: fixing the logic to detect the level of precision needed #507 ? @lucbv

• dot-based GEMM (Cuda spec) handling of beta==0 case: Issue Dot-based GEMM Cuda specialization - update for beta == 0 case #509 @seheracer
  Follow-up - possible slight loosening of tolerance in failing Trilinos tests to allow for diffs caused by the parallelism introduced by these performance enhancements

• spgemm symbolic phase fixes (mem pool, ...) @brian-kelley
  See KokkosKernels: SpGEMM issue causes Tpetra: mult_A_B_newmatrix hangs in Allreduce trilinos/Trilinos#6237

• update of Ifpack2 calls to *gauss_seidel_apply in packages/ifpack2/src/Ifpack2_Relaxation_def.hpp - needs to be added to the kokkos-promotion branch

• resolve issues during Trilinos configuration with call to KOKKOS_CHECK at line 113 of kokkos-kernels/CMakeLists.txt (function is defined in kokkos file kokkos/cmake/KokkosConfig.cmake.in) @jjwilke

Adding @srajama1 @crtrott and @jjwilke so they are in the loop.

Edit: Added link to kokkos-promotion branch of Trilinos.

[ndellingwood]

Issues to resolve exposed by Trilinos testing:

• batched_lu_serial: fix for ambiguous ? operation error (Sacado) : PR batched_lu_serial: fix for ambiguous ? operation error #512

[ndellingwood]

• Recent issue in ATDM testing with Tpetra MV following Dot based gemm: Add different beta cases and allow fewer than 32 teams  #510 -related patch
  See TpetraCore_MultiVector_UnitTests_MPI_4 and TpetraCore_gemm_m_eq_<X>_MPI_1 tests in build Trilinos-atdm-sems-rhel7-cuda-9.2-Volta70-complex-shared-release-debug failing starting 2019-12-07  trilinos/Trilinos#6418

[ndellingwood]

• gemv transpose mode with TPLs gemv transpose mode issues with cublas and M == 0 #539
• perf test link errors Link errors building KokkosSparse_gs.exe perf test #527

[ndellingwood]

This was all completed for 3.0, closing.