Build failures on kokkos-dev with clang/7.0.1 cuda/9.2 and blas/cublas/cusparse tpls

[lucbv]

@ndellingwood
I felt adventurous and tried to do a tpl build on kokkos-dev with clang, here is what my configure script looks like:

#!/usr/bin/bash                                                                                                                                               

module load sems-env
module load kokkos-env
module load sems-cmake/3.12.2
module load sems-clang/7.0.1
module load sems-cuda/9.2

KOKKOS_PATH="/ascldap/users/lberge/kokkos"
KOKKOS_DEVICES="Serial,Cuda"
KOKKOS_ARCHS="Kepler35"
KOKKOSKERNELS_PATH="/ascldap/users/lberge/kk_lucbv"
KOKKOSKERNELS_LAYOUTS="LayoutLeft"                # the layout types to instantiate.                                                                          
KOKKOSKERNELS_SCALARS="double"                    # the scalar types to instantiate =double,float...                                                          
KOKKOSKERNELS_ORDINALS="int"                      # ordinal types to instantiate                                                                              
KOKKOSKERNELS_OFFSETS="int"                       # offset types to instantiate                                                                               
KOKKOSKERNELS_OPTIONS="eti-only"                  # options for kokkoskernels                                                                                 
KOKKOSKERNELS_TPLS="blas,cublas,cusparse"         # tpls to be used as backend when possible                                                                  

KOKKOSKERNELS_INSTALL="/ascldap/users/lberge/kk_lucbv_build/clang_7.0.1_cuda_9.2_tpls"
KOKKOS_INSTALL="${KOKKOSKERNELS_INSTALL}/kokkos-install"

CUDA_PATH="${SEMS_CUDA_ROOT}"
CUDA_OPTIONS=""

rm -rf ${KOKKOS_INSTALL}/CMake*
rm -rf ${KOKKOSKERNELS_INSTALL}/CMake*

cd ${KOKKOSKERNELS_INSTALL}


${KOKKOSKERNELS_PATH}/cm_generate_makefile.bash \
    --kokkos-path="${KOKKOS_PATH}" \
    --kokkoskernels-path="${KOKKOSKERNELS_PATH}" \
    --prefix="${KOKKOSKERNELS_INSTALL}" \
    --with-devices="${KOKKOS_DEVICES}" \
    --arch="${KOKKOS_ARCHS}" \
    --compiler="${SEMS_CLANG_ROOT}/bin/clang++" \
    --cxxflags="-O3 -Wall -Wshadow -pedantic -Werror -Wsign-compare -Wtype-limits -Wuninitialized -std=c++11" \
    --cxxstandard="11" \
    --ldflags="" \
    --with-cuda="${CUDA_PATH}" \
    --with-scalars="${KOKKOSKERNELS_SCALARS}" \
    --with-ordinals="${KOKKOSKERNELS_ORDINALS}" \
    --with-offsets="${KOKKOSKERNELS_OFFSETS}" \
    --with-layouts="${KOKKOSKERNELS_LAYOUTS}" \
    --with-tpls="${KOKKOSKERNELS_TPLS}" \
    --no-examples \
    --with-options=enable_large_mem_tests |& tee configure_clang_cuda_tpls.log

It actually mostly works but there are a few issues with sptrsv and spgemm, I will start working on this a bit... it would be nice for that build to work so we can increase the coverage we have with "cm_test_all_sandia --spot-check-tpls"
make_clang_cuda_tpls.log

[ndellingwood]

@lucbv errors look pretty straightforward (remove unused vars in sptrsv, likely reorder of initialization or listing of member variables in the spgemm handle?), let me know if you want me to put in the sptrsv fix. Once this is cleaned up, we can add this to cm_test_all_sandia for kokkos-dev-2 and I'll set up a nightly build there.

[lucbv]

Yeah, I'm looking into a quick clean-up of the easy things and if I stumble upon something hard I might ask you about it, but that does not seem to likely, hopefully I can make a PR for that quickly.

[lucbv]

Since PR #630 is now merged this issue should be resolved. There is another fix in the cusparse spmv branch but that gets merged separately.

[ndellingwood]

clang/7+cuda was already in the cm_test_all_sandia script, just added a nightly test.