Incline CPU Testing + CI

[cameronrutherford]

From #23, need to add same CI config to Incline, and create issue for Spack modules on Incline in CI

[cameronrutherford]

Note we will probably have to have incline-cpu vs incline-gpu, and as @nkoukpaizan might want to use other compilers, we can also have ascent-llvm vs ascent-gcc and go from there? I think keeping this name tied to a spack config is key, and making the names simple and easy to use would be nice as well.

[jaelynlitz]

Incline build and ctest work, but when trying make test_install it tries to compile with clang15.0.0 and throws some nvcc and cuda errors - error running locally pasted to provide some more clarity to the same error in the pipeline https://gitlab.pnnl.gov/exasgd/resolve-mirror/-/jobs/131463

[05:47 PM] [litz372@dmi01 ReSolve] $ source buildsystem/incline-env.sh 
[05:47 PM] [litz372@dmi01 ReSolve] $ cd build
[05:47 PM] [litz372@dmi01 build] $ ctest
Test project /qfs/projects/exasgd/src/jaelyn-spack/ReSolve/build
    Start 1: klu_klu_test
1/6 Test #1: klu_klu_test .....................   Passed    0.05 sec
    Start 2: matrix_test
2/6 Test #2: matrix_test ......................   Passed    0.01 sec
    Start 3: matrix_handler_test
3/6 Test #3: matrix_handler_test ..............   Passed    0.01 sec
    Start 4: vector_handler_test
4/6 Test #4: vector_handler_test ..............   Passed    0.01 sec
    Start 5: gram_schmidt_test
5/6 Test #5: gram_schmidt_test ................   Passed    0.01 sec
    Start 6: logger_test
6/6 Test #6: logger_test ......................   Passed    0.01 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) =   0.12 sec
[05:47 PM] [litz372@dmi01 build] $ make test_install
/qfs/projects/exasgd/src/jaelyn-spack/ReSolve/install
/qfs/projects/exasgd/src/jaelyn-spack/ReSolve/tests/functionality/
/qfs/projects/exasgd/src/jaelyn-spack/ReSolve/install/share/examples/resolve_consumer
-- The CXX compiler identification is Clang 15.0.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/rocm-5.3.0/llvm/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /qfs/projects/exasgd/resolve/spack-ci/install/linux-centos7-zen/gcc-8.4.0/cmake-3.27.7-tu2rruqvexd7ngddj2f3cac3bpiw4bnr/share/cmake-3.27/Modules/CMakeDetermineCUDACompiler.cmake:180 (message):
  Failed to find nvcc.

  Compiler requires the CUDA toolkit.  Please set the CUDAToolkit_ROOT
  variable.
Call Stack (most recent call first):
  /qfs/projects/exasgd/src/jaelyn-spack/ReSolve/install/share/resolve/cmake/ReSolveConfig.cmake:10 (enable_language)
  CMakeLists.txt:10 (find_package)


-- Configuring incomplete, errors occurred!
make[3]: *** [examples/CMakeFiles/test_install.dir/build.make:70: examples/CMakeFiles/test_install] Error 1
make[2]: *** [CMakeFiles/Makefile2:569: examples/CMakeFiles/test_install.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:576: examples/CMakeFiles/test_install.dir/rule] Error 2
make: *** [Makefile:283: test_install] Error 2

[jaelynlitz]

Started working on the module rebuild pipelines in the meantime... some perms issues for the service account runner: https://gitlab.pnnl.gov/exasgd/resolve-mirror/-/jobs/131472#L75

Ping Kevin Fox? Or we just make the resolve dir writeable for the exasgd group (like for exago ci)?

[pelesh]

Incline build and ctest work, but when trying make test_install it tries to compile with clang15.0.0 and throws some nvcc and cuda errors - error running locally pasted to provide some more clarity to the same error in the pipeline https://gitlab.pnnl.gov/exasgd/resolve-mirror/-/jobs/131463

That's my bad, @jaelynlitz. If you don't mind, please create issue and assign it to me. In the meantime, you can comment out make test_install until I fix this.

[pelesh]

Incline build and ctest work, but when trying make test_install it tries to compile with clang15.0.0 and throws some nvcc and cuda errors - error running locally pasted to provide some more clarity to the same error in the pipeline https://gitlab.pnnl.gov/exasgd/resolve-mirror/-/jobs/131463

I believe this issue is fixed in #34.

[jaelynlitz]

Incline build and test pipelines pass after a rebase on develop with fixes from @ryandanehy and @pelesh! If we want to merge this progress, I can remove the code associated with the spack rebuild pipelines and tackle those in a separate PR.

[pelesh]

Incline build and test pipelines pass after a rebase on develop with fixes from @ryandanehy and @pelesh! If we want to merge this progress, I can remove the code associated with the spack rebuild pipelines and tackle those in a separate PR.

I think that would be a good break point. I suggest we merge this progress, first.

[cameronrutherford]

Ready to merge

[pelesh]

Will merge as soon as all tests pass.