Overhaul of CMake build system to provide both TriBITS and standalone systems #491

jjwilke · 2019-11-05T21:18:01Z

No description provided.

jjwilke · 2019-11-05T21:18:19Z

WIP - needs more testing to make sure rebase didn't break anything

ndellingwood · 2019-11-07T17:42:40Z

@jjwilke what additional testing is needed before merge? Let me know if you'd like me to try some builds.

jjwilke · 2019-11-07T17:43:46Z

Ideally we would test the same configurations that were previously tested with the raw makefiles? Not sure how feasible this is. I haven't had a chance to look at the current testing.

ndellingwood · 2019-11-14T19:25:49Z

@jjwilke I just pushed in an initial go of the cm_generate_makefile script based on the one in kokkos.
I tested locally with Serial+OpenMP enabled, and user installs of BLAS+LAPACK not on one of the paths that CMake searches.
Here is a sample incantation I tested enabling various options:

mkdir kokkos-kernels/build/cm_gen_test
cd kokkos-kernels/build/cm_gen_test
../../cm_generate_makefile.bash --kokkoskernels-path=${PWD}/../../ --kokkos-path=${PWD}/../../../kokkos-cmake --with-serial --with-scalars=double,complex_double --with-ordinals=int --with-offsets=int --with-layouts=layoutleft --with-tpls=blas --with-openmp --compiler=g++  --user-blas-path="/Users/ndellin/Research/TPL/GCC6.4/BLAS-3.6.0" --user-blas-lib="blas" --user-lapack-path="/Users/ndellin/Research/TPL/GCC6.4/lapack-3.6.1" --user-lapack-lib="lapack" --extra-linker-flags="-lgfortran, -lm"

ndellingwood · 2019-11-18T20:10:46Z

@jjwilke I ran into an issue with Cuda builds looking for cuBLAS, and have a question when installing Kokkos.
I pushed a small change to cm_generate_makefile - on my local system (MacOSX) when I install kokkos, the Kokkos_DIR path will need <PATH_TO_INSTALL>/kokkos-install/lib/cmake/Kokkos, but on kokkos-dev-2 it will need <PATH_TO_INSTALL>/kokkos-install/lib64/cmake/Kokkos. It's easy to check for either directory (I pushed the change in), but I wanted to check that this is consistent with what is expected.

With the change above pushed, if I try to configure kokkos-kernels with cuda enabled on kokkos-dev-2, I run into this error:

...
-- Found CUDA: /projects/sems/install/rhel7-x86_64/sems/compiler/cuda/9.2/base (found version "9.2") 
CMake Error at cmake/kokkoskernels_tpls.cmake:258 (INSTALL):
  INSTALL TARGETS given target "KokkosKernels::CUBLAS" which is an alias.
Call Stack (most recent call first):
  cmake/kokkoskernels_tpls.cmake:329 (KOKKOSKERNELS_EXPORT_IMPORTED_TPL)
  cmake/kokkoskernels_tpls.cmake:411 (KOKKOSKERNELS_IMPORT_TPL)
  CMakeLists.txt:371 (INCLUDE)


CMake Error at cmake/kokkoskernels_tpls.cmake:2 (GLOBAL_APPEND):
  Unknown CMake command "GLOBAL_APPEND".
Call Stack (most recent call first):
  cmake/kokkoskernels_tpls.cmake:267 (KOKKOSKERNELS_APPEND_CONFIG_LINE)
  cmake/Modules/FindTPLCUSPARSE.cmake:10 (KOKKOSKERNELS_EXPORT_IMPORTED_TPL)
  cmake/kokkoskernels_tpls.cmake:324 (FIND_PACKAGE)
  cmake/kokkoskernels_tpls.cmake:412 (KOKKOSKERNELS_IMPORT_TPL)
  CMakeLists.txt:371 (INCLUDE)

Edit: Here is my cm_generate_makefile call:

mkdir kokkos-kernels testing/CMakeTesting
cd kokkos-kernels testing/CMakeTesting
../../cm_generate_makefile.bash --kokkoskernels-path=${PWD}/../../ --kokkos-path=${PWD}/../../../kokkos-stable --with-serial --with-cuda --with-scalars=double,complex_double --with-ordinals=int --with-offsets=int --with-layouts=layoutleft --arch=SNB,Volta70 --compiler=${PWD}/../../../kokkos-stable/bin/nvcc_wrapper

ndellingwood · 2019-11-18T21:54:22Z

cmake/kokkoskernels_tpls.cmake

+    IF (${LIB_TYPE} STREQUAL "INTERFACE_LIBRARY")
+      # This is not an imported target
+      # This an interface library that we created
+      INSTALL(


@jjwilke I'm running into issues during configuration when hitting this INSTALL call with a Cuda build due to CUBLAS.
By default, CUBLAS is enabled when Cuda is enabled (I think this was preexisting behavior).
When this install call is hit I see errors like:

CMake Error at cmake/kokkoskernels_tpls.cmake:273 (INSTALL): INSTALL TARGETS given target "KokkosKernels::CUBLAS" which is an alias. Call Stack (most recent call first): cmake/kokkoskernels_tpls.cmake:352 (KOKKOSKERNELS_EXPORT_IMPORTED_TPL) cmake/kokkoskernels_tpls.cmake:437 (KOKKOSKERNELS_IMPORT_TPL) CMakeLists.txt:371 (INCLUDE)

This is for TARGETS set to KokkosKernels::CUBLAS
The ${CMAKE_INSTALL_BINDIR} and ${CMAKE_INSTALL_LIBDIR} are empty strings, though in the FindTPLCUBLAS module ${CUDA_CUBLAS_LIBRARIES} returns the path to the cublas shared lib.

I pushed a fix. I will make this more robust in 3.1. Stupid mistake on my part.

Just tested to confirm it is working, thanks for fast fix!

ndellingwood · 2019-11-18T22:07:08Z

Merge of #424 (and a necessary follow on PR right after) to reduce filename length for the generated cpp files for eti will require changes to

Makefile.kokkos-kernels
src/CMakeLists.txt

Lexicon:

double -> dbl 
complex -> cmplx
float -> flt 
CudaSpace -> CudaS
CudaUVMSpace -> CudaUS
CudaHostPinnedSpace -> CudaHPS

@jjwilke I'll push the necessary updates above to this branch with a merge of develop and fix of the perf_test/sparse/CMakeLists.txt

ndellingwood

Posted a couple questions regarding appended paths ${CMAKE_INSTALL_PREFIX} during install, maybe irrelevant for stand-alone builds.

ndellingwood · 2019-11-18T22:28:52Z

cmake/kokkoskernels_tribits.cmake

+    INCLUDE(CMakePackageConfigHelpers)
+    configure_package_config_file(cmake/KokkosKernelsConfig.cmake.in
+                                  "${KokkosKernels_BINARY_DIR}/KokkosKernelsConfig.cmake"
+                                  INSTALL_DESTINATION ${CMAKE_INSTALL_PREFIX}/lib/cmake/KokkosKernels)


On some systems (e.g. kokkos-dev-2) when I install kokkos, the appended path to ${CMAKE_INSTALL_PREFIX} is lib64/cmake/KokkosKernels instead of lib/cmake/KokkosKernels. Is a change needed here to make this consistent between the two packages?

Yeah, we should probably use the same trick with GNUInstallDirs

ndellingwood · 2019-11-18T22:29:05Z

cmake/kokkoskernels_tribits.cmake

+    INSTALL(FILES
+      "${KokkosKernels_BINARY_DIR}/KokkosKernelsConfig.cmake"
+      "${KokkosKernels_BINARY_DIR}/KokkosKernelsConfigVersion.cmake"
+      DESTINATION ${CMAKE_INSTALL_PREFIX}/lib/cmake/KokkosKernels)


Same comment as above -
On some systems (e.g. kokkos-dev-2) when I install kokkos, the appended path to ${CMAKE_INSTALL_PREFIX} is lib64/cmake/KokkosKernels instead of lib/cmake/KokkosKernels. Is a change needed here to make this consistent between the two packages?

ditto - I can fix this quickly tonight

ndellingwood · 2019-11-18T22:29:13Z

cmake/kokkoskernels_tribits.cmake

+      "${KokkosKernels_BINARY_DIR}/KokkosKernelsConfigVersion.cmake"
+      DESTINATION ${CMAKE_INSTALL_PREFIX}/lib/cmake/KokkosKernels)
+
+    INSTALL(EXPORT KokkosKernelsTargets DESTINATION lib/cmake/KokkosKernels NAMESPACE Kokkos::)


Same comment as above -
On some systems (e.g. kokkos-dev-2) when I install kokkos, the appended path to ${CMAKE_INSTALL_PREFIX} is lib64/cmake/KokkosKernels instead of lib/cmake/KokkosKernels. Is a change needed here to make this consistent between the two packages?

jjwilke · 2019-11-18T22:48:31Z

the cm_generate_makefile.bash should probably use Kokkos_ROOT instead of Kokkos_DIR to avoid lib/lib64 issues

ndellingwood · 2019-11-18T23:03:36Z

cmake/kokkoskernels_tpls.cmake

-  KOKKOSKERNELS_IMPORT_TPL(CUBLAS)
-  KOKKOSKERNELS_IMPORT_TPL(CUSPARSE)
-  KOKKOSKERNELS_IMPORT_TPL(CUBLAS)
+  KOKKOSKERNELS_IMPORT_TPL(CUBLAS INTERFACE)
  KOKKOSKERNELS_IMPORT_TPL(CUSPARSE)


Why doesn't CUSPARSE also have INTERFACE target property?

It has to do with whether it's a single library or involves multiple libraries. You can only have a single IMPORTED_LOCATION for an imported library.

ndellingwood · 2019-11-18T23:11:42Z

I can now configure for Cuda builds on white and kokkos-dev-2 with the cm_generate_makefile script, for example:

<PATH_TO_KERNELS>/cm_generate_makefile.bash --kokkoskernels-path=${PWD}/../../ --kokkos-path=${PWD}/../../../kokkos-testing --with-serial --with-cuda --with-scalars=double,complex_double --with-ordinals=int --with-offsets=int --with-layouts=layoutleft --arch=Power8,Pascal60 --compiler=${PWD}/../../../kokkos-testing/bin/nvcc_wrapper

ndellingwood · 2019-11-18T23:27:27Z

Edit: Update - this is an issue in the cm_generate_makefile script

Now I have build errors with Cuda

kokkos-dev-2:

SHAs:
kokkos/kokkos@4cbdceb develop branch
f3e82cb cmake-rebased branch

Environment

[ndellin@kokkos-dev-2 CMake-testing]$ module list
Currently Loaded Modulefiles:
  1) sems-env            2) sems-git/2.10.1     3) sems-cmake/3.12.2   4) sems-cuda/9.2       5) sems-gcc/7.2.0

Configure:

<PATH_TO_KERNELS>/cm_generate_makefile.bash --kokkoskernels-path=${PWD}/../../ --kokkos-path=${PWD}/../../../kokkos-testing --with-serial --with-cuda --with-scalars=double,complex_double --with-ordinals=int --with-offsets=int --with-layouts=layoutleft --arch=Power8,Pascal60 --compiler=${PWD}/../../../kokkos-testing/bin/nvcc_wrapper

Build errors:

[ndellin@kokkos-dev-2 CMake-testing]$ make -j1 2>&1 | tee make.error
[  0%] Building CXX object src/CMakeFiles/kokkoskernels.dir/dummy.cpp.o
g++: error: unrecognized command line option ‘-arch=sm_70’
g++: error: unrecognized command line option ‘-Xcudafe’
g++: error: unrecognized command line option ‘--diag_suppress=esa_on_defaulted_function_ignored’
make[2]: *** [src/CMakeFiles/kokkoskernels.dir/dummy.cpp.o] Error 1
make[1]: *** [src/CMakeFiles/kokkoskernels.dir/all] Error 2
make: *** [all] Error 2

ndellingwood · 2019-11-18T23:33:13Z

~~The Cuda build error occurs also when calling cmake directly, for example (stripping away scalar, ordinal, etc. options)~~

cmake <PATH_TO_KERNELS> -DKokkos_ENABLE_CUDA=ON -DKokkos_DIR=<PATH_TO_INSTALLED_KOKKOS>/lib64/cmake/Kokkos -DKokkosKernels_ENABLE_TESTS=ON -DKokkos_ARCH_VOLTA70=ON -DKokkos_ARCH_SNB=ON

Edit: Updated

I didn't set nvcc_wrapper in my cmake line above. Add -DCMAKE_CXX_COMPILER=<KOKKOS_PATH>/bin/nvcc_wrapper resolved the issue, so there is a problem in the cm_generate_makefile

ndellingwood · 2019-11-18T23:44:15Z

Pushed a fix for the cm_generate_makefile for the missing compiler command option to the kokkos-kernels configure line.

ndellingwood · 2019-11-18T23:51:20Z

Cuda build on kokkos-dev-2 had link errors to CUBLAS

[ 96%] Built target sparse_spmv
Scanning dependencies of target KokkosKernels_sparse_cuda
../../src/libkokkoskernels.a(KokkosSparse_spmv_eti_spec_inst_double_int_int_LayoutLeft_Cuda_CudaSpace.cpp.o): In function `KokkosBlas::Impl::Scal<Kokkos::View<double*, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<1u> >, double, Kokkos::View<double const*, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<1u> >, 1, true, true>::scal(Kokkos::View<double*, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<1u> > const&, double const&, Kokkos::View<double const*, Kokkos::LayoutLeft, Kokkos::Device<Kokkos::Cuda, Kokkos::CudaSpace>, Kokkos::MemoryTraits<1u> > const&)':
tmpxft_0001897f_00000000-5_KokkosSparse_spmv_eti_spec_inst_double_int_int_LayoutLeft_Cuda_CudaSpace.cudafe1.cpp:(.text._ZN10KokkosBlas4Impl4ScalIN6Kokkos4ViewIPdJNS2_10LayoutLeftENS2_6DeviceINS2_4CudaENS2_9CudaSpaceEEENS2_12MemoryTraitsILj1EEEEEEdNS3_IPKdJS5_S9_SB_EEELi1ELb1ELb1EE4scalERKSC_RSD_RKSF_[_ZN10KokkosBlas4Impl4ScalIN6Kokkos4ViewIPdJNS2_10LayoutLeftENS2_6DeviceINS2_4CudaENS2_9CudaSpaceEEENS2_12MemoryTraitsILj1EEEEEEdNS3_IPKdJS5_S9_SB_EEELi1ELb1ELb1EE4scalERKSC_RSD_RKSF_]+0xf2): undefined reference to `cublasDscal_v2'
../../src/libkokkoskernels.a(KokkosBlas_Cuda_tpl.cpp.o): In function `KokkosBlas::Impl::CudaBlasSingleton::CudaBlasSingleton()::{lambda()#1}::operator()() const':
tmpxft_00017020_00000000-5_KokkosBlas_Cuda_tpl.cudafe1.cpp:(.text+0x214): undefined reference to `cublasDestroy_v2'
../../src/libkokkoskernels.a(KokkosBlas_Cuda_tpl.cpp.o): In function `KokkosBlas::Impl::CudaBlasSingleton::CudaBlasSingleton()':
tmpxft_00017020_00000000-5_KokkosBlas_Cuda_tpl.cudafe1.cpp:(.text+0x231): undefined reference to `cublasCreate_v2'
collect2: error: ld returned 1 exit status
make[2]: *** [perf_test/sparse/sparse_spmv_struct] Error 1
make[1]: *** [perf_test/sparse/CMakeFiles/sparse_spmv_struct.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs...

ndellingwood · 2019-11-19T00:33:56Z

cmake/kokkoskernels_tpls.cmake

+
+
+MACRO(kokkoskernels_create_imported_tpl NAME)
+  CMAKE_PARSE_ARGUMENTS(TPL


@jjwilke I'm having link errors with CUBLAS in a Cuda build.
If I check output after the parse by adding this line:
MESSAGE(STATUS "AFTER PARSE: TPL_UNPARSED_ARGUMENTS = ${TPL_UNPARSED_ARGUMENTS}")

I see the following:
AFTER PARSE: TPL_UNPARSED_ARGUMENTS = LIBRARIES;/projects/sems/install/rhel7-x86_64/sems/compiler/cuda/9.2/base/lib64/libcublas.so

It looks like there is inconsistency in the FindTPLCUBLAS.cmake file when passing args to this marco, LIBRARIES is passed in instead of LIBRARY. If I change "LIBRARY" in the macro to "LIBRARIES" it parses but causes errors.

Making the change in FindTPLCUBLAS to pass LIBRARY using CUDA_CUBLAS_LIBRARY as library path parses correctly and configures, will test out a rebuild.

@jjwilke Same linking error with the change I mention above but the macro parse correctly. Should I push the change in to FindTPLCUBLAS (change "LIBRARIES" to "LIBRARY")?

LIBRARIES should be LINK_LIBRARIES

ndellingwood · 2019-11-19T05:01:10Z

The TPL fix allowed my build to complete.

Running ctest shows this test seg-faults:

[ndellin@kokkos-dev-2 CMake-testing]$ ctest -R sparse_cuda -VV
...
2: [ RUN      ] cuda.sparse_findRelOffset_double_int_int_TestExecSpace
1/1 Test #2: sparse_cuda ......................***Exception: SegFault 14.19 sec

The issue is that findRelOffset (in kokkos-kernels/src/sparse/KokkosSparse_findRelOffset.hpp) is accepts Views with allocations on the GPU, however throughout the function all executable code runs on the host. This happens to work in Trilinos because UVM is required.

The test is disabled in Cuda builds in the Makefile build system with a note that this is known to only run sequentially on host (same with trsv), but was re-enabled with these CMake updates. I'll open a separate issue as this is relevant to the Tpetra (and broader Trilinos) effort to move away from UVM requirements.

jjwilke · 2019-11-19T17:01:52Z

The failing tests should now be conditionally removed when UVM is not enabled.

mhoemmen · 2019-11-19T17:04:37Z

@ndellingwood wrote:

The issue is that findRelOffset (in kokkos-kernels/src/sparse/KokkosSparse_findRelOffset.hpp) is accepts Views with allocations on the GPU, however throughout the function all executable code runs on the host.

The findRelOffset function is marked as KOKKOS_FUNCTION and can certainly be called from either device or host. Are you perhaps referring to whatever function is calling findRelOffset?

ndellingwood · 2019-11-19T17:37:37Z

The findRelOffset function is marked as KOKKOS_FUNCTION and can certainly be called from either device or host. Are you perhaps referring to whatever function is calling findRelOffset?

Yes, along with the possibility this routine can be called as a free function from the host with Views or pointers to device data (which is what the unit test was doing). The View case can be checked easily by checking if the data is accessible on host. The tougher check is when a pointer to device data is passed to the function and findRelOffset is called from the host, which occurs in the unit test and caused the seg fault with Cuda (the input indices only needs to have operator[] implemented).

ndellingwood · 2019-11-19T20:15:01Z

@jjwilke On kokkos-dev-2 I tested with cusparse enabled and double+complex_double; everything configured and built successfully, but the following spgemm test had a runtime error:

2: [ RUN      ] cuda.sparse_spgemm_kokkos_complex_double_int_int_TestExecSpace
2: terminate called after throwing an instance of 'std::runtime_error'
2:   what():  cudaFuncGetAttributes( &attr, cuda_parallel_launch_local_memory<DriverType>) error( cudaErrorMisalignedAddress): misaligned address /home/ndellin/kokkos-kernels-stable/testing/CMake-testing/kokkos-install/include/Cuda/Kokkos_Cuda_KernelLaunch.hpp:439
2: Traceback functionality not available
2: 
1/1 Test #2: sparse_cuda ......................***Exception: Child aborted 24.36 sec

Do you think the misaligned address message indicates that changes related to kokkos/kokkos#2255 had an impact here?

Edit: My cm_generate_makefile configuration

<PATH_TO_SCRIPT>/cm_generate_makefile.bash --kokkoskernels-path=<PATH_TO_KERNELS> --kokkos-path=<PATH_TO_KOKKOS> --with-serial --with-cuda --with-scalars=double,complex_double --with-ordinals=int --with-offsets=int --with-layouts=layoutleft --arch=SNB,Volta70 --compiler=<PATH_TO_KOKKOS>/bin/nvcc_wrapper --with-tpls=cublas,cusparse --debug=yes --kokkos-debug=yes

jjwilke · 2019-11-19T23:22:02Z

@dsunder @crtrott My guess is that PR is the culprit. I think we could just delete the alignas line 62 of KokkosComplex.hpp to make sure? This goes beyond my expertise to fix.

jjwilke · 2019-11-20T15:37:04Z

Updated: I can confirm that after removing the alignas from Kokkos::complex the test passes on kokkos-dev-2.

ndellingwood · 2019-11-20T18:08:29Z

@jjwilke Thanks for confirming, I put in issue #500 with some addiontal debug info I gathered so far.

ndellingwood · 2019-11-20T23:11:12Z

@jjwilke looks like we're in a game of whack-a-mole with the TPL changes.

I tried a clean rebuild of kokkos-kernels, serial, with user installs of blas and lapack and I am running into cmake configure issues:

cmake-rebased SHA: b26e213
configure errors:

...
-- Found TPLBLAS: /Users/ndellin/Research/TPL/GCC6.4/BLAS-3.6.0/libblas.a
CMake Error at cmake/kokkoskernels_tpls.cmake:45 (TARGET_LINK_LIBRARIES):
  TARGET_LINK_LIBRARIES can not be used on an ALIAS target.
Call Stack (most recent call first):
  cmake/kokkoskernels_tpls.cmake:248 (KOKKOSKERNELS_CREATE_IMPORTED_TPL)
  cmake/Modules/FindTPLBLAS.cmake:2 (KOKKOSKERNELS_FIND_IMPORTED)
  cmake/kokkoskernels_tpls.cmake:337 (FIND_PACKAGE)
  cmake/kokkoskernels_tpls.cmake:421 (KOKKOSKERNELS_IMPORT_TPL)
  CMakeLists.txt:373 (INCLUDE)


CMake Error at cmake/kokkoskernels_tpls.cmake:271 (INSTALL):
  INSTALL TARGETS given target "BLAS" which is an alias.
Call Stack (most recent call first):
  cmake/kokkoskernels_tpls.cmake:342 (KOKKOSKERNELS_EXPORT_IMPORTED_TPL)
  cmake/kokkoskernels_tpls.cmake:421 (KOKKOSKERNELS_IMPORT_TPL)
  CMakeLists.txt:373 (INCLUDE)


-- Found TPLBLAS: /Users/ndellin/Research/TPL/GCC6.4/lapack-3.6.1/liblapack.a
CMake Error at cmake/kokkoskernels_tpls.cmake:45 (TARGET_LINK_LIBRARIES):
  TARGET_LINK_LIBRARIES can not be used on an ALIAS target.
Call Stack (most recent call first):
  cmake/kokkoskernels_tpls.cmake:248 (KOKKOSKERNELS_CREATE_IMPORTED_TPL)
  cmake/Modules/FindTPLLAPACK.cmake:2 (KOKKOSKERNELS_FIND_IMPORTED)
  cmake/kokkoskernels_tpls.cmake:337 (FIND_PACKAGE)
  cmake/kokkoskernels_tpls.cmake:422 (KOKKOSKERNELS_IMPORT_TPL)
  CMakeLists.txt:373 (INCLUDE)


CMake Error at cmake/kokkoskernels_tpls.cmake:271 (INSTALL):
  INSTALL TARGETS given target "LAPACK" which is an alias.
Call Stack (most recent call first):
  cmake/kokkoskernels_tpls.cmake:342 (KOKKOSKERNELS_EXPORT_IMPORTED_TPL)
  cmake/kokkoskernels_tpls.cmake:422 (KOKKOSKERNELS_IMPORT_TPL)
  CMakeLists.txt:373 (INCLUDE)


-- Performing Test KOKKOSKERNELS_TPL_BLAS_RETURN_COMPLEX
-- Performing Test KOKKOSKERNELS_TPL_BLAS_RETURN_COMPLEX - Failed
...

configure line:
../../cm_generate_makefile.bash --kokkoskernels-path=<KERNELSPATH> --kokkos-path=<KOKKOSPATH> --with-serial --with-scalars=double,complex_double --with-ordinals=int --with-offsets=int --with-layouts=layoutleft --with-tpls=blas --with-openmp --compiler=g++ --user-blas-path="/Users/ndellin/Research/TPL/GCC6.4/BLAS-3.6.0" --user-blas-lib="blas" --user-lapack-path="/Users/ndellin/Research/TPL/GCC6.4/lapack-3.6.1" --user-lapack-lib="lapack" --extra-linker-flags="-lgfortran, -lm"

jjwilke · 2019-11-20T23:47:33Z

@ndellingwood Yep, got one of the lines wrong. Sorry about that. Missed testing the code path with custom libraries. Should be working now?

ndellingwood · 2019-11-21T00:21:59Z

@jjwilke working now, thanks for quick fix!

ndellingwood · 2019-11-26T23:31:38Z

I put in PR #503 with some fixes for the spgemm alignment issue that have come up so far. Testing is blocked on complex_float due to a bug in kokkos, filed an issue in kokkos/kokkos#2567.

When the kokkos issue is resolved and #503 is merged I think we should be ready to merge this PR to allow for more extensive testing by others, @jjwilke would you be okay with that or prefer to hold off?

jjwilke · 2019-11-27T02:17:21Z

Let's do it! I can rebase and remove conflicts.

ndellingwood · 2019-11-27T19:40:40Z

Added cm_test_all_sandia script pattern matching that from kokkos. Functioned properly with a few tests on kokkos-dev-2, but needs more robust testing.

This script acts like, and will replace, the existing gnu-based generate_makefile script in order to configure makefiles for kokkos-kernels using the stand-alone CMake build system.

script for testing with the CMake build system

ndellingwood · 2019-12-03T23:32:33Z

Issue #500 (with PR #503) and #506 (with PR in progress #507) needed before we can resume nightly testing against Kokkos' develop branch, but those issues are unrelated to this CMake PR - this PR just allows us to resume testing against Kokkos' develop branch which exposed the issues.

Ready for merge @jjwilke , here goes!

jjwilke mentioned this pull request Nov 5, 2019

Standalone CMake outside with TriBITS compatibility #471

Closed

jjwilke force-pushed the cmake-rebased branch from 945bf07 to 893180c Compare November 6, 2019 01:43

ndellingwood reviewed Nov 18, 2019

View reviewed changes

ndellingwood mentioned this pull request Nov 18, 2019

Issue 395 -- Filenames are too long #424

Merged

ndellingwood reviewed Nov 18, 2019

View reviewed changes

jjwilke force-pushed the cmake-rebased branch from c8122ed to f3e82cb Compare November 18, 2019 22:45

ndellingwood reviewed Nov 18, 2019

View reviewed changes

ndellingwood reviewed Nov 19, 2019

View reviewed changes

ndellingwood mentioned this pull request Nov 19, 2019

findRelOffset unit test bug: seg-faults when tested in Cuda builds #498

Closed

jjwilke force-pushed the cmake-rebased branch 2 times, most recently from 7deaf7a to 2b1c748 Compare November 19, 2019 16:59

ndellingwood mentioned this pull request Nov 20, 2019

spgemm cuda build with complex_double: error( cudaErrorMisalignedAddress) with Kokkos develop branch #500

Closed

This was referenced Nov 21, 2019

Post-CMake merge and 3.0 items #486

Closed

FindMKL module: which mkl versions to prioritize #480

Closed

"standard" install location #467

Closed

atomic_add issue with complex<float> kokkos/kokkos#2567

Closed

This was referenced Nov 27, 2019

spgemm: shmem alignment updates for numeric routines #503

Merged

statement is unreachable warning messages in gs handle with cuda #504

Closed

jjwilke and others added 7 commits November 27, 2019 11:43

cmake overhaul: Issue #120

960510e

Add cm_generate_makefile script

8a44648

This script acts like, and will replace, the existing gnu-based generate_makefile script in order to configure makefiles for kokkos-kernels using the stand-alone CMake build system.

print TPL info at end, other TPL fixes

a9d4ac7

Conditionally remove relOffset when UVM not enabled: #498

e0b0993

move to GNUInstallDirs

b07096e

cm_test_all_sandia script added

037b770

script for testing with the CMake build system

cm_test_all_sandia: cleanup

c4f5f2a

jjwilke force-pushed the cmake-rebased branch from 066fd9b to c4f5f2a Compare November 27, 2019 20:46

ndellingwood approved these changes Dec 3, 2019

View reviewed changes

ndellingwood merged commit 1bb1b1f into develop Dec 3, 2019

ndellingwood mentioned this pull request Jan 28, 2020

Kokkos + KokkosKernels Promotion To 2.9.99 trilinos/Trilinos#6671

Merged

ndellingwood deleted the cmake-rebased branch March 10, 2020 16:56



		MACRO(kokkoskernels_create_imported_tpl NAME)
		CMAKE_PARSE_ARGUMENTS(TPL

Overhaul of CMake build system to provide both TriBITS and standalone systems #491

Overhaul of CMake build system to provide both TriBITS and standalone systems #491

Conversation

jjwilke commented Nov 5, 2019

jjwilke commented Nov 5, 2019

ndellingwood commented Nov 7, 2019

jjwilke commented Nov 7, 2019

ndellingwood commented Nov 14, 2019

ndellingwood commented Nov 18, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ndellingwood commented Nov 18, 2019

ndellingwood left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jjwilke commented Nov 18, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ndellingwood commented Nov 18, 2019

ndellingwood commented Nov 18, 2019 • edited Loading

ndellingwood commented Nov 18, 2019 • edited Loading

ndellingwood commented Nov 18, 2019

ndellingwood commented Nov 18, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ndellingwood commented Nov 19, 2019

jjwilke commented Nov 19, 2019

mhoemmen commented Nov 19, 2019

ndellingwood commented Nov 19, 2019

ndellingwood commented Nov 19, 2019 • edited Loading

jjwilke commented Nov 19, 2019

jjwilke commented Nov 20, 2019

ndellingwood commented Nov 20, 2019

ndellingwood commented Nov 20, 2019

jjwilke commented Nov 20, 2019

ndellingwood commented Nov 21, 2019

ndellingwood commented Nov 26, 2019

jjwilke commented Nov 27, 2019

ndellingwood commented Nov 27, 2019

ndellingwood commented Dec 3, 2019

ndellingwood commented Nov 18, 2019 •

edited

Loading

ndellingwood commented Nov 18, 2019 •

edited

Loading

ndellingwood commented Nov 18, 2019 •

edited

Loading

ndellingwood commented Nov 19, 2019 •

edited

Loading