Calls to Kokkos::min and Kokkos:max are incorrect if ENABLE_KOKKOS=On

[dogunter]

I've been trying to build RAJAPerf with the cmake option ENABLE_KOKKOS=On but it fails with two error messages,

"/home/dog/kokkos/testing/RAJAPerf/src/basic-kokkos/REDUCE3_INT-Kokkos.cpp", line 58: error: namespace "Kokkos" has no member "min"
        m_vmin = Kokkos::min(m_vmin, static_cast<Int_type>(min_value));
                         ^

"/home/dog/kokkos/testing/RAJAPerf/src/basic-kokkos/REDUCE3_INT-Kokkos.cpp", line 59: error: namespace "Kokkos" has no member "max"
        m_vmax = Kokkos::max(m_vmax, static_cast<Int_type>(max_value));
                         ^

Steps to reproduce:

$ module load cmake
$ module load nvhpc/23.1
$ git clone --recursive https://github.com/LLNL/RAJAPerf.git
$ cd RAJAPerf
$ mkdir build; cd build
$ cmake   -DENABLE_KOKKOS=On ..
$ make -j

It appears to be a simple case issue. The correct call should be to Kokkos::Min and Kokkos:Max, respectively.

[rhornung67]

pnig @ajpowelsnl

[dogunter]

I will. I've discovered a more serious flaw in that Kokkos::Min and Kokkos::Max aren't being used as Kokkos defined them.

[ajpowelsnl]

Hi @dogunter and @rhornung67 -- Many thanks for the heads up. I'll look into this.

[ajpowelsnl]

Hi @dogunter and @rhornung67 -- I poked around with this for a couple of hours, and I think the issue might be that you need to update your Kokkos (in $RAJAPERF_ROOT/tpl/kokkos). The code in REDUCE3_INT looks good, and I believe the problems you're having are possibly an effect of somewhat recent namespace changes for Kokkos::min, Kokkos::max.

I would suggest trying the following:

1. in $RAJAPERF_ROOT:

git submodule update --recursive --init

2. cd $RAJAPERF_ROOT/tpl/kokkos

3. git checkout develop

4. git pull

5. cd $RAJAPERF_ROOT

6. mkdir build_kokkos

7. make yourself a "do_cmake" build script in $RAJAPERF_ROOT

# Use <= clang-12 + cuda-11-ish
cmake \
-DENABLE_KOKKOS=ON \
-DKOKKOS_ENABLE_CXX17=OFF \
-DCMAKE_CXX_COMPILER=clang++ or g++  \
-DRAJA_PERFSUITE_ENABLE_TESTS=OFF \
-DENABLE_CUDA=ON \
-DKokkos_ARCH_VOLTA70=ON \
-DCMAKE_CUDA_ARCHITECTURES=70 \
-DCMAKE_BUILD_TYPE=Release ..

8. cd build_kokkos
9. ../do_cmake to configure
10. make -j20 to build

LMK if this suggestion works. We'll continue to troubleshoot if it doesn't. Apologies in advance for the delayed response.

[rhornung67]

@ajpowelsnl you should be able to put up a PR to address this issue. RAJA folks are not maintaining the Kokkos part.

[ajpowelsnl]

@ajpowelsnl you should be able to put up a PR to address this issue. RAJA folks are not maintaining the Kokkos part.

OK, I need to figure out how to make git submodule pull in Kokkos develop.

[rhornung67]

Check out the RAJAPerf develop branch and make sure all submodules are updated recursively. Then make a new branch off that to submit a PR. Go into the Kokkos submodule and checkout the develop branch. Commit that change and make a PR.

[ajpowelsnl]

Check out the RAJAPerf develop branch and make sure all submodules are updated recursively. Then make a new branch off that to submit a PR. Go into the Kokkos submodule and checkout the develop branch. Commit that change and make a PR.

What I would like (I think) is for Kokkos to behave like the other submodules (such that git submodule update --recursive --init updated a specific branch, develop, ideally, all in one swell foop), so folks wouldn't run into these problems. IOW, make this all automatic under the hood.

[rhornung67]

The root cause of the issue that was reported was that the RAJAPerf code was changed but the kokkos submodule version was not changed to be consistent with that change, I believe.

Running 'git submodule update' as you describe will update each submodule, including submodules of submodules to the specific commit hash, tag, etc. in the associated submodule project repo that the version of RAJAPerf is pinned to. Each submodule has to be updated if you want a new version.

We don't want everything pinned to the latest develop branch of each submodule project. I don't think you can even do that with git submodules.

[ajpowelsnl]

Hi @rhornung67 and @dogunter -- I've submitted a #328 with the requested update. LMK if this does not fix you. Apologies for the delay.