Adding CMake Presets #1536

MarcelKoch · 2024-01-19T14:12:47Z

This PR can be used to discuss the addition of CMake presets. So far, I've added only very basic presets which are compatible with the current workflows.

But these preset can be used to set common standards, especially regarding warnings. ATM, we only enforce the -Wpedantic warnings for our development builds. We could use the presets to define a more rigorous set of warnings.

There might also be other options we might want to set here, this is only a starting point for a discussion.

The workflow using the presets would be as follows:

cd ginkgo
cmake . --presets develop [cmake options...]
cmake --build build-develop

Also, the presets are usually picked up by IDEs.

upsj · 2024-01-19T15:23:05Z

CMakePresets.json

+    {
+      "name": "develop",
+      "inherits": "base",
+      "cacheVariables": {


From our previous discussions, I would suggest adding CMAKE_EXPORT_COMPILE_COMMANDS=ON, CMAKE_COLOR_DIAGNOSTICS=ON, GINKGO_DEVEL_TOOLS=ON, CMAKE_LINK_DEPENDS_NO_SHARED=ON

upsj · 2024-01-19T15:27:10Z

Another useful feature might be the environment object, with which we can configure path-independent CXX and CUDACXX compiler executable names

MarcelKoch · 2024-01-19T15:28:23Z

CMakePresets.json

+      "inherits": "base",
+      "cacheVariables": {
+        "CMAKE_BUILD_TYPE": "Debug",
+        "CMAKE_CXX_FLAGS": "$env{CXXFLAGS} -Wpedantic"


In terms of warnings I was thinking of

-Wall -Wextra -Wconversion -Wunused

MarcelKoch · 2024-01-19T15:30:17Z

Another useful feature might be the environment object, with which we can configure path-independent CXX and CUDACXX compiler executable names

Are you meaning something like CXX=g++? I think that would make sense if we would extend the presets to be used in our CI, but otherwise I'm not sure of the benefit of that.

upsj · 2024-01-19T15:44:07Z

I could imagine develop-clang being useful from time to time, especially with its better diagnostics

greole · 2024-01-21T07:47:34Z

I guess that should also work with cmakes new --workflow option.

greole · 2024-01-21T07:52:10Z

CMakePresets.json

+    {
+      "name": "base",
+      "hidden": true,
+      "generator": "Ninja",


Do we want to require Ninja as dependency? Often needs a separate module to be loaded and is not available on some systems.

If we use presets version < 3, then we need to specify a generator. But even if we don't need to specify it, I would still prefer to set Ninja, since it seems to handle dependencies better than make, and it's not as verbose. It's always possible to overwrite the settings from the preset, by using the normal CLI arguments.

greole · 2024-01-21T07:59:07Z

cmake/build_helpers.cmake

@@ -113,26 +113,6 @@ macro(ginkgo_modify_flags name)
    string(REPLACE "\"" "\\\"" ${name}_MODIFY "${${name}}")
 endmacro()

-# Extract the clang version from a clang executable path


I guess this is superseded by CMAKE_CXX_COMPILER_VERSION?

Maybe, but this function was never used.

pratikvn · 2024-01-22T13:05:39Z

Can we extend this approach to enable builds for HPC systems ? Frontier, Horeka etc.

upsj · 2024-01-22T13:08:26Z

@pratikvn I think the issue there is that they depend on loaded modules, and the executable names (gcc etc) are the same otherwise, so the configs may not help and differ that much between different systems.

We could consider adding a '-march=native` preset that also includes other optimizations (mixed precision, maybe jacobi full optimizations)

MarcelKoch · 2024-01-22T13:23:39Z

I tend to agree that adding supported machines may clutter the file a bit too much. But it is possible to include configurations from other files, so perhaps we could put these configs into a separate file.

MarcelKoch · 2024-01-24T10:00:02Z

CMakeCIPresets.json

+            }
+        },
+        {
+            "name": "ci-cuda-msvc-nompi-release",


I chose the naming scheme ci-[device (multiple connected with +)]-[toolchain]-[nompi|mpi]-[build_type][optional configs]. Should this be kept?
Also, in our ci we seem to use / as a separator instead of -. We could do the same here, with the side effect, that we would create nested folders.

I think it was just a convention we used when building our containers. IMO using - seems reasonable. Does having nested folders have any advantages ?

The only impact (not sure if it's an advantage) is that the directories used by the Gitlab runner share the same nested structure.

I don't think there would be a difference, it's just preference. Usually you would only use one CI preset anyway.

greole · 2024-03-12T09:47:46Z

CMakePresets.json

+                "rhs": "Windows"
+            }
+        }
+    ]


Would it makes sense to also add buildPresets like

Suggested change

]

],

"buildPresets": [

{

"name": "develop-gcc",

"configurePreset": "develop-gcc"

}

]

that way one can build with cmake --build --preset develop-gcc without knowing the exact binaryDir

For the 'normal' configuration I can see this making sense. But for the CI configs it would probably be too much.

I think buildPresets make especially sense in combination with the workflow option. So that a user can configure and build a preset with a single command. However, workflows are only available for version 6 and above, which needs cmake 3.25.

Just another thought on the buildPresets, we could also add a format preset with a targets
key

"targets": [ "format", ]

However, I don't know how to then just pick the "current" binaryDir via the preset.

I think we'll probably move away from that with pre-commit handling the formatting entirely? Also make format or ninja format is not really more complicated than calling it via preset.

The thing I like about presets is that it allows you to run cmake --build --preset format from the source directory without knowing the corresponding build directory. However, you are right that moving towards precommit makes it unnecessary.

greole · 2024-03-12T09:53:29Z

CMakePresets.json

+        {
+            "name": "base",
+            "hidden": true,
+            "binaryDir": "${sourceDir}/build-${presetName}"


My personal preference would be nesting the build folder and adding buildPresets. So that there won't be multiple build- folders in the sourceDir

Suggested change

"binaryDir": "${sourceDir}/build-${presetName}"

"binaryDir": "${sourceDir}/build/${presetName}"

I would agree. Although typically there wouldn't be too many different build dirs anyway.

Thinking about it, I usually build in a build/<branchName> directory to keep different builds separated. But I guess that information is not available for CMakePresets.

No, but it's always possible to overwrite this from the CLI

If you really want to go for build/<branchName>, it might be possible to realize it via git hooks.

greole

LGTM!

I think, however, we should also mention it the presets in the README.md and maybe give an example of how to use presets in combination with normal flags to enable compiling specific backends.
And if possible add a release version of the presets.

greole · 2024-04-08T05:46:28Z

CMakePresets.json

+            "name": "base-dev",
+            "hidden": true,
+            "cacheVariables": {
+                "CMAKE_BUILD_TYPE": "Debug",


Nowadays I use presets almost everywhere, thus having a Release version would be also great.

The CMake variable is now marked as advanced. There is no incompatibility with other CMake options anymore. The source tuning_variables.cpp is added to the device libs cuda, dpcpp, and hip if the CMake option is set.

MarcelKoch · 2024-08-13T15:18:35Z

.gitlab/scripts.yml

+    - |
+      if [ "$(get_cache_val GINKGO_RUN_EXAMPLES)" == "ON" ]; then
+         export EX_ARG="reference"
+         ninja run_all_examples
+         ninja validate_all_examples
+         if [ "$(get_cache_val GINKGO_BUILD_OMP)" == "ON" ]; then
+            export EX_ARG="omp"
+            ninja run_all_examples
+            ninja validate_all_examples
+        fi
+        if [ "$(get_cache_val GINKGO_BUILD_CUDA)" == "ON" ]; then
+            export EX_ARG="cuda"
+            ninja run_all_examples
+            ninja validate_all_examples
+        fi
+        if [ "$(get_cache_val GINKGO_BUILD_HIP)" == "ON" ]; then
+            export EX_ARG="hip"
+            ninja run_all_examples
+            ninja validate_all_examples
+        fi
+      fi


Was there a replacement for this, or do we just not test this anymore?

MarcelKoch · 2024-08-13T15:22:23Z

.gitlab-ci.yml

-    CXX_COMPILER: "clang++"
-    BUILD_OMP: "ON"
-    MPI_AS_ROOT: "ON"
-    BUILD_MPI: "ON"
-    CXX_FLAGS: "-Wpedantic -D_GLIBCXX_DEBUG=1"
-    # The tests are prohibitively slow in Debug
-    BUILD_TYPE: "Release"
+    CI_CMAKE_PRESET: "ci-omp-clang-mpi-glibcxx+debug-release"


does this config even make sense to run with _GLIBCXX_DEBUG in release mode?

ginkgo-bot added reg:build This is related to the build system. reg:documentation This is related to documentation. labels Jan 19, 2024

MarcelKoch force-pushed the cmake-presets branch from e292f25 to cf81f69 Compare January 19, 2024 14:23

upsj reviewed Jan 19, 2024

View reviewed changes

MarcelKoch commented Jan 19, 2024

View reviewed changes

greole reviewed Jan 21, 2024

View reviewed changes

MarcelKoch commented Jan 24, 2024

View reviewed changes

MarcelKoch self-assigned this Jan 26, 2024

MarcelKoch force-pushed the cmake-presets branch 5 times, most recently from 9e4ac1d to e996ec5 Compare January 30, 2024 16:54

greole reviewed Mar 12, 2024

View reviewed changes

MarcelKoch added this to the Ginkgo 1.8.0 milestone Apr 5, 2024

greole approved these changes Apr 8, 2024

View reviewed changes

tcojean modified the milestones: Ginkgo 1.8.0, Ginkgo 1.9.0 May 31, 2024

MarcelKoch added 4 commits August 13, 2024 13:51

remove unused cmake function

b75ed2d

don't present BUILD_SHARED_LIBS as a ginkgo option

ebbb239

update GINKGO_BENCHMARK_ENABLE_TUNING usage

137ad34

The CMake variable is now marked as advanced. There is no incompatibility with other CMake options anymore. The source tuning_variables.cpp is added to the device libs cuda, dpcpp, and hip if the CMake option is set.

fix cmake log

6f94f83

MarcelKoch added 2 commits August 13, 2024 13:55

add base CMakePresets.json

30c7497

add CI presets

e6b0e53

MarcelKoch force-pushed the cmake-presets branch from e996ec5 to 4e07903 Compare August 13, 2024 15:08

MarcelKoch added 6 commits August 13, 2024 17:13

use cmake presets in gitlab ci

1275604

fixup! add CI presets

e4c1dba

fixup! add base CMakePresets.json

2735b8e

fixup! use cmake presets in gitlab ci

7703c52

fixup! add CI presets

fd5326a

fixup! add base CMakePresets.json

c5f535d

MarcelKoch force-pushed the cmake-presets branch from 4e07903 to c5f535d Compare August 13, 2024 15:14

MarcelKoch commented Aug 13, 2024

View reviewed changes

MarcelKoch added 2 commits August 13, 2024 17:23

fixup! use cmake presets in gitlab ci

947fc6c

fixup! add CI presets

beadd1b

MarcelKoch marked this pull request as ready for review August 13, 2024 15:28

MarcelKoch requested review from upsj, greole and pratikvn August 13, 2024 15:28

MarcelKoch added 4 commits August 14, 2024 08:58

fixup! add CI presets

93374ce

fixup! add CI presets

583ab2f

fixup! use cmake presets in gitlab ci

8c6a5c7

fixup! use cmake presets in gitlab ci

a5d4575

MarcelKoch mentioned this pull request Oct 14, 2024

Move the majority of jobs to the full pipeline #1694

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding CMake Presets #1536

Adding CMake Presets #1536

MarcelKoch commented Jan 19, 2024 •

edited

Loading

upsj Jan 19, 2024

upsj commented Jan 19, 2024

MarcelKoch Jan 19, 2024

MarcelKoch commented Jan 19, 2024

upsj commented Jan 19, 2024

greole commented Jan 21, 2024

greole Jan 21, 2024

MarcelKoch Jan 26, 2024

greole Jan 21, 2024

MarcelKoch Jan 26, 2024

pratikvn commented Jan 22, 2024

upsj commented Jan 22, 2024

MarcelKoch commented Jan 22, 2024

MarcelKoch Jan 24, 2024

pratikvn Jan 25, 2024

upsj Jan 25, 2024 •

edited

Loading

MarcelKoch Jan 25, 2024

greole Mar 12, 2024

MarcelKoch Mar 12, 2024

greole Mar 12, 2024

greole Apr 4, 2024

upsj Apr 4, 2024 •

edited

Loading

greole Apr 4, 2024

greole Mar 12, 2024

MarcelKoch Mar 12, 2024

greole Mar 12, 2024

MarcelKoch Mar 12, 2024

MarcelKoch Mar 12, 2024

greole left a comment

greole Apr 8, 2024

MarcelKoch Aug 13, 2024

MarcelKoch Aug 13, 2024

-    ]
+    ],
+      "buildPresets": [
+    {
+      "name": "develop-gcc",
+      "configurePreset": "develop-gcc"
+    }
+    ]

	"binaryDir": "${sourceDir}/build-${presetName}"
	"binaryDir": "${sourceDir}/build/${presetName}"

Adding CMake Presets #1536

Are you sure you want to change the base?

Adding CMake Presets #1536

Conversation

MarcelKoch commented Jan 19, 2024 • edited Loading

Choose a reason for hiding this comment

upsj commented Jan 19, 2024

Choose a reason for hiding this comment

MarcelKoch commented Jan 19, 2024

upsj commented Jan 19, 2024

greole commented Jan 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pratikvn commented Jan 22, 2024

upsj commented Jan 22, 2024

MarcelKoch commented Jan 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

upsj Jan 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

upsj Apr 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

greole left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MarcelKoch commented Jan 19, 2024 •

edited

Loading

upsj Jan 25, 2024 •

edited

Loading

upsj Apr 4, 2024 •

edited

Loading