Debug build type

[xmnlab]

Comment:

hi everyone! is there debug build type for llvm already available?

if not, I would like to know if would be possible to have a debug build type for that, maybe we could have it tagged in a different label, for example conda-forge/label/dev

thanks!

[xmnlab]

A friendly reminder about this.

maybe we could use this other PR as a base:
https://github.com/conda-forge/python-feedstock/pull/597/files#diff-f3725a55bf339595bf865fec73bda8ac99f283b0810c205442021f29c06eea9a

cc @ocefpaf

[xmnlab]

hi everyone! a friendly reminder about this issue.

additionally, it would be nice to have llvm assertion enabled. I can open an initial PR for that .. and if that is not ok to have it here .. I can proposal it as a new recipe on staged-recipe. thanks!

[ocefpaf]

I think we can publish a debug version in a dev/debug label. I'm not sure what was the verdict for the Python debug variant.

[h-vetinari]

Yeah, that shouldn't be the default install IMO. I also expect the debug builds to time out on our infrastructure (for osx, it already does about 50% of the time). You can open a PR and see if you can make it work, and if it does, I can point it to an appropriate branch.

[xmnlab]

awesome! thank you so much @ocefpaf and @h-vetinari

[gshimansky]

For reference, this is a patch to windows build that creates debug binaries (that don't also break ABI so I can continue to use clang from conda and link with Release MSVC runtime) of llvm-14 for me. I call conda build like conda build --keep-old-work recipe and then rename work_moved_* into work to have the sources in the same location where llvm is built and therefore allow debugger to find them automatically. You need more than 200Gb of disk space to build everything.

diff --git a/recipe/bld.bat b/recipe/bld.bat
index 53fe8ef..fa24a36 100644
--- a/recipe/bld.bat
+++ b/recipe/bld.bat
@@ -9,9 +9,12 @@ set "CC=cl.exe"
 set "CXX=cl.exe"

 cmake -G "Ninja" ^
-    -DCMAKE_BUILD_TYPE="Release" ^
+    -DCMAKE_BUILD_TYPE="Debug" ^
     -DCMAKE_PREFIX_PATH=%LIBRARY_PREFIX% ^
     -DCMAKE_INSTALL_PREFIX:PATH=%LIBRARY_PREFIX% ^
+    -DCMAKE_CXX_FLAGS_DEBUG=/Z7 ^
+    -DLLVM_USE_CRT_DEBUG=MD ^
+    -DLLVM_ABI_BREAKING_CHECKS=FORCE_OFF ^
     -DLLVM_USE_INTEL_JITEVENTS=ON ^
     -DLLVM_ENABLE_LIBXML2=ON ^
     -DLLVM_ENABLE_RTTI=ON ^
diff --git a/recipe/meta.yaml b/recipe/meta.yaml
index 1cb9131..508407f 100644
--- a/recipe/meta.yaml
+++ b/recipe/meta.yaml
@@ -16,7 +16,7 @@ source:
     - patches/no-windows-symlinks.patch

 build:
-  number: 1
+  number: 2
   merge_build_host: false

 requirements:

[xmnlab]

@gshimansky thank you so much for sharing.
I will start to work on that today!

[gshimansky]

Adding to my previous comment, if you need a debug build of llvm, it is likely that you will still have to build it locally on your system:

1. You need a source tree with files locations matching those recorded in debug information or you will have to lookup every file that you try to open. Alternatively it is possible to enable relative file paths somewhere in cmake.
2. There are multiple variations of debug builds, e.g. you may prefer to link with Debug runtime (-DLLVM_USE_CRT_DEBUG=MDd) instead of Release because it allows better debugging of STL structures, etc. But linking with Debug runtime requires to rebuild every dependency library so that all of them link with Debug runtime too (it may be problematic).
3. You may want to enable ABI breaking checks because with them llvm checks more conditions in runtime and it may let you catch bugs easier. But again, in this case it is necessary to rebuild dependencies that link with llvm libraries, e.g. clang.
4. There may be more modifications, so, if there exists a debug binary build in conda, it may not be what is best for everyone.

[xmnlab]

thanks for the detailed explanation @gshimansky !
so in another words, it is not feasible to try to build it for debugging on conda-forge?

if so, maybe it could be feasible using a post-install (post-link) ... although maybe it is not a great approach ..
something similar we did for https://github.com/conda-forge/cudatoolkit-dev-feedstock ..
of course in a different feedstock (maybe llvmdev-dbg?)

[gshimansky]

It is not unfeasible to create a debug binary build of llvm but the users who install it may be disappointed because it wouldn't serve their needs. When I first started to build llvm with debug I thought that a ready binary would be great and I could debug my application without going through my own build of llvm. When I finished making changes to the build recipe I realized that even if there was a debug binary in conda-forge, I would most likely still had to modify it to make it suitable for my purpose and then had to go through a complete build process.

[xmnlab]

got it .. makes sense! thank you so much @gshimansky for the detailed explanation!

[gshimansky]

This is a patch for Linux build of llvm-14 to enable debug

diff --git a/recipe/build.sh b/recipe/build.sh
index e009aae..8b0fe82 100644
--- a/recipe/build.sh
+++ b/recipe/build.sh
@@ -12,7 +12,7 @@ if [[ "$target_platform" == "linux-64" ]]; then
 fi

 if [[ "$CC_FOR_BUILD" != "" && "$CC_FOR_BUILD" != "$CC" ]]; then
-  CMAKE_ARGS="${CMAKE_ARGS} -DCROSS_TOOLCHAIN_FLAGS_NATIVE=-DCMAKE_C_COMPILER=$CC_FOR_BUILD;-DCMAKE_CXX_COMPILER=$CXX_FOR_BUILD;-DCMAKE_C_FLAGS=-O2;-DCMAKE_CXX_FLAGS=-O2;-DCMAKE_EXE_LINKER_FLAGS=-Wl,-rpath,${BUILD_PREFIX}/lib;-DCMAKE_MODULE_LINKER_FLAGS=;-DCMAKE_SHARED_LINKER_FLAGS=;-DCMAKE_STATIC_LINKER_FLAGS=;-DLLVM_INCLUDE_BENCHMARKS=OFF;"
+  CMAKE_ARGS="${CMAKE_ARGS} -DCROSS_TOOLCHAIN_FLAGS_NATIVE=-DCMAKE_C_COMPILER=$CC_FOR_BUILD;-DCMAKE_CXX_COMPILER=$CXX_FOR_BUILD;-DCMAKE_C_FLAGS=-O0;-DCMAKE_CXX_FLAGS=-O0;-DCMAKE_EXE_LINKER_FLAGS=-Wl,-rpath,${BUILD_PREFIX}/lib;-DCMAKE_MODULE_LINKER_FLAGS=;-DCMAKE_SHARED_LINKER_FLAGS=;-DCMAKE_STATIC_LINKER_FLAGS=;-DLLVM_INCLUDE_BENCHMARKS=OFF;"
   CMAKE_ARGS="${CMAKE_ARGS} -DLLVM_HOST_TRIPLE=$(echo $HOST | sed s/conda/unknown/g) -DLLVM_DEFAULT_TARGET_TRIPLE=$(echo $HOST | sed s/conda/unknown/g)"
 fi

@@ -23,7 +23,7 @@ if [[ "$target_platform" == "linux-ppc64le" ]]; then
 fi

 cmake -DCMAKE_INSTALL_PREFIX="${PREFIX}" \
-      -DCMAKE_BUILD_TYPE=Release \
+      -DCMAKE_BUILD_TYPE=Debug \
       -DHAVE_LIBEDIT=OFF \
       -DLLVM_HAVE_LIBXAR=OFF \
       -DLLVM_ENABLE_LIBXML2=OFF \
@@ -41,6 +41,9 @@ cmake -DCMAKE_INSTALL_PREFIX="${PREFIX}" \
       -DLLVM_BUILD_LLVM_DYLIB=yes \
       -DLLVM_LINK_LLVM_DYLIB=yes \
       -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly \
+      -DLLVM_OPTIMIZE_SANITIZED_BUILDS=OFF \
+      -DCMAKE_CXX_FLAGS_DEBUG="-O0 -g" \
+      -DCMAKE_C_FLAGS_DEBUG="-O0 -g" \
       ${CMAKE_ARGS} \
       -GNinja \
       ../llvm
diff --git a/recipe/meta.yaml b/recipe/meta.yaml
index 1cb9131..508407f 100644
--- a/recipe/meta.yaml
+++ b/recipe/meta.yaml
@@ -16,7 +16,7 @@ source:
     - patches/no-windows-symlinks.patch

 build:
-  number: 1
+  number: 2
   merge_build_host: false

 requirements:

I also had to rebuild clang, probably because when I built debug Linux version I didn't know about -DLLVM_ABI_BREAKING_CHECKS, so my binary could no longer work with clang binary from conda-forge.

[xmnlab]

thanks for the example and for the explanation @gshimansky
I will play with that on this weekend!

[xmnlab]

@gshimansky I didn't have too much progress on this topic .. but I started on this again today.
I will share my updates here soon. thanks!