Getting warning bad fde: FDE is really a CIE - GCC 6.3.1 extern "C"

[suedama1756]

Environment:

• Redhat Enterprise Server Linux Server release 7.6
• G++: 6.3.1 20170216 (Redhat 6.3.1-3)
• LLVM 9.0

I'm getting an "bad fde: FDE is really a CIE" warning when making calls to extern "C" functions in GCC 6.3.1. I can make the issue go away if I mark the external function with NoUnwind, however, I don't think this is the correct solution as the function may throw.

Taking a wild guess I tried setting the personality of the external function to __gxx_personality_v0 however this then fails validation as external functions cannot be given a personality.

I'm using LLVM 9.0 as a JIT, I'm using the ORC2 API, however, the issue is also demonstrable using the Kaleidascope JIT here: https://llvm.org/docs/tutorial/index.html The error is raised from the DwarfParser.hpp at the point the generated function is resolved (jitted)

Is there a correct way to deal with this, is it something I need to even worry about?

[dwblaikie]

you don't mention how any of this relates to clang/llvm - where is the error being produced from? where is llvm/clang used in this experiment?

[suedama1756]

you don't mention how any of this relates to clang/llvm - where is the error being produced from? where is llvm/clang used in this experiment?

I kind of figured it would be obvious to those that knew LLVM, however, in hindsight a little more information was probably required. Question updated...

[lhames]

At a minimum it’s usually helpful to provide your platform (Linux, Windows, Darwin, etc.), platform version, the LLVM version you are using and a little bit of context on what you were trying to do when you saw the error. Your update to the question helps a lot. Could you also include the platform you are running on?

Are you able to run this under a debugger and obtain a backtrace for the error? It’s coming from libunwind, so if you have the source for your LLVM build you should be able to grep it for “FDE is really a CIE” to find the line number to break on.

If this is happening in the JIT then my initial best guess is a bug in the eh-frame parsing / re-writing logic.

[suedama1756]

I've updated with version information. I do not have the full source for LLVM build I'm using, its a corporate box which I have limited access to so may take me a while to sort out. In the mean time do you have any ideas on a) are there any function attributes I can explicitly set other than nounwind that could help here. b) is this an issue I need to even care about?

[llvmbot]

@llvm/issue-subscribers-clang-codegen

[lhames]

Very belatedly: I think this is a common issue when using libunwind with JIT'd code on Linux. For a long time the JIT used "host-is-linux" as a proxy for "unwinder is libgcc_s". Since libgcc_s's implementation of __register_frame provided a fast-path if you registered the whole eh-frame section that's what we did. The problem happens when we call __register_frame this way and the unwinder is actually libunwind: The eh-frame section starts with a CIE, and it triggers this error.

@suedama1756 I think it is a problem -- we won't try to register the subsequent frames, so exceptions thrown through the JIT'd code will not behave correctly.

We're in the process of working through a fix for this: bab3981 added new functions to libunwind that have the behavior we want. We need to update the JIT to detect when these functions are available and use them. This will allow the JIT to work correctly with newer versions of libunwind. (Versions before bab3981 probably won't be fixed).

[lygstate]

Very belatedly: I think this is a common issue when using libunwind with JIT'd code on Linux. For a long time the JIT used "host-is-linux" as a proxy for "unwinder is libgcc_s". Since libgcc_s's implementation of __register_frame provided a fast-path if you registered the whole eh-frame section that's what we did. The problem happens when we call __register_frame this way and the unwinder is actually libunwind: The eh-frame section starts with a CIE, and it triggers this error.

@suedama1756 I think it is a problem -- we won't try to register the subsequent frames, so exceptions thrown through the JIT'd code will not behave correctly.

We're in the process of working through a fix for this: bab3981 added new functions to libunwind that have the behavior we want. We need to update the JIT to detect when these functions are available and use them. This will allow the JIT to work correctly with newer versions of libunwind. (Versions before bab3981 probably won't be fixed).

I think I also faced this issue, I am using mesa/llvmpipe that used llvm jit, and it's raised error:

libunwind: __unw_add_dynamic_fde: bad fde: FDE is really a CIE
libunwind: __unw_add_dynamic_fde: bad fde: FDE is really a CIE
libunwind: __unw_add_dynamic_fde: bad fde: FDE is really a CIE
libunwind: __unw_add_dynamic_fde: bad fde: FDE is really a CIE
libunwind: __unw_add_dynamic_fde: bad fde: FDE is really a CIE
libunwind: __unw_add_dynamic_fde: bad fde: FDE is really a CIE

What should I need to do to fixes this? #55584

[lhames]

@lygstate -- @housel committed fixes for libunwind (bab3981), and LLVM and the ORC runtime (981523b) that address this issue in the case where exceptions are registered via the ORC runtime.

You could use the ORC runtime in your JIT to fix this issue. We should also add checks for the new __unw_add_dynamic_eh_frame_section and __unw_remove_dynamic_eh_frame_section functions to the non-ORC-runtime registration paths.

[lhames]

I've filed #55681 to teach LLVM to use @housel's registration functions.

[DoDoENT]

Are there any workarounds or at least a way to silence this?

I've compiled neovim with a custom-built LLVM toolchain (15.0.6) and its luaJIT engine keeps popping this warning all over the place. It makes the editor unusable...

[DoDoENT]

Interestingly, libunwind seems to log that using _LIBUNWIND_DEBUG_LOG macro which should be no-op in non-debug builds, and I definitely compiled my toolchain in release mode.

It appears some cmake script somewhere is naughty and forcibly removes NDEBUG in release builds...

[DoDoENT]

Ah, assertions are always enabled for libunwind. I'll disable them in my build until this issue gets resolved...

[mortezadadgar]

Are there any workarounds or at least a way to silence this?

I've compiled neovim with a custom-built LLVM toolchain (15.0.6) and its luaJIT engine keeps popping this warning all over the place. It makes the editor unusable...

Have you found any solution i'm facing same issue

[lhames]

@mortezadadgar Are you running on Darwin or Linux? If Linux, did you mean to use libunwind as the unwinder?

[mortezadadgar]

I use gentoo, just switched over clang profile I suppose this profile uses libunwind

$ portageq envvar LDFLAGS
-Wl,-O1 -Wl,--as-needed -fuse-ld=lld -rtlib=compiler-rt -unwindlib=libunwind -Wl,--as-needed

[DoDoENT]

Have you found any solution i'm facing same issue

Didn't bother. I'm using my own custom toolchain that I built without assertions so it's silent. And apparently everything works, so it may be that this is just a cosmetic warning...

[mortezadadgar]

@DoDoENT which option disables assertions

[DoDoENT]

@mortezadadgar -DLIBUNWIND_ENABLE_ASSERTIONS=OFF. See my dockerfile.

[mortezadadgar]

@mortezadadgar -DLIBUNWIND_ENABLE_ASSERTIONS=OFF. See my dockerfile.

that's already is disabled in llvm-libunwind ebuild, I suppose it's a bug in clang-15.0.7

[DoDoENT]

... I suppose it's a bug in clang-15.0.7

I wouldn't say. I've disabled that back in my 15.0.6 docker and it works ever since (the latest dockerfile is on 16.0.4, but you can still find the 15.0.7 in git history. Also, the image is available on docker hub.

[soleera]

@mortezadadgar -DLIBUNWIND_ENABLE_ASSERTIONS=OFF. See my dockerfile.

that's already is disabled in llvm-libunwind ebuild, I suppose it's a bug in clang-15.0.7

TL;DR: This is a Gentoo-specific bug which is unrelated to the bug in libunwind that @lhames is talking about!

This is due to the cmake eclass clearing all flags from the selected build type (defaults to RelWithDebInfo in EAPI 8). The intent was to remove the default compiler optimizations, but unfortunately this is also where -DNDEBUG is defined. At the same time, the conditional logic in libunwind's CMakeLists.txt relies on the assumption that NDEBUG will only be undefined in DEBUG build types; when this assumption is broken, LIBUNWIND_ENABLE_ASSERTIONS does nothing.

This same issue has occurred with other LLVM ebuilds, and there was a suggestion to re-enable -DNDEBUG in cmake.ebuild by default, however the current position seems to be that this is something to be done in ebuilds on a per-package basis. As such this should be considered a bug with the llvm-libunwind ebuild on Gentoo; in the meantime, all you need to do is copy the llvm-libunwind ebuild to a local repository, and add the following code under multilib_src_configure():

	# Respect upstream build type assumptions (#910436)
	[[ "${CMAKE_BUILD_TYPE^^}" == DEBUG ]] || append-flags -DNDEBUG

[mortezadadgar]

@soleera Thanks for letting me know about this

[thesamesam]

Fixed on the Gentoo side in gentoo/gentoo@f8fa80b, thanks!

[theoparis]

Still running into this with my own custom LLVM 19 (HEAD) build... -DLIBUNWIND_ENABLE_ASSERTIONS=OFF and -DCMAKE_C_FLAGS="-DNDEBUG" + -DCMAKE_CXX_FLAGS="-DNDEBUG" didn't seem to fix neovim.

[mikezackles]

It seems like I'm running into this as well, specifically when c++ exceptions occur inside the clang interpreter. I'm building against the main branch with a custom 2-stage linux build that attempts to use the llvm unwinder everywhere. The stage2-check-clang target fails with OP's message on the interpreter repl exception tests (both unit & lit). If I disable those tests as in the patch below, the remaining clang tests pass:

diff --git a/clang/test/Interpreter/simple-exception.cpp b/clang/test/Interpreter/simple-exception.cpp
index 6749acd6e6bd..ca816dafc0e7 100644
--- a/clang/test/Interpreter/simple-exception.cpp
+++ b/clang/test/Interpreter/simple-exception.cpp
@@ -1,5 +1,6 @@
 // clang-format off
 // UNSUPPORTED: system-aix
+// UNSUPPORTED: x86_64-linux
 // XFAIL for arm and arm64, or running on Windows.
 // XFAIL: target=arm{{.*}}, system-windows
 // RUN: cat %s | clang-repl | FileCheck %s
diff --git a/clang/unittests/Interpreter/ExceptionTests/InterpreterExceptionTest.cpp b/clang/unittests/Interpreter/ExceptionTests/InterpreterExceptionTest.cpp
index 2f1c4efb381f..79ed95574be0 100644
--- a/clang/unittests/Interpreter/ExceptionTests/InterpreterExceptionTest.cpp
+++ b/clang/unittests/Interpreter/ExceptionTests/InterpreterExceptionTest.cpp
@@ -122,6 +122,11 @@ extern "C" int throw_exception() {
                               Triple.getArch() == llvm::Triple::aarch64_32))
     GTEST_SKIP();
 
+  // https://github.com/llvm/llvm-project/issues/86
+  if (Triple.isOSLinux())
+    GTEST_SKIP();
+
   llvm::cantFail(Interp->ParseAndExecute(ExceptionCode));
   testing::internal::CaptureStdout();
   auto ThrowException =