Unexported symbols in lldb-shell :: Unwind/windows-unaligned-x86_64.test #101710
Labels
Comments
kendalharland
changed the title
|
@llvm/issue-subscribers-lldb Author: Kendal Harland (kendalharland)
CC @labath @mstorsjo
OverviewMy local build of this test produces a binary whose To make this test pass, I had to create this local patch which is certainly not the correct way to fix things but demonstrates my problem well. BackgroundIn particular, here are the problems I observe:
As a workaround I’ve manually exported the symbols in build.py using e.g.
Without my local patch, the full failure output with verbose information is below. As you can see, Questions
|
For us, it fails on |
Thanks for confirming! IIRC this is because LLDB does not load the debug information automatically (I am also unsure if it is supposed to) This error should be resolved via the extra |
|
Related to #100474 |
|
This test does pass for me. In a build with LLVM's own PDB reader is indeed a bit incomplete, but AFAIK all buildbots that run configurations with LLDB on Windows run with |
I think this part is expected. .globl makes the symbol visible to other compile units in the same library. To export it to other libraries, extra steps (like the ones you did are needed). At least on windows that is -- with elf, global symbols are exported by default, and must be explicitly hidden if desired.
I guess that's a bug. If they're in the debug info, lldb should find them, whether or not they are exported.
It should, and I think it does so in some cases, but it's quite possible this does not work all the time.
Like I said above, I think the exportedness of the symbol is not the issue, and I don't expect any windows compiler to export them by default (well, maybe some cygwin stuff, which tries to emulate elf behavior). That said, all of the above things could affect the exact kind of debug info (PDB) being produced, and thus lldb's ability to parse it.
From where I'm sitting, the ideal outcome would be to flip the settings to use the "native" pdb reader by default. :) |
|
Thanks for the added context @labath! Flipping the settings to use the native reader by default seems reasonable to me.
Makes sense. This tells me that although I can fix the problem locally by My full set of setup and build commands is: I believe the main difference here might be that you are setting up your build with MSVC and I'm using Clang with MSVC CLI. I can start there and see if that fixes the problem. |
|
Can confirm that switching to MSVC does not fix this for me. |
|
For some reason, the test is now failing for me too, in my local test environment. I have no idea how it worked for me two weeks ago though. But now I see what's going wrong. The issue is that when building the assembly source file, When the assembly files is compiled, the command run looks like To see this in action, run the following: Note that the output contains a couple of sections like When linking the output binary, we do generate PDB debug info, as that's what we request. But as one object file did include DWARF debug info, the output also contains a little bit of DWARF. When LLDB inspects the binary, it looks for both DWARF and PDB. In this case, it finds a little bit of DWARF debug info, and LLDB concludes the search for related debug info, not looking for the PDB. That's why it can't resolve the symbols that normally would be provided by the debug info (either PDB, or DWARF when the test is run in mingw mode). So in this case, there are many ways around this (some more proof of concept, others more reasonable):
This does seem like a clear regression in the Clang 18 timeframe. But what's puzzling me is how the test does seem to work for some, like I would presume for @dzhidzhoev and probably also on some buildbots, and like how it did seem to work for me 2 weeks ago? Can the |
Woah, great job tracking this down. FYI I'm leaving for an extended vacation today; I'll try out the solutions you've mentioned above in my local runs when I return. I appreciate the thorough explanation! |
Could providing a small binary/yaml be a better solution, to reduce this test's dependency on the compiler and assembler default pipeline? |
That would probably work, but it's not exactly ideal. I think the ideal thing would be to get Clang fixed here, to make it not produce DWARF (unless explicitly told to) when building in MSVC mode. Is this test passing for you? Can you look into how that works - rerun the individual test with |
mstorsjo
referenced
this issue
Aug 23, 2024
…cially The -g flag has been selecting whether to emit dwarf or codeview based on the target ABI since 2018, so simply aliasing these flags does the right thing for clang-cl. This moves some code from Clang::ConstructJob to renderDebugOptions to make things a little clearer now that we don't need to keep track of whether we're doing codeview or not in multiple places, and also combines the duplicate handling of the cl vs clang handling of jmc flags as a result. This is mostly NFC, but some -cc1 flags may be rendered in a slightly different order because of the code that was moved around. Differential Revision: https://reviews.llvm.org/D157794
It's not passing on my setup. |
Oh, ok, that explains things! I would have expected there to be some existing buildbots that does test LLDB on Windows/x86_64, but perhaps there isn't one? |
mstorsjo
added a commit
to mstorsjo/llvm-project
that referenced
this issue
Aug 30, 2024
This fixes a regression from f58330c. That commit changed the clang-cl options /Zi and /Z7 to be implemented as aliases of -g rather than having separate handling. This had the unintended effect, that when assembling .s files with clang-cl, the /Z7 option (which implies using CodeView debug info) was treated as a -g option, which causes `ClangAs::ConstructJob` to pick up the option as part of `Args.getLastArg(options::OPT_g_Group)`, which sets the `WantDebug` variable. Within `Clang::ConstructJob`, we check for whether explicit `-gdwarf` or `-gcodeview` options have been set, and if not, we pick the default debug format for the current toolchain. However, in `ClangAs`, if debug info has been enabled, it always adds DWARF debug info. Add similar logic in `ClangAs` - check if the user has explicitly requested either DWARF or CodeView, otherwise look up the toolchain default. If we (either implicitly or explicitly) should be producing CodeView, don't enable the default `ClangAs` DWARF generation. This fixes the issue, where assembling a single `.s` file with clang-cl, with the /Z7 option, causes the file to contain some DWARF sections. This causes the output executable to contain DWARF, in addition to the separate intended main PDB file. By having the output executable contain DWARF sections, LLDB only looks at the (very little) DWARF info in the executable, rather than looking for a separate standalone PDB file. This caused an issue with LLDB's tests, llvm#101710.
mstorsjo
added a commit
that referenced
this issue
Sep 3, 2024
…Z7 (#106686) This fixes a regression from f58330c. That commit changed the clang-cl options /Zi and /Z7 to be implemented as aliases of -g rather than having separate handling. This had the unintended effect, that when assembling .s files with clang-cl, the /Z7 option (which implies using CodeView debug info) was treated as a -g option, which causes `ClangAs::ConstructJob` to pick up the option as part of `Args.getLastArg(options::OPT_g_Group)`, which sets the `WantDebug` variable. Within `Clang::ConstructJob`, we check for whether explicit `-gdwarf` or `-gcodeview` options have been set, and if not, we pick the default debug format for the current toolchain. However, in `ClangAs`, if debug info has been enabled, it always adds DWARF debug info. Add similar logic in `ClangAs` - check if the user has explicitly requested either DWARF or CodeView, otherwise look up the toolchain default. If we (either implicitly or explicitly) should be producing CodeView, don't enable the default `ClangAs` DWARF generation. This fixes the issue, where assembling a single `.s` file with clang-cl, with the /Z7 option, causes the file to contain some DWARF sections. This causes the output executable to contain DWARF, in addition to the separate intended main PDB file. By having the output executable contain DWARF sections, LLDB only looks at the (very little) DWARF info in the executable, rather than looking for a separate standalone PDB file. This caused an issue with LLDB's tests, #101710.
llvmbot
pushed a commit
to llvmbot/llvm-project
that referenced
this issue
Sep 3, 2024
…Z7 (llvm#106686) This fixes a regression from f58330c. That commit changed the clang-cl options /Zi and /Z7 to be implemented as aliases of -g rather than having separate handling. This had the unintended effect, that when assembling .s files with clang-cl, the /Z7 option (which implies using CodeView debug info) was treated as a -g option, which causes `ClangAs::ConstructJob` to pick up the option as part of `Args.getLastArg(options::OPT_g_Group)`, which sets the `WantDebug` variable. Within `Clang::ConstructJob`, we check for whether explicit `-gdwarf` or `-gcodeview` options have been set, and if not, we pick the default debug format for the current toolchain. However, in `ClangAs`, if debug info has been enabled, it always adds DWARF debug info. Add similar logic in `ClangAs` - check if the user has explicitly requested either DWARF or CodeView, otherwise look up the toolchain default. If we (either implicitly or explicitly) should be producing CodeView, don't enable the default `ClangAs` DWARF generation. This fixes the issue, where assembling a single `.s` file with clang-cl, with the /Z7 option, causes the file to contain some DWARF sections. This causes the output executable to contain DWARF, in addition to the separate intended main PDB file. By having the output executable contain DWARF sections, LLDB only looks at the (very little) DWARF info in the executable, rather than looking for a separate standalone PDB file. This caused an issue with LLDB's tests, llvm#101710. (cherry picked from commit fcb7b39)
llvmbot
pushed a commit
to llvmbot/llvm-project
that referenced
this issue
Sep 3, 2024
…Z7 (llvm#106686) This fixes a regression from f58330c. That commit changed the clang-cl options /Zi and /Z7 to be implemented as aliases of -g rather than having separate handling. This had the unintended effect, that when assembling .s files with clang-cl, the /Z7 option (which implies using CodeView debug info) was treated as a -g option, which causes `ClangAs::ConstructJob` to pick up the option as part of `Args.getLastArg(options::OPT_g_Group)`, which sets the `WantDebug` variable. Within `Clang::ConstructJob`, we check for whether explicit `-gdwarf` or `-gcodeview` options have been set, and if not, we pick the default debug format for the current toolchain. However, in `ClangAs`, if debug info has been enabled, it always adds DWARF debug info. Add similar logic in `ClangAs` - check if the user has explicitly requested either DWARF or CodeView, otherwise look up the toolchain default. If we (either implicitly or explicitly) should be producing CodeView, don't enable the default `ClangAs` DWARF generation. This fixes the issue, where assembling a single `.s` file with clang-cl, with the /Z7 option, causes the file to contain some DWARF sections. This causes the output executable to contain DWARF, in addition to the separate intended main PDB file. By having the output executable contain DWARF sections, LLDB only looks at the (very little) DWARF info in the executable, rather than looking for a separate standalone PDB file. This caused an issue with LLDB's tests, llvm#101710. (cherry picked from commit fcb7b39)
llvmbot
pushed a commit
to llvmbot/llvm-project
that referenced
this issue
Sep 3, 2024
…Z7 (llvm#106686) This fixes a regression from f58330c. That commit changed the clang-cl options /Zi and /Z7 to be implemented as aliases of -g rather than having separate handling. This had the unintended effect, that when assembling .s files with clang-cl, the /Z7 option (which implies using CodeView debug info) was treated as a -g option, which causes `ClangAs::ConstructJob` to pick up the option as part of `Args.getLastArg(options::OPT_g_Group)`, which sets the `WantDebug` variable. Within `Clang::ConstructJob`, we check for whether explicit `-gdwarf` or `-gcodeview` options have been set, and if not, we pick the default debug format for the current toolchain. However, in `ClangAs`, if debug info has been enabled, it always adds DWARF debug info. Add similar logic in `ClangAs` - check if the user has explicitly requested either DWARF or CodeView, otherwise look up the toolchain default. If we (either implicitly or explicitly) should be producing CodeView, don't enable the default `ClangAs` DWARF generation. This fixes the issue, where assembling a single `.s` file with clang-cl, with the /Z7 option, causes the file to contain some DWARF sections. This causes the output executable to contain DWARF, in addition to the separate intended main PDB file. By having the output executable contain DWARF sections, LLDB only looks at the (very little) DWARF info in the executable, rather than looking for a separate standalone PDB file. This caused an issue with LLDB's tests, llvm#101710. (cherry picked from commit fcb7b39)
tru
pushed a commit
to llvmbot/llvm-project
that referenced
this issue
Sep 18, 2024
…Z7 (llvm#106686) This fixes a regression from f58330c. That commit changed the clang-cl options /Zi and /Z7 to be implemented as aliases of -g rather than having separate handling. This had the unintended effect, that when assembling .s files with clang-cl, the /Z7 option (which implies using CodeView debug info) was treated as a -g option, which causes `ClangAs::ConstructJob` to pick up the option as part of `Args.getLastArg(options::OPT_g_Group)`, which sets the `WantDebug` variable. Within `Clang::ConstructJob`, we check for whether explicit `-gdwarf` or `-gcodeview` options have been set, and if not, we pick the default debug format for the current toolchain. However, in `ClangAs`, if debug info has been enabled, it always adds DWARF debug info. Add similar logic in `ClangAs` - check if the user has explicitly requested either DWARF or CodeView, otherwise look up the toolchain default. If we (either implicitly or explicitly) should be producing CodeView, don't enable the default `ClangAs` DWARF generation. This fixes the issue, where assembling a single `.s` file with clang-cl, with the /Z7 option, causes the file to contain some DWARF sections. This causes the output executable to contain DWARF, in addition to the separate intended main PDB file. By having the output executable contain DWARF sections, LLDB only looks at the (very little) DWARF info in the executable, rather than looking for a separate standalone PDB file. This caused an issue with LLDB's tests, llvm#101710. (cherry picked from commit fcb7b39)
tru
pushed a commit
to llvmbot/llvm-project
that referenced
this issue
Sep 18, 2024
…Z7 (llvm#106686) This fixes a regression from f58330c. That commit changed the clang-cl options /Zi and /Z7 to be implemented as aliases of -g rather than having separate handling. This had the unintended effect, that when assembling .s files with clang-cl, the /Z7 option (which implies using CodeView debug info) was treated as a -g option, which causes `ClangAs::ConstructJob` to pick up the option as part of `Args.getLastArg(options::OPT_g_Group)`, which sets the `WantDebug` variable. Within `Clang::ConstructJob`, we check for whether explicit `-gdwarf` or `-gcodeview` options have been set, and if not, we pick the default debug format for the current toolchain. However, in `ClangAs`, if debug info has been enabled, it always adds DWARF debug info. Add similar logic in `ClangAs` - check if the user has explicitly requested either DWARF or CodeView, otherwise look up the toolchain default. If we (either implicitly or explicitly) should be producing CodeView, don't enable the default `ClangAs` DWARF generation. This fixes the issue, where assembling a single `.s` file with clang-cl, with the /Z7 option, causes the file to contain some DWARF sections. This causes the output executable to contain DWARF, in addition to the separate intended main PDB file. By having the output executable contain DWARF sections, LLDB only looks at the (very little) DWARF info in the executable, rather than looking for a separate standalone PDB file. This caused an issue with LLDB's tests, llvm#101710. (cherry picked from commit fcb7b39)
weliveindetail
pushed a commit
to weliveindetail/llvm-project
that referenced
this issue
Oct 10, 2024
…Z7 (llvm#106686) This fixes a regression from f58330c. That commit changed the clang-cl options /Zi and /Z7 to be implemented as aliases of -g rather than having separate handling. This had the unintended effect, that when assembling .s files with clang-cl, the /Z7 option (which implies using CodeView debug info) was treated as a -g option, which causes `ClangAs::ConstructJob` to pick up the option as part of `Args.getLastArg(options::OPT_g_Group)`, which sets the `WantDebug` variable. Within `Clang::ConstructJob`, we check for whether explicit `-gdwarf` or `-gcodeview` options have been set, and if not, we pick the default debug format for the current toolchain. However, in `ClangAs`, if debug info has been enabled, it always adds DWARF debug info. Add similar logic in `ClangAs` - check if the user has explicitly requested either DWARF or CodeView, otherwise look up the toolchain default. If we (either implicitly or explicitly) should be producing CodeView, don't enable the default `ClangAs` DWARF generation. This fixes the issue, where assembling a single `.s` file with clang-cl, with the /Z7 option, causes the file to contain some DWARF sections. This causes the output executable to contain DWARF, in addition to the separate intended main PDB file. By having the output executable contain DWARF sections, LLDB only looks at the (very little) DWARF info in the executable, rather than looking for a separate standalone PDB file. This caused an issue with LLDB's tests, llvm#101710.
weliveindetail
pushed a commit
to weliveindetail/llvm-project
that referenced
this issue
Oct 16, 2024
…Z7 (llvm#106686) This fixes a regression from f58330c. That commit changed the clang-cl options /Zi and /Z7 to be implemented as aliases of -g rather than having separate handling. This had the unintended effect, that when assembling .s files with clang-cl, the /Z7 option (which implies using CodeView debug info) was treated as a -g option, which causes `ClangAs::ConstructJob` to pick up the option as part of `Args.getLastArg(options::OPT_g_Group)`, which sets the `WantDebug` variable. Within `Clang::ConstructJob`, we check for whether explicit `-gdwarf` or `-gcodeview` options have been set, and if not, we pick the default debug format for the current toolchain. However, in `ClangAs`, if debug info has been enabled, it always adds DWARF debug info. Add similar logic in `ClangAs` - check if the user has explicitly requested either DWARF or CodeView, otherwise look up the toolchain default. If we (either implicitly or explicitly) should be producing CodeView, don't enable the default `ClangAs` DWARF generation. This fixes the issue, where assembling a single `.s` file with clang-cl, with the /Z7 option, causes the file to contain some DWARF sections. This causes the output executable to contain DWARF, in addition to the separate intended main PDB file. By having the output executable contain DWARF sections, LLDB only looks at the (very little) DWARF info in the executable, rather than looking for a separate standalone PDB file. This caused an issue with LLDB's tests, llvm#101710.
|
We can finally confirm that this is fixed: swiftlang/swift#76894 (comment) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
CC @labath @mstorsjo
Related to swiftlang#9141
Overview
My local build of this test produces a binary whose
realign_stackandcall_funcsymbols are not visible tolldbso the test fails to match to the expected output ofthread backtraceas these symbol names appear as empty strings in the stack trace output.To make this test pass, I had to create this local patch which is certainly not the correct way to fix things but demonstrates my problem well.
Background
In particular, here are the problems I observe:
realign_stackandcall_funcsymbols which are defined as.globlin windows-unaligned-x86_64-asm.s, even thoughlldb-pdbutilindicates that these symbols are “public” in the final executable (The commands below are run in powershell but I built and ran these tests in cmd.exe):LLDB's
image lookupcommand fails to find these symbols by address or name.As a workaround I’ve manually exported the symbols in build.py using e.g.
/EXPORT:realign_stackLLDB doesn’t appear to load PDB symbols from a separate
.pdbfile automatically. It’s unclear if the original test author ran into this problem from their PR discussion so as a workaround I added an explicitadd-dsym ...call to load the symbols.LLDB seems to have a hard time parsing PDB files and crashes when
add-dsymis called. I believe this is a known issue [1], [2]. Given that there are 2 PDB readers to choose from: LLDB’s implementation (default) and the Windows Native reader, as a workaround I’ve used the native one by settingLLDB_USE_NATIVE_PDB_READER=1.Without my local patch, the full failure output with verbose information is below. As you can see,
funcandmainare symbolized in lldb, but the two frames between those calls have no symbol name. The Clang and lld-link invocations are also shown below.Questions
realign_stackandcall_funcsymbols not to exported in one build vs another? Different assembler? Compiler version? Linker version? I would like to fix this locally and then update the test'sREQUIRESstatement if possible.LLDB_USE_NATIVE_PBD_READER=1, there must be some steps I skipped in setting up PDB support and I'd like to document these.The text was updated successfully, but these errors were encountered: