Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Druntime test merge fix #4883

Merged
merged 12 commits into from
Mar 20, 2025
Merged

Druntime test merge fix #4883

merged 12 commits into from
Mar 20, 2025

Conversation

the-horo
Copy link
Contributor

Fix the druntime tests merge conflicts as well as resolve some issues with the test code. Some changes are specific to LDC and should remain here, some are bugs that should be fixed upstream but currently they're all in separate commits and will be split and squashed later.

@the-horo the-horo mentioned this pull request Mar 17, 2025
Draft
kinke
kinke reviewed Mar 17, 2025
View reviewed changes
runtime/druntime/test/common.mak
@@ -1,5 +1,5 @@
# set explicitly in the make cmdline in druntime/Makefile (`test/%/.run` rule):
ifneq (,$(findstring ldmd2,$(DMD)))
ifdef IN_LDC
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm, much better, I like! 👍

@kinke
Copy link
Member

kinke commented Mar 18, 2025

It might be worth reconsidering splitting the D flags into compile- and link-flags - there's no harm in including link-flags in compiles too, just a bit longer/more explicit command lines in the make output (normally hidden).

@the-horo
Copy link
Contributor Author

It might be worth reconsidering splitting the D flags into compile- and link-flags - there's no harm in including link-flags in compiles too, just a bit longer/more explicit command lines in the make output (normally hidden).

DFLAGS and extra_dflags do apply to both compilation and linking, on top of most of the tests doing compilation+linking in one compiler invocation (stdcpp is the only one that does manual compilation because of the need to link the C++ stdlib).

I would rather not change this design because I purposefully made it follow the way other build systems do it, and if they do it this way I assume that there is a reason for it. So, until I fully understand this reason I would rather go with the possibly sub-optimal design.

@the-horo the-horo force-pushed the pr-2.111 branch 3 times, most recently from d2ca7c6 to e60ead6 Compare March 18, 2025 09:28
@the-horo
Copy link
Contributor Author

I've marked which commits should be upstreamed.

The failure which I gathered are:

  • windows druntime-test-exceptions-debug > winstack
  • windows druntime-test-shared > loadDR
  • alpine druntime-test-shared > loadDR
  • macos druntime-test-shared > load_linkdep
  • most platforms importc_compare

The loadDR test on windows logs:

C:/Program Files/LLVM/bin/clang-cl.exe  /FoD:/a/ldc/build/runtime/druntime-test-shared-debug/       src/loadDR.c    /FeD:/a/ldc/build/runtime/druntime-test-shared-debug/loadDR.exe
Testing loadDR
D:/a/ldc/build/runtime/druntime-test-shared-debug/loadDR.exe D:/a/ldc/build/lib/druntime-ldc-debug-shared.lib
Assertion failed: h != NULL, file src/loadDR.c, line 11

  • D:/a/ldc/build/runtime/druntime-test-shared-debug/loadDR.exe D:/a/ldc/build/lib/druntime-ldc-debug-shared.lib

Shouldn't the library path on windows by the .dll instead of the .lib?

@the-horo
Copy link
Contributor Author

Yes! The old makefile changed the suffix from .lib to .dll:

ldc/runtime/druntime/test/shared/Makefile

Line 82 in 61c11b7

$(ROOT)/loadDR.done $(ROOT)/host.done: RUN_ARGS:=$(DRUNTIMESO:.lib=.dll)

But I changed that as I assumed the DRUNTIMESO would be the dll:

ldc/runtime/druntime/test/shared/Makefile

Line 149 in e60ead6

$(ROOT)/loadDR.done: private run_args = $(DRUNTIMESO)

which it is within dmd but not within ldc.

@kinke
Copy link
Member

kinke commented Mar 18, 2025

DFLAGS and extra_dflags do apply to both compilation and linking

Ah okay, that is fine then.

@kinke
Copy link
Member

kinke commented Mar 18, 2025

I changed that as I assumed the DRUNTIMESO would be the dll

IIRC, upstream changed that not too long ago. I went with DRUNTIMESO as import .lib because that's needed much more often than the DLL (so no need for variable druntime_for_linking). Anyway, switching to the DLL should be a matter of using TARGET_FILE here:

ldc/runtime/DRuntimeIntegrationTests.cmake

Line 31 in 61c11b7

set(shared_druntime_path "$<TARGET_LINKER_FILE:druntime-ldc${SHARED_LIB_SUFFIX}>")

@kinke
Copy link
Member

kinke commented Mar 18, 2025

And with -g for the compiled test files, we'll e.g. get file+line infos in stack traces for failing (debug) tests:

core.exception.AssertError@src/lib.d(6): Assertion failure
----------------
/Users/runner/work/ldc/ldc/runtime/druntime/src/core/exception.d:569 onAssertError [0x100f9d403]
/Users/runner/work/ldc/ldc/runtime/druntime/src/core/exception.d:808 _d_assert [0x10100ca3f]
??:? void lib._assert(bool) [0x100f32d13]
??:? void lib.runTestsImpl() [0x100f3364b] => which line?
??:? runTests [0x100f334fb]
??:? runDepTests [0x100f23e27]
??:? _Dmain [0x100dbb73b]

@the-horo
Copy link
Contributor Author

the-horo commented Mar 19, 2025 •
edited
Loading

I've looked a little bit more:

  • windows druntime-test-exceptions-debug > winstack
    Fixed by passing -g

  • windows druntime-test-shared > loadDR
    Works correctly now that the .dll is passed on the command line

  • alpine druntime-test-shared > loadDR
    Before:

1808: /usr/bin/cc -Wall -Wl,-rpath,/__w/ldc/build/lib -g -o/__w/ldc/build/runtime/druntime-test-shared-debug/loadDR src/loadDR.c -ldl  -pthread
1808: Testing loadDR
1808: /__w/ldc/build/runtime/druntime-test-shared-debug/loadDR /__w/ldc/build/lib/libdruntime-ldc-debug-shared.so

Now:

1840: /usr/bin/clang        src/loadDR.c  -ldl  -o /__w/ldc/build/runtime/druntime-test-shared-debug/loadDR
1840: Testing loadDR
1840: /__w/ldc/build/runtime/druntime-test-shared-debug/loadDR /__w/ldc/build/lib/libdruntime-ldc-debug-shared.so.111.0
1840: Aborting from rt/sections_elf_shared.d(966) Failed to get library handle.gmake: *** [../common.mak:182: /__w/ldc/build/runtime/druntime-test-shared-debug/loadDR.done] Aborted (core dumped)
  • macos druntime-test-shared > load_linkdep
    This test should be skipped, using OS before including common.mak leads to it being run

    ldc/runtime/druntime/test/shared/Makefile

    Lines 16 to 22 in 8069d9a

    # FIXME: `load_linkdep` needs a non-dummy `getDependencies()` in rt.sections_elf_shared,
    # not implemented yet on Darwin and Windows
    ifneq (osx,$(OS))
    ifneq (windows,$(OS))
    TESTS+=load_linkdep
    endif
    endif

A new test started failing:

  • windows druntime-test-shared > dll_gc_proxy_teardown
    It seems to be caused by flags not being properly passed.

kinke
kinke reviewed Mar 19, 2025
View reviewed changes
runtime/druntime/test/shared/Makefile
@@ -179,12 +147,14 @@ $(ROOT)/linkD$(DOTEXE): private extra_ldlibs += $(abs_root)/lib$(for_linking)
$(ROOT)/linkDR.done: $(ROOT)/lib$(DOTDLL)
$(ROOT)/linkDR$(DOTEXE): $(DRUNTIME_DEP) utils.h
$(ROOT)/linkDR$(DOTEXE): private extra_ldlibs += $(ldl) $(druntime_for_linking)
ifneq ($(OS),windows)
$(ROOT)/linkDR$(DOTEXE): private extra_ldflags += -Wl,-rpath,$(dir $(druntime_for_linking))
endif

$(ROOT)/loadDR.done: $(ROOT)/lib$(DOTDLL) $(DRUNTIMESO)
$(ROOT)/loadDR.done: private run_args = $(DRUNTIMESO)
$(ROOT)/loadDR$(DOTEXE): utils.h
$(ROOT)/loadDR$(DOTEXE): private extra_ldlibs += $(ldl)
Copy link
Member

@kinke kinke Mar 19, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess Alpine/musl somehow needs the rpath too (as for linkDR, and what we previously enforced in CMake, CFLAGS_BASE) - the assertion/abort comes from being unable to load a dependency of either druntime or the lib.so.

Edit: I've already used ifeq (1,$(shell which apk &>/dev/null && echo 1)) as rough Alpine detection.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You've already added a check like this to upstream's exceptions test but I think it will be easier if we keep the musl detection in cmake (since we already have a snippet for that) and just pass IS_MUSL=1 to the makefiles.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay setting the rpath worked. - Are you going to upstream that IS_MUSL then?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so this failed for ldc because it uses -soname when building the shared druntime when upstream doesn't. I already split other -rpath fixes in a commit for upstream so I think it's appropriate to do the same here. I should probably move the IS_MUSL conditional up in common.mak and change it to something that detects musl not just Alpine since the tests fail in my Musl chroot as well

the-horo added 5 commits March 20, 2025 14:14
@the-horo
runtime/druntime/test: Fix
bd04964 
Signed-off-by: Andrei Horodniceanu <a.horodniceanu@proton.me>
@the-horo
[dmd] druntime/test/common.mak: Invoke test executable without ./
974e208 
The $(OBJDIR) variable already contains the ./ leading prefix and,
when it is set to an absolute path (like ldc does), prepending ./ is
an error.

Signed-off-by: Andrei Horodniceanu <a.horodniceanu@proton.me>
@the-horo
CI: macOS install a more recent GNU make
c8bd21c 
The druntime testsuite requires a more recent version of make then
what is shipped by default with macos.

Signed-off-by: Andrei Horodniceanu <a.horodniceanu@proton.me>
@the-horo
[dmd] druntime/test/shared/Makefile: Add trailing slash to /Fo argument
e371a10 
As per msvc docs, when a directory is specified to /Fo it should end
with a slash. cl.exe seems to handle the argument anyway but
clang-cl.exe fails with it, which is used by ldc.

Signed-off-by: Andrei Horodniceanu <a.horodniceanu@proton.me>
@the-horo
[dmd] druntime/test: Pass -rpath to druntime for posix
b7aa743 
When linking executables through CC also pass -rpath to the directory
that contains the druntime library, in case the shared variant would
be linked.

This is currently only a problem for ldc because the posix linkers
encode the full path to the druntime library in DT_NEEDED (which
avoids requiring -rpath during runtime) only if said library has no
soname. ldc's druntime is built with soname but dmd's isn't which hid
the issue.

Signed-off-by: Andrei Horodniceanu <a.horodniceanu@proton.me>
the-horo added 7 commits March 20, 2025 14:14
@the-horo
[dmd] druntime/test/stdcpp/windows: Don't overwrite TARGET_ARCH
ef83030 
This just discarded the -mMODEL flag on windows which is obviously
undesired.

Signed-off-by: Andrei Horodniceanu <a.horodniceanu@proton.me>
@the-horo
DRuntimeIntegrationTests.cmake: Don't pass -rpath
6200bed 
The -rpath argument is correctly handled by the tests which need it.

Signed-off-by: Andrei Horodniceanu <a.horodniceanu@proton.me>
@the-horo
[dmd] druntime/test/common.mak: Pass -dllimport in dflags instead of …
138e7b5 
…ldflags

Since it's a flag that affects compilation for correctness it should be
passed in dflags rather than ldflags.d. There were no issues visible
because the compilation and linking are done in one step in most of
the tests.

Signed-off-by: Andrei Horodniceanu <a.horodniceanu@proton.me>
@the-horo
[dmd] druntime/test/exceptions: run winstack test regardless of BUILD
2992f36 
Signed-off-by: Andrei Horodniceanu <a.horodniceanu@proton.me>
@the-horo
DRuntimeIntegrationTests.cmake: Pass DRUNTIMESO=druntime.dll on windows
3a4a879 
Pass the .dll instead of the .lib when invoking the testsuite.

Signed-off-by: Andrei Horodniceanu <a.horodniceanu@proton.me>
@the-horo
DRuntimeIntegrationTests.cmake: Pass IS_MUSL
baae9d8 
Signed-off-by: Andrei Horodniceanu <a.horodniceanu@proton.me>
@the-horo
druntime/test/shared loadDR: Pass -rpath on musl
169b61b 
The test fails on musl without the flag:
```
/usr/bin/cc  -g -Wall       src/loadDR.c  -ldl  -o /root/build/runtime/druntime-test-shared-debug/loadDR
Testing loadDR
/root/build/runtime/druntime-test-shared-debug/loadDR /root/build/lib/libdruntime-ldc-debug-shared.so.111.0
Aborting from rt/sections_elf_shared.d(966) Failed to get library handle.gmake: *** [../common.mak:192: /root/build/runtime/druntime-test-shared-debug/loadDR.done] Aborted (core dumped)
```

Signed-off-by: Andrei Horodniceanu <a.horodniceanu@proton.me>
@kinke kinke merged commit e69bed7 into ldc-developers:merge-2.111 Mar 20, 2025
6 of 20 checks passed
@kinke
Copy link
Member

kinke commented Mar 20, 2025

Thanks buddy!

@the-horo
Copy link
Contributor Author

There are still a few more tests failing which I accidentally overlooked but they should only be a few, I will go through them again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kinke kinke kinke left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@the-horo @kinke