-
-
Notifications
You must be signed in to change notification settings - Fork 267
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CI: Bump LDC-LLVM to v18.1.3 (except for Android and macOS arm64) #4604
Conversation
I tried to fix the ASan and fuzzer lit tests. |
Oh nice, thx Johan! Note that this is totally a draft and will probably be split up into at least 2 PRs, so expect further force-pushes etc. My main interest here is Android and native TLS - I've gotten rid of our TLS hack in our LDC-LLVM 18, and the prebuilt Android LLVM binaries are built with the latest NDK, requiring Android >= 10. After very few mods here, it seems to cross-compile & -link just fine for 32-bit Android, but fails with undefined The current macOS arm64 error wrt. unknown stack probe is most likely caused by the prebuilt LLVM binaries being compiled on macOS 14 (I've switched to the native M1 CI runners in the LLVM repo). So that's probably going to need #4541 in some form. Windows required 2 further CMake tweaks; the current libxml2 issue for Win64 is going to be fixed in some hours. |
d622e96
to
068285e
Compare
dae4d1a
to
b29b6bb
Compare
@thewilsonator: 2 out of the 3 remaining lit-test failures are dcompute-related (
It'd be nice if you could look into that. |
Oh WTF - sick & tired of the macOS issues. I thought I could just cherry-pick #4541 and fix the 'unsupported stack probe' error that way - as both LLVM and LDC are compiled on the same GHA macos-14 image (and with its default Xcode version) then. And hoped that the But no, of course not. First of all, the 'unsupported stack probe' error persisted. Disabling LTO and PGO works around that. But now the stage-2 compiler (compiled with itself) crashes multiple times with |
Okay, slowly getting somewhere now wrt. macOS arm64. The |
I.e., in https://github.com/ldc-developers/ldc/actions/runs/8486647167/job/23253351620?pr=4604, there's now a single 'pure virtual function called' error, for the |
@thewilsonator I tried to find the cause of the crash of |
The macOS arm64 LTO problems appear serious. The 'unsupported stack probe' seems to come from the C++ IR (Apple clang v15.0 vs. LLVM 18 from stage-1 bootstrap compiler); disabling LTO on the C++ side fixed that error. But the LTO'd |
The numerous |
Built LLVM from |
Also that code hasn't been changed for a while. Only changed with some extension addition. |
The
With enabled assertions? The tests pass without assertions, as shown by the vanilla-LLVM-workflow job. |
Yep my bad, built |
Has this ever worked on an LLVM with assertions enabled? e.g. LLVM 16/17 (with opaque pointers enabled?). i.e. is this an LLVM regression or a problem on our behalf? |
Yes, works for me with LLVM 17. (where same build params with LLVM 18 result in assertion hit) There is a difference in how 17 and 18 transform the IR in the pipeline. Compare the output using |
Wrt. macOS arm64, I was about to give up, running out of ideas - but then switching from default Xcode v15.0 to latest v15.2 (in the GHA image) finally seems to have vanquished the With LTO for the D parts only, the |
And 3 In the meantime, I'm preparing new prebuilt binaries for our LLVM fork - rebased onto latest upstream |
I can try to reproduce the LTO+PGO issue locally, but will take a little bit, tinkering with build settings edit: as before, I can repro driver/config_diag.d issue with LTO (works without LTO). Updating to new macos and latest xcode now. |
Oh no, the Edit: The problem with the macOS arm64 GHA runners is that they only support the macos-14 image, so I cannot simply use the macos-12 image as for the x86_64 job and see if it works with an older OS/toolchain image, which has no issues on x86 at least. |
@JohanEngelen: If you have some time, it'd be great if you could try to reproduce the apparently consistent failure with enabled optimizations only (no stacktrace unfortunately, so I don't know which test it is):
AFAICT, the ctor expects the buffer to be at least pointer-aligned. And this stack-allocated buffer with alignment 1 looks very suspicious then, but there might be more: https://github.com/dlang/phobos/blob/a2ade9dec49e70c6acd447df52321988a4c2fb9f/std/experimental/allocator/building_blocks/kernighan_ritchie.d#L649-L651 Edit: Yep, at least https://github.com/dlang/phobos/blob/a2ade9dec49e70c6acd447df52321988a4c2fb9f/std/experimental/allocator/building_blocks/kernighan_ritchie.d#L919 too. |
One potentially good news at least - with full LTO (for the D parts alone), |
Aaand we have a lucky CI job again (no virtual-func errors), with full LTO for the D parts alone, and no PGO: https://github.com/ldc-developers/ldc/actions/runs/8495061837/job/23270913949?pr=4604 - only the 2 expected dcompute lit-test failures, and the 2 (shared/static) optimized kernighan_ritchie unittest assertions.
|
Can you point me to what you meant by "not disabling the EH unwind tables" ? |
Building LLVM with (default) |
Testcase: void main() {
throw new Exception("Test exception");
} Before updating macOS (Sonoma 14.2 iirc), this test crashed with Moving on to the kernighan_ritchie unittests. |
Correct, this was the bug. Fixed in LDC phobos (upstream: dlang/phobos#8965). |
Perfect, thx.
With the same Xcode version (which one?)?! And what about the random |
b91210e
to
90cabdc
Compare
LLVM issue for the SPIRV regression llvm/llvm-project#87315 |
For macOS arm64, no improvements wrt. the virtual-func issue with LLVM v18.1.3 and Xcode v15.3 (newly available in the GHA image). |
3af7563
to
17d43af
Compare
I've removed our own TLS emulation for Android, and switched to the latest NDK (r26d). Switching to native TLS supported since Android 10/11 requires a few compiler and druntime changes; I'll follow up. The macOS arm64 LLVM binaries were built natively on macOS 14 arm64, with Xcode v15.3. Experiments with a native LDC build on such a macOS 14 arm64 CI runner show random 'Pure virtual function called' errors for the compiler itself (compiled with itself). Not sure whether these are regressions (with a failure rate of maybe very roughly 1:1000), or happening with the current cross-compiled macOS arm64 binaries too - opting for the safe variant of keeping LLVM 17 for the macOS arm64 job.
Don't know exactly when that LLVM-SPIRV bug was fixed but it works with current LLVM main branch. |
No description provided.