GHA: Experiment with native arm64 DMD build on macos-14 #17035

kinke · 2024-10-28T14:48:49Z

No description provided.

dlang-bot · 2024-10-28T14:48:52Z

Thanks for your pull request and interest in making D better, @kinke! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

My PR is fully covered with tests (you can see the coverage diff by visiting the details link of the codecov check)
My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
I have provided a detailed rationale explaining my changes
New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.

If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#17035"

kinke · 2024-10-28T19:15:09Z

The macos-14 GHA runners are ARM64 ones, so using Rosetta emulation all the way. And that seems to be uber-slow for the DMD testsuite (maybe because spawning a lot of compile & link processes?).

It might be worth a try to use an LDC host compiler for that job, and building/using an ARM64 DMD (which obviously still generates x86_64 code). Edit: Oh, no dmd.root.longdouble.longdouble_soft support when building DMD for a non-x86 architecture.

kinke · 2025-03-16T15:16:05Z

.github/workflows/main.yml

-            os: macos-13
-            host_dmd: dmd
+          - job_name: macOS 14 arm64, LDC
+            os: macos-14


macos-14 onwards are native M1 runners. These macOS arm64 runners offer a builtin x86_64 emulator - as opposed to the Linux AArch64 GitHub Actions runners. This means we can still generate x86_64 code and run it directly on these runners.

kinke · 2025-03-16T15:21:12Z

.github/workflows/main.yml

-            host_dmd: dmd
+          - job_name: macOS 14 arm64, LDC
+            os: macos-14
+            host_dmd: ldc


Switching the host D compiler to LDC on these macOS arm64 runners means that a native arm64 LDC build is used, generating arm64 binaries. So the fresh DMD is built as a native arm64 executable - but still defaults to generating x86_64 code.

[Using a DMD host compiler yields a fresh x86_64 DMD build, which then passes all tests on the arm64 box. In that case, both the host compiler and the fresh DMD need to run in the Rosetta 2 emulator, as both are x86_64 executables.]

kinke · 2025-03-16T15:24:37Z

.github/workflows/main.yml

+          extern(C) int main() { puts("Hello world from native arm64!"); return 0; }
+          EOF
+
+          generated/osx/release/64/dmd -betterC -arm -run hello.d


This would be the very first CI test of DMD compiling & linking a native arm64 executable. It currently fails to link; AFAICT, because the Mach-O object file still specifies the x86_64 architecture, although it contains arm64 code.

I've tried to implement the trivial Mach-O header changes; now the linker complains about an invalid relocation:

ld: relocation at '_main'+0x0008 is not supported: r_address=0x8, r_type=1, r_extern=0, r_pcrel=1, r_length=2 in '/Users/runner/work/dmd/dmd/hello.o'

I'm not going down that rabbit hole. ;)

kinke · 2025-03-16T15:25:11Z

.github/workflows/main.yml

      - name: Test dmd
+        if: success() || failure()


[continue if the arm64 smoke test step failed]

kinke · 2025-03-16T15:26:04Z

ci/cirrusci.sh

+      sudo ln -s $candidateDir/make /usr/local/bin/make
+      break
+    fi
+  done


[the location of GNU make is different for the arm64 macOS runners (vs. x86_64 macos-13 runners)]

kinke · 2025-03-16T15:42:18Z

compiler/src/dmd/backend/x86/cg87.d

@@ -742,7 +742,7 @@ ubyte loadconst(elem* e, int im)
    // Note that for this purpose, -0 is not regarded as +0,
    // since FLDZ loads a +0
    assert(sz <= zeros.length);
-    zero = (memcmp(p, zeros.ptr, sz) == 0);
+    zero = (memcmp(p, zeros.ptr, sz < targ_ldouble.sizeof ? sz : targ_ldouble.sizeof) == 0);


This shows one of the current problems for cross-compiling with DMD - it currently assumes the target real is the same as the host real_t (targ_ldouble in the backend).

Any native arm64 DMD build will never be able to use a hardware x87 real_t (and dmd.root.longdouble.longdouble_soft is based on x87 asm, so needs an x86_64 host). The general AArch64 ABI specifies 128-bit quadruple precision for long double, but Apple and Microsoft both opted for double precision.

So this means that the native macOS arm64 DMD build here for CI uses a double-precision real_t for representing compile-time reals. This means that real.max actually overflows to infinity when cross-compiling to x86_64 etc., because the compiler cannot represent the target value.

LDC has the same basic problem/limitation, mentioning this in https://wiki.dlang.org/Cross-compiling_with_LDC#Limitations. GDC uses a software real_t type.

Wrt. DMD, there might easily be more locations in the backend with assumptions that targ_ldouble matches the target real format. I was actually surprised the native arm64 build doesn't seem to crash for the testsuite. :) But the number of test failures is, I think, much higher than expected from a pure compile-time-real-is-less-precise-than-target-real issue.

stale

kinke · 2025-03-16T15:48:23Z

@WalterBright: These are some first steps towards arm64 CI for DMD.

…n emulated fresh x86_64 DMD build]

thewilsonator previously approved these changes Oct 28, 2024

View reviewed changes

kinke force-pushed the gha_bump_macos branch from 629d134 to 35ea522 Compare October 28, 2024 15:18

kinke force-pushed the gha_bump_macos branch 4 times, most recently from c368a6c to e7b262e Compare October 29, 2024 22:10

dlang-bot added the Merge:needs rebase label Dec 4, 2024

GHA: Test macos-14 arm64 image

835d2b9

kinke force-pushed the gha_bump_macos branch 2 times, most recently from 1ca35cb to c1fa370 Compare March 15, 2025 23:30

[switch to LDC host compiler, and so a native arm64 DMD build]

70b5f03

kinke force-pushed the gha_bump_macos branch 2 times, most recently from 82a0489 to 20be77e Compare March 16, 2025 01:59

kinke changed the title ~~GHA: Bump macOS images to 13 and 14~~ GHA: Experiment with native arm64 DMD build on macos-14 Mar 16, 2025

kinke force-pushed the gha_bump_macos branch from 20be77e to ac0e0e0 Compare March 16, 2025 02:11

kinke removed the Merge:needs rebase label Mar 16, 2025

kinke commented Mar 16, 2025

View reviewed changes

kinke force-pushed the gha_bump_macos branch from ac0e0e0 to 1646627 Compare March 16, 2025 15:22

kinke commented Mar 16, 2025

View reviewed changes

.github/workflows/main.yml

- name: Test dmd

if: success() || failure()

Copy link

Contributor Author

kinke Mar 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[continue if the arm64 smoke test step failed]

kinke commented Mar 16, 2025

View reviewed changes

kinke added 3 commits March 16, 2025 17:18

[fix up target real properties if host real_t is not x87]

9c858d3

[set up Apple clang for cross-linking automatically]

f013b70

CI: Add native arm64 hello-world smoke test

7cd178d

kinke force-pushed the gha_bump_macos branch from 1646627 to 7cd178d Compare March 16, 2025 16:18

[add 2nd macos-14 CI job, using latest DMD as host compiler, and so a…

2ab4b9d

…n emulated fresh x86_64 DMD build]

kinke force-pushed the gha_bump_macos branch from 2e1951a to 2ab4b9d Compare March 16, 2025 16:44

[add tentative support for arm64 Mach-O object files]

bbb794b

kinke mentioned this pull request Mar 17, 2025

Fix: Add -arch x86_64 to CXXFLAGS on macOS ARM64 to Resolve Linker Errors in Mixed C++ and D Testss #21012

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GHA: Experiment with native arm64 DMD build on macos-14 #17035

GHA: Experiment with native arm64 DMD build on macos-14 #17035

kinke commented Oct 28, 2024

dlang-bot commented Oct 28, 2024

kinke commented Oct 28, 2024 •

edited

Loading

kinke Mar 16, 2025

kinke Mar 16, 2025

kinke Mar 16, 2025

kinke Mar 16, 2025

kinke Mar 16, 2025

kinke Mar 16, 2025

kinke Mar 16, 2025

kinke commented Mar 16, 2025

GHA: Experiment with native arm64 DMD build on macos-14 #17035

Are you sure you want to change the base?

GHA: Experiment with native arm64 DMD build on macos-14 #17035

Conversation

kinke commented Oct 28, 2024

dlang-bot commented Oct 28, 2024

Bugzilla references

Testing this PR locally

kinke commented Oct 28, 2024 • edited Loading

kinke Mar 16, 2025

Choose a reason for hiding this comment

kinke Mar 16, 2025

Choose a reason for hiding this comment

kinke Mar 16, 2025

Choose a reason for hiding this comment

kinke Mar 16, 2025

Choose a reason for hiding this comment

kinke Mar 16, 2025

Choose a reason for hiding this comment

kinke Mar 16, 2025

Choose a reason for hiding this comment

kinke Mar 16, 2025

Choose a reason for hiding this comment

kinke commented Mar 16, 2025

kinke commented Oct 28, 2024 •

edited

Loading