Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GHA: Experiment with native arm64 DMD build on macos-14 #17035

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft

GHA: Experiment with native arm64 DMD build on macos-14 #17035

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
835d2b9
GHA: Test macos-14 arm64 image
kinke Mar 15, 2025
70b5f03
[switch to LDC host compiler, and so a native arm64 DMD build]
kinke Mar 15, 2025
9c858d3
[fix up target `real` properties if host `real_t` is not x87]
kinke Mar 16, 2025
f013b70
[set up Apple clang for cross-linking automatically]
kinke Mar 16, 2025
7cd178d
CI: Add native arm64 hello-world smoke test
kinke Mar 16, 2025
2ab4b9d
[add 2nd macos-14 CI job, using latest DMD as host compiler, and so a…
kinke Mar 16, 2025
bbb794b
[add tentative support for arm64 Mach-O object files]
kinke Mar 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 20 additions & 3 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,11 @@ jobs:
container_image: alpine:3.21
host_dmd: ldmd2
# macOS
- job_name: macOS 13 x64, DMD (latest)
os: macos-13
- job_name: macOS 14 arm64, LDC
os: macos-14
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

macos-14 onwards are native M1 runners. These macOS arm64 runners offer a builtin x86_64 emulator - as opposed to the Linux AArch64 GitHub Actions runners. This means we can still generate x86_64 code and run it directly on these runners.

host_dmd: ldc
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switching the host D compiler to LDC on these macOS arm64 runners means that a native arm64 LDC build is used, generating arm64 binaries. So the fresh DMD is built as a native arm64 executable - but still defaults to generating x86_64 code.

[Using a DMD host compiler yields a fresh x86_64 DMD build, which then passes all tests on the arm64 box. In that case, both the host compiler and the fresh DMD need to run in the Rosetta 2 emulator, as both are x86_64 executables.]

- job_name: macOS 14 x64, DMD (latest)
os: macos-14
host_dmd: dmd
# Disabled because of failure https://issues.dlang.org/show_bug.cgi?id=24518
# - job_name: macOS 13 x64, DMD (coverage)
Expand All @@ -79,7 +82,8 @@ jobs:
name: ${{ matrix.job_name }}
runs-on: ${{ matrix.os }}
container: ${{ matrix.container_image }}
timeout-minutes: 40
# for some reason, the compiler testsuite is extremely slow with x86_64 emulation on macos-14 runners
timeout-minutes: ${{ matrix.os == 'macos-14' && 60 || 40 }}
env:
# for ci/run.sh:
OS_NAME: ${{ startsWith(matrix.os, 'ubuntu') && 'linux' || (startsWith(matrix.os, 'macos') && 'osx' || (startsWith(matrix.os, 'windows') && 'windows' || '')) }}
Expand Down Expand Up @@ -156,7 +160,20 @@ jobs:
uses: seanmiddleditch/gha-setup-vsdevenv@v4
with:
arch: ${{ env.MODEL == '64' && 'x64' || 'x86' }}
- name: 'macOS arm64: Run arm64 smoke test'
if: matrix.os == 'macos-14'
run: |
set -uexo pipefail

cat >hello.d <<EOF
version (AArch64) {} else static assert(0);
extern(C) int puts(const(char)* s);
extern(C) int main() { puts("Hello world from native arm64!"); return 0; }
EOF

generated/osx/release/64/dmd -betterC -arm -run hello.d
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be the very first CI test of DMD compiling & linking a native arm64 executable. It currently fails to link; AFAICT, because the Mach-O object file still specifies the x86_64 architecture, although it contains arm64 code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried to implement the trivial Mach-O header changes; now the linker complains about an invalid relocation:

ld: relocation at '_main'+0x0008 is not supported: r_address=0x8, r_type=1, r_extern=0, r_pcrel=1, r_length=2 in '/Users/runner/work/dmd/dmd/hello.o'

I'm not going down that rabbit hole. ;)

- name: Test dmd
if: success() || failure()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[continue if the arm64 smoke test step failed]

run: ci/run.sh test_dmd
- name: Test druntime
if: '!matrix.coverage && (success() || failure())'
Expand Down
7 changes: 6 additions & 1 deletion ci/cirrusci.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,12 @@ if [ "$OS_NAME" == "linux" ]; then
elif [ "$OS_NAME" == "osx" ]; then
# upgrade GNU make
brew install make
sudo ln -s /usr/local/opt/make/libexec/gnubin/make /usr/local/bin/make
for candidateDir in /usr/local/opt/make/libexec/gnubin /opt/homebrew/opt/make/libexec/gnubin; do
if [[ -f $candidateDir/make ]]; then
sudo ln -s $candidateDir/make /usr/local/bin/make
break
fi
done
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[the location of GNU make is different for the arm64 macOS runners (vs. x86_64 macos-13 runners)]

elif [ "$OS_NAME" == "freebsd" ]; then
packages="git gmake devel/llvm12"
if [ "$HOST_DMD" == "dmd-2.079.0" ] ; then
Expand Down
3 changes: 3 additions & 0 deletions compiler/src/dmd/backend/mach.d
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,13 @@ enum
CPU_TYPE_X86_64 = cast(cpu_type_t)7 | 0x1000000,
CPU_TYPE_POWERPC = cast(cpu_type_t)18,
CPU_TYPE_POWERPC64 = CPU_TYPE_POWERPC | 0x1000000,
CPU_TYPE_ARM = cast(cpu_type_t)12,
CPU_TYPE_ARM64 = CPU_TYPE_ARM | 0x1000000,

// cpusubtype
CPU_SUBTYPE_POWERPC_ALL = cast(cpu_subtype_t)0,
CPU_SUBTYPE_I386_ALL = cast(cpu_subtype_t)3,
CPU_SUBTYPE_ARM64_ALL = cast(cpu_subtype_t)0,

// filetype
MH_OBJECT = 1,
Expand Down
5 changes: 3 additions & 2 deletions compiler/src/dmd/backend/machobj.d
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -642,9 +642,10 @@
{
mach_header_64 header = void;

const isARM64 = config.target_cpu == TARGET_AArch64;

Check warning on line 645 in compiler/src/dmd/backend/machobj.d

View check run for this annotation

Codecov / codecov/patch

compiler/src/dmd/backend/machobj.d#L645 

Added line #L645 was not covered by tests
header.magic = MH_MAGIC_64;
header.cputype = CPU_TYPE_X86_64;
header.cpusubtype = CPU_SUBTYPE_I386_ALL;
header.cputype = isARM64 ? CPU_TYPE_ARM64 : CPU_TYPE_X86_64;
header.cpusubtype = isARM64 ? CPU_SUBTYPE_ARM64_ALL : CPU_SUBTYPE_I386_ALL;

Check warning on line 648 in compiler/src/dmd/backend/machobj.d

View check run for this annotation

Codecov / codecov/patch

compiler/src/dmd/backend/machobj.d#L647-L648 

Added lines #L647 - L648 were not covered by tests
header.filetype = MH_OBJECT;
header.ncmds = 4;
header.sizeofcmds = cast(uint)(segment_command_64.sizeof +
Expand Down
4 changes: 2 additions & 2 deletions compiler/src/dmd/backend/x86/cg87.d
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -649,7 +649,7 @@ ubyte loadconst(elem* e, int im)
immutable double[7] dval =
[0.0,1.0,PI,LOG2T,LOG2E,LOG2,LN2];

static if (real.sizeof < 10)
static if (!is(targ_ldouble == real))
{
import dmd.root.longdouble;
immutable targ_ldouble[7] ldval =
Expand Down Expand Up @@ -742,7 +742,7 @@ ubyte loadconst(elem* e, int im)
// Note that for this purpose, -0 is not regarded as +0,
// since FLDZ loads a +0
assert(sz <= zeros.length);
zero = (memcmp(p, zeros.ptr, sz) == 0);
zero = (memcmp(p, zeros.ptr, sz < targ_ldouble.sizeof ? sz : targ_ldouble.sizeof) == 0);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shows one of the current problems for cross-compiling with DMD - it currently assumes the target real is the same as the host real_t (targ_ldouble in the backend).

Any native arm64 DMD build will never be able to use a hardware x87 real_t (and dmd.root.longdouble.longdouble_soft is based on x87 asm, so needs an x86_64 host). The general AArch64 ABI specifies 128-bit quadruple precision for long double, but Apple and Microsoft both opted for double precision.

So this means that the native macOS arm64 DMD build here for CI uses a double-precision real_t for representing compile-time reals. This means that real.max actually overflows to infinity when cross-compiling to x86_64 etc., because the compiler cannot represent the target value.

LDC has the same basic problem/limitation, mentioning this in https://wiki.dlang.org/Cross-compiling_with_LDC#Limitations. GDC uses a software real_t type.

Wrt. DMD, there might easily be more locations in the backend with assumptions that targ_ldouble matches the target real format. I was actually surprised the native arm64 build doesn't seem to crash for the testsuite. :) But the number of test failures is, I think, much higher than expected from a pure compile-time-real-is-less-precise-than-target-real issue.

if (zero && config.target_cpu >= TARGET_PentiumPro)
return 0xEE; // FLDZ is the only one with 1 micro-op

Expand Down
6 changes: 4 additions & 2 deletions compiler/src/dmd/lib/scanmach.d
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
//import core.sys.darwin.mach.loader;
import dmd.backend.mach;
import dmd.root.string : fTuple;
import dmd.target : target;

nothrow:

Expand Down Expand Up @@ -79,9 +80,10 @@
header64 = cast(mach_header_64*)buf;
if (buflen < mach_header_64.sizeof)
return corrupt(__LINE__);
if (header64.cputype != CPU_TYPE_X86_64)
const expectedCPUType = target.isAArch64 ? CPU_TYPE_ARM64 : CPU_TYPE_X86_64;
if (header64.cputype != expectedCPUType)

Check warning on line 84 in compiler/src/dmd/lib/scanmach.d

View check run for this annotation

Codecov / codecov/patch

compiler/src/dmd/lib/scanmach.d#L83-L84 

Added lines #L83 - L84 were not covered by tests
{
eSink.error(Loc.initial, "Mach-O object module `%s` has cputype = %d, should be %d", module_name, header64.cputype, CPU_TYPE_X86_64);
eSink.error(Loc.initial, "Mach-O object module `%s` has cputype = %d, should be %d", module_name, header64.cputype, expectedCPUType);

Check warning on line 86 in compiler/src/dmd/lib/scanmach.d

View check run for this annotation

Codecov / codecov/patch

compiler/src/dmd/lib/scanmach.d#L86 

Added line #L86 was not covered by tests
return;
}
if (header64.filetype != MH_OBJECT)
Expand Down
23 changes: 22 additions & 1 deletion compiler/src/dmd/link.d
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -443,10 +443,31 @@ public int runLINK(bool verbose, ErrorSink eSink)
argv.push("-g");
if (target.isX86_64)
argv.push("-m64");
else
else if (target.isX86)
argv.push("-m32");
version (OSX)
{
// might need to set up Apple clang for cross-linking
if (target.os == Target.OS.OSX)
{
version (AArch64)
{
if (target.isX86_64)
{
argv.push("-arch");
argv.push("x86_64");
}
}
else
{
if (target.isAArch64)
{
argv.push("-arch");
argv.push("arm64");
}
}
}

/* Without this switch, ld generates messages of the form:
* ld: warning: could not create compact unwind for __Dmain: offset of saved registers too far to encode
* meaning they are further than 255 bytes from the frame register.
Expand Down
16 changes: 16 additions & 0 deletions compiler/src/dmd/target.d
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -485,6 +485,22 @@
}
}

// fix up RealProperties if the target real is x87, but host real_t isn't
if (realsize - realpad == 10 && RealProperties.mant_dig != 64) {
import dmd.root.ctfloat : CTFloat;

bool isOutOfRange;
RealProperties.max = CTFloat.parse("0x1.fffffffffffffffep+16383", isOutOfRange);
RealProperties.min_normal = CTFloat.parse("0x1p-16382", isOutOfRange);
RealProperties.epsilon = CTFloat.parse("0x1p-63", isOutOfRange);
RealProperties.dig = 18;
RealProperties.mant_dig = 64;
RealProperties.max_exp = 16_384;
RealProperties.min_exp = -16_381;
RealProperties.max_10_exp = 4932;
RealProperties.min_10_exp = -4931;

Check warning on line 501 in compiler/src/dmd/target.d

View check run for this annotation

Codecov / codecov/patch

compiler/src/dmd/target.d#L492-L501 

Added lines #L492 - L501 were not covered by tests
}

c.initialize(params, this);
cpp.initialize(params, this);
objc.initialize(params, this);
Expand Down
Loading