Add macOS/arm64 cross-compilation Azure CI job incl. prebuilt package

[kinke]

Accompanying LLVM CI addition: ldc-developers/llvm-project@c65e187

[kinke]

Can anyone with M1 hardware give this native macOS/arm64 package a try? https://github.com/ldc-developers/ldc/releases/download/CI/ldc2-f092783d-osx-arm64.tar.xz
@p0nce maybe?

[p0nce]

Can this wait tomorrow?

[p0nce]

Hello, here is the result of my tests.
I get this ICE with almost every program:

ponce@ponces-Mac repro % ldc2 main.d                           
0  ldc2                     0x0000000102c3f720 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 52
1  ldc2                     0x0000000102c3e740 llvm::sys::RunSignalHandlers() + 72
2  ldc2                     0x0000000102c3fd18 SignalHandler(int) + 292
3  libsystem_platform.dylib 0x0000000189fa0c44 _sigtramp + 56
4  ldc2                     0x0000000102ed7810 (anonymous namespace)::pushVarDtorCleanup(IRState*, VarDeclaration*) + 404
5  ldc2                     0x0000000102ed7810 (anonymous namespace)::pushVarDtorCleanup(IRState*, VarDeclaration*) + 404
6  ldc2                     0x0000000102ebc990 ToElemVisitor::visit(DeclarationExp*) + 400
7  ldc2                     0x0000000102ec4b58 ToElemVisitor::visit(CommaExp*) + 236
8  ldc2                     0x0000000102ed0480 ToElemVisitor::visit(DotVarExp*) + 392
9  ldc2                     0x0000000102ed5dc8 ToElemVisitor::call(IRState*, CallExp*, llvm::Value*) + 1100
10 ldc2                     0x0000000102ed5474 toInPlaceConstruction(DLValue*, Expression*) + 696
11 ldc2                     0x0000000102ecc278 ToElemVisitor::visit(AssignExp*) + 1532
12 ldc2                     0x0000000102e81f28 DtoVarDeclaration(VarDeclaration*) + 1296
13 ldc2                     0x0000000102e821d0 DtoDeclarationExp(Dsymbol*) + 332
14 ldc2                     0x0000000102ebc94c ToElemVisitor::visit(DeclarationExp*) + 332
15 ldc2                     0x0000000102eadf4c ToIRVisitor::visit(ExpStatement*) + 480
16 ldc2                     0x0000000102eae20c ToIRVisitor::visit(CompoundStatement*) + 348
17 ldc2                     0x0000000102eae20c ToIRVisitor::visit(CompoundStatement*) + 348
18 ldc2                     0x0000000102e78e7c DtoDefineFunction(FuncDeclaration*, bool) + 13452
19 ldc2                     0x0000000102e5ecb0 CodegenVisitor::visit(TemplateInstance*) + 288
20 ldc2                     0x0000000102e8886c codegenModule(IRState*, Module*) + 3320
21 ldc2                     0x0000000102ef65d8 ldc::CodeGenerator::emit(Module*) + 580
22 ldc2                     0x0000000102f0f5b0 codegenModules(Array<Module*>&) + 668
23 ldc2                     0x0000000102e2c2ec mars_mainBody(Param&, Array<char const*>&, Array<char const*>&) + 16712
24 ldc2                     0x0000000102fcafec _D2rt6dmain212_d_run_main2UAAamPUQgZiZ6runAllMFZv + 120
25 ldc2                     0x0000000102fcac4c _d_run_main + 148
26 ldc2                     0x0000000102f0a1b8 main + 824
27 libdyld.dylib            0x0000000189f74f54 start + 4
zsh: segmentation fault  ldc2 main.d
ponce@ponces-Mac repro % 

For example, this generates the ICE:

import std.stdio;
int main(string [] args) // build with: ldc2 main.d
{
    writeln("hello\n");
    return 0;
}

This doesn't generate the ICE:

int main(string [] args) // build with: ldc2 main.d
{
    return 0;
}

This doesn't generate the ICE either (and when run it works correctly):

import core.stdc.stdio;
int main(string [] args) // build with: ldc2 main.d
{
    printf("hello\n"); // Output: hello
    return 0;
}

When there is the ICE, it will also crash when using -mtriple x86_64-apple-macos main.d.

[p0nce]

Sounds related to exceptions?

Without throwing, this code compiles. But ICE when there is a throw.

int main(string [] args)
{
    try
    {
         // Uncomment to have the ICE
         //throw new Exception("lol");
    }
    catch(Exception e)
    {}
    return 0;
}

[kinke]

Thx for testing; please give https://github.com/ldc-developers/ldc/releases/download/CI/ldc2-f26956a4-osx-arm64.tar.xz a shot.

[p0nce]

yes, I've tested https://github.com/ldc-developers/ldc/releases/download/CI/ldc2-f26956a4-osx-arm64.tar.xz

• it works well for arm64, passing intel-intrinsics, etc
• it is very quick on the DTK, probably even more on M1. A full debug build of any of our product is 7 sec, with 20 dub package, vs 12.15s previously => native LDC takes 38% less time to build than under Rosetta.
• it cannot cross-compile to x86_64

With any main.d

ldc2 -mtriple x86_64-apple-macos main.d
ld: warning: ignoring file main.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
Undefined symbols for architecture arm64:
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture arm64

[kinke]

Alright perfect, so the problem was somehow really that the LDC assertions were enabled, while the linked LLVM was built with disabled assertions.

Not being able to cross-compile to x64 is expected, I haven't added those libs. (Expecting the user to copy the libs dir from the osx-x86_64 artifact and setting up ldc2.conf manually according to the regular cross-compilation steps on the Wiki page.)

Wrt. performance comparison with Rosetta, what x64 package have you used? CI builds of osx-x86_64 and osx-arm64 cannot be compared because the former use enabled assertions, the latter don't.

[p0nce]

I think it was LDC-1.24-beta1, not sure at all. Was a CI build.

[kinke]

[I should have been more precise, I meant master builds (from the CI LDC release), not tagged versions like a beta1. The comparison may have been between v1.24 and v1.25 but that probably doesn't really matter from a compile-time perspective.]

[p0nce]

For us on Apple hardware going from Rosetta to native code brings about 30% to 90% speed-up.

[kinke]

I assume there's still the myriad of linker warnings?

[p0nce]

Yes.