x/build: add darwin builders with macOS 14 (macOS Sonoma)

[dmitshur]

macOS 14 (macOS Sonoma) was announced today during WWDC23 and is described to be "Coming this fall" (https://www.apple.com/macos/sonoma-preview/). Edit: It will be publicly released on September 26, 2023.

This is the tracking issue (that can be split into darwin/arm64 and darwin/amd64 ones if helpful) to eventually have builder coverage for it.

CC @golang/release, @prattmic, @golang/darwin.

[janartist]

command-line-arguments /usr/local/go/pkg/tool/darwin_arm64/link: running clang failed: exit status 1 ld: warning: '/private/var/folders/q_/0lwmyybd6293ltjj6y8p93pw0000gn/T/go-link-2066507266/go.o' has malformed LC_DYSYMTAB, expected 110 undefined symbols to start at index 53486, found 206 undefined symbols starting at index 167 0 0x100b08380 __assert_rtn + 72 1 0x100ac1b3c ___ZN2ld16LayoutExecutable27writeContentWithoutLinkEditENSt3__14spanIhLm18446744073709551615EEEy_block_invoke + 9936 2 0x183925950 _dispatch_client_callout2 + 20 3 0x18393a1a4 _dispatch_apply_invoke_and_wait + 176 4 0x183939464 _dispatch_apply_with_attr_f + 1176 5 0x183939650 dispatch_apply + 96 6 0x100ac1ce0 void mapReduce<ld::Atom const*, mach_o::Error>(std::__1::span<ld::Atom const*, 18446744073709551615ul>, unsigned long, void (unsigned long, mach_o::Error&, std::__1::span<ld::Atom const*, 18446744073709551615ul>) block_pointer, void (std::__1::span<mach_o::Error, 18446744073709551615ul>) block_pointer) + 336 7 0x100ac7718 ld::LayoutExecutable::writeToFile(char const*) + 16268 8 0x100a787d0 main + 8784 ld: Assertion failed: (0 && "Kind::arm64_adrp_ldr missing extra info"), function applyFixup, file Fixup.cpp, line 743. clang: error: linker command failed with exit code 1 (use -v to see invocation)

macos14 sonoma build error,Is there a version update?

[cherrymui]

@janartist not yet. Apple's new linker is still in beta and known to have bugs and incompatible changes. We're working on get them fixed. CL https://golang.org/cl/505415 and the stack contain a number of workarounds. That said, while I've seen a number of issues, I haven't seen this particular failure mode. This looks like a bug in the Apple linker (at least it should give a user-friendly error message instead of assertion failure.) Could you share what program you're trying to build? Thanks.

This issue is about setting up the builder. I'm going to file another issue for the linker issue.

[cherrymui]

Filed #61229 for the linker issue. You're welcome to post linker related failures there, or start a new issue. Thanks.

[cherrymui]

@janartist Could you share what program you're building to trigger the assertion failure? If we have a reproducer, we can report it to Apple and have the issue fixed. Thanks.

[thalesfsp]

Any update on this?

[heschi]

We are unlikely to add the builder until the final release is available.

[thalesfsp]

@heschi Any workaround? Tkx!

[heschi]

This issue tracks making an improvement to the infrastructure of the project, not a bug fix. You might be looking for #61229, or maybe you should file a new issue.

[minarc]

I updated my system to macOS 14.0 beta (23A5312d) today, and it seems like the issue has been resolved.

[momo5502]

command-line-arguments /usr/local/go/pkg/tool/darwin_arm64/link: running clang failed: exit status 1 ld: warning: '/private/var/folders/q_/0lwmyybd6293ltjj6y8p93pw0000gn/T/go-link-2066507266/go.o' has malformed LC_DYSYMTAB, expected 110 undefined symbols to start at index 53486, found 206 undefined symbols starting at index 167 0 0x100b08380 __assert_rtn + 72 1 0x100ac1b3c ___ZN2ld16LayoutExecutable27writeContentWithoutLinkEditENSt3__14spanIhLm18446744073709551615EEEy_block_invoke + 9936 2 0x183925950 _dispatch_client_callout2 + 20 3 0x18393a1a4 _dispatch_apply_invoke_and_wait + 176 4 0x183939464 _dispatch_apply_with_attr_f + 1176 5 0x183939650 dispatch_apply + 96 6 0x100ac1ce0 void mapReduce<ld::Atom const*, mach_o::Error>(std::__1::span<ld::Atom const*, 18446744073709551615ul>, unsigned long, void (unsigned long, mach_o::Error&, std::__1::span<ld::Atom const*, 18446744073709551615ul>) block_pointer, void (std::__1::span<mach_o::Error, 18446744073709551615ul>) block_pointer) + 336 7 0x100ac7718 ld::LayoutExecutable::writeToFile(char const*) + 16268 8 0x100a787d0 main + 8784 ld: Assertion failed: (0 && "Kind::arm64_adrp_ldr missing extra info"), function applyFixup, file Fixup.cpp, line 743. clang: error: linker command failed with exit code 1 (use -v to see invocation)

macos14 sonoma build error,Is there a version update?

The assertion Kind::arm64_adrp_ldr missing extra info comes when ADRP+LDR pairs are too far spread apart.
I think this is an extension to apple's chained fixup relocation format where offsets to the LDR instruction are encoded within the ADRP instruction to be later relocated at runtime. If the pair is 0x100 instructions apart (or more, as the offset is a byte), it can not be encoded anymore and this assertion is thrown.
I think this requires either a fix in the linker or, without switching to ld classic, it can only be mitigated by emitting the ADRP+LDR pairs closely together in the compiler. (this is just my best guess from reversing the linker. I might be wrong)

[dmitshur]

We've added a macOS 14 builder on the LUCI side:

https://ci.chromium.org/ui/p/golang/builders/ci/gotip-darwin-amd64_14

At this point, given we're working towards migrating to LUCI and macOS builders supply is more constrained than other builder types, we're not going to add it to the previous infrastructure.