Build xcode-locator as a universal binary #14168

EdSchouten · 2021-10-26T11:34:47Z

One of Buildbarn's users is attempting to build it on a Mac M1 system
that does not have Rosetta installed:

This currently fails with the following error message:

ERROR: <storage>/external/com_google_protobuf/BUILD:130:11: Compiling src/google/protobuf/extension_set.cc failed: I/O exception during sandboxed execution: com.google.devtools.build.lib.shell.ExecFailedException: java.io.IOException: Cannot run program "<tmp>/install/71ed47cad951a20fff87381f54639763/xcode-locator": error=86, Bad CPU type in executable

Let's address this by shipping a copy of xcode-locator that is built
both for ARM64 and x86-64.

brentleyjones · 2021-10-26T16:17:09Z

Related, though different: #13452

keith · 2021-10-26T17:21:44Z

yea mine doesn't touch this tool, I was just allowing this one to run with rosetta for now

keith · 2021-10-26T17:23:09Z

This CI issue looks unrelated, I filed bazelbuild/continuous-integration#1269

keith · 2021-10-26T18:55:59Z

so this compile isn't hermetic because of an issue with linking on the M1, you have to use the same approach I did here to resolve that for now https://github.com/bazelbuild/bazel/pull/13452/files#diff-a974c428038be37758ceec95a81b78f4fd5cb7051e8d6c9e20c13a8337b0c8ddR112 by passing -Wl,-no_adhoc_codesign and then explicitly signing the binary. I think this will be fixed in a future version of the linker, but who knows when it will come

keith · 2021-10-26T18:56:05Z

That is why the test is failing here

EdSchouten · 2021-10-28T07:58:38Z

Interesting. I just added -Wl,-no_adhoc_codesign to DARWIN_XCODE_LOCATOR_COMPILE_COMMAND and the resulting executable is still not deterministic.

One of Buildbarn's users is attempting to build it on a Mac M1 system that does not have Rosetta installed: buildbarn/bb-remote-execution#89 This currently fails with the following error message: ERROR: <storage>/external/com_google_protobuf/BUILD:130:11: Compiling src/google/protobuf/extension_set.cc failed: I/O exception during sandboxed execution: com.google.devtools.build.lib.shell.ExecFailedException: java.io.IOException: Cannot run program "<tmp>/install/71ed47cad951a20fff87381f54639763/xcode-locator": error=86, Bad CPU type in executable Let's address this by shipping a copy of xcode-locator that is built both for ARM64 and x86-64.

EdSchouten · 2021-10-28T08:04:47Z

Ah! You also need -Wl,-no_uuid. Updated the PR to do separate codesigning. Thanks for your input, @keith !

keith · 2021-10-28T13:59:36Z

You shouldn't need that as well since the default is to produce a UUID based on the contents 🤔

keith · 2021-10-28T21:50:38Z

Ok I debugged that and verified it's a bug in ld64 that leads to non hermetic links. I filed FB9727658 with apple and also submitted swiftlang/llvm-project#3483 as a cherry pick to attempt to get the fix into Xcode sooner. But passing -no_uuid should be harmless for this use case for now anyways

keith · 2021-10-28T23:05:01Z

So the reason I hadn't reproduced this is because there was a difference in this behavior based on whether the -Wl,-flag came before or after the input source file name https://reviews.llvm.org/D112767

I think this current solution is the right move just to be safe

The content based uuid embedded by ld64 contains the basename of the file being built. In the case of multiarch builds this filename is randomly generated. This doesn't hit in all cases based on the order of the arguments to clang, but since this shouldn't have a downside for this use case, it's safer to exclude this for the future. More conversation bazelbuild#14168 Filed with apple as FB9727658

The content based uuid embedded by ld64 contains the basename of the file being built. In the case of multiarch builds this filename is randomly generated. This doesn't hit in all cases based on the order of the arguments to clang, but since this shouldn't have a downside for this use case, it's safer to exclude this for the future. More conversation bazelbuild#14168 Filed with apple as FB9727658 (cherry picked from commit d2aab87)

keith · 2021-10-28T23:10:41Z

@oquenchil can you review?

The content based uuid embedded by ld64 contains the basename of the file being built. In the case of multiarch builds this filename is randomly generated. This doesn't hit in all cases based on the order of the arguments to clang, but since this shouldn't have a downside for this use case, it's safer to exclude this for the future. More conversation #14168 Filed with apple as FB9727658 Closes #14190. PiperOrigin-RevId: 407784532

The content based uuid embedded by ld64 contains the basename of the file being built. In the case of multiarch builds this filename is randomly generated. This doesn't hit in all cases based on the order of the arguments to clang, but since this shouldn't have a downside for this use case, it's safer to exclude this for the future. More conversation #14168 Filed with apple as FB9727658 (cherry picked from commit d2aab87)

EdSchouten · 2021-11-26T14:09:26Z

Ping! Would be nice if this ended up in Bazel 5.

meteorcloudy · 2021-11-29T09:21:20Z

An internal test is failing with:

ERROR: No signing identity found for 'Apple Development: Google Development'

Can you help fixing this? Maybe related to --force --sign, is that necessary?

Update: the change submitted successfully, maybe it's a false alarm.

meteorcloudy · 2021-11-29T13:56:34Z

@EdSchouten Can you make a PR to release-5.0.0rc2?

One of Buildbarn's users is attempting to build it on a Mac M1 system that does not have Rosetta installed: buildbarn/bb-remote-execution#89 This currently fails with the following error message: ERROR: <storage>/external/com_google_protobuf/BUILD:130:11: Compiling src/google/protobuf/extension_set.cc failed: I/O exception during sandboxed execution: com.google.devtools.build.lib.shell.ExecFailedException: java.io.IOException: Cannot run program "<tmp>/install/71ed47cad951a20fff87381f54639763/xcode-locator": error=86, Bad CPU type in executable Let's address this by shipping a copy of xcode-locator that is built both for ARM64 and x86-64. Closes bazelbuild#14168. PiperOrigin-RevId: 412864310 (cherry picked from commit 76b3c24)

One of Buildbarn's users is attempting to build it on a Mac M1 system that does not have Rosetta installed: buildbarn/bb-remote-execution#89 This currently fails with the following error message: ERROR: <storage>/external/com_google_protobuf/BUILD:130:11: Compiling src/google/protobuf/extension_set.cc failed: I/O exception during sandboxed execution: com.google.devtools.build.lib.shell.ExecFailedException: java.io.IOException: Cannot run program "<tmp>/install/71ed47cad951a20fff87381f54639763/xcode-locator": error=86, Bad CPU type in executable Let's address this by shipping a copy of xcode-locator that is built both for ARM64 and x86-64. Closes #14168. PiperOrigin-RevId: 412864310 (cherry picked from commit 76b3c24) Co-authored-by: Ed Schouten <eschouten@apple.com>

meteorcloudy · 2021-12-01T10:23:00Z

Merged into release-5.0.0rc2

One of Buildbarn's users is attempting to build it on a Mac M1 system that does not have Rosetta installed: buildbarn/bb-remote-execution#89 This currently fails with the following error message: ERROR: <storage>/external/com_google_protobuf/BUILD:130:11: Compiling src/google/protobuf/extension_set.cc failed: I/O exception during sandboxed execution: com.google.devtools.build.lib.shell.ExecFailedException: java.io.IOException: Cannot run program "<tmp>/install/71ed47cad951a20fff87381f54639763/xcode-locator": error=86, Bad CPU type in executable Let's address this by shipping a copy of xcode-locator that is built both for ARM64 and x86-64. Closes bazelbuild#14168. PiperOrigin-RevId: 412864310

coeuvre · 2023-01-24T16:19:31Z

Just observed that the resulting executable is not deterministic with either Xcode 13.4 or 14.0.

keith · 2023-01-24T16:25:23Z

There's a known bug in the linker. It has been reported. I don't know of a workaround

google-cla bot added the cla: yes label Oct 26, 2021

EdSchouten requested a review from meteorcloudy October 26, 2021 11:35

steeve mentioned this pull request Oct 26, 2021

Consider implementing xcode-locator in osascript #14174

Closed

EdSchouten force-pushed the eschouten/20211026-xcode-locator branch from 980ab9e to 447f012 Compare October 28, 2021 08:04

keith approved these changes Oct 28, 2021

View reviewed changes

keith mentioned this pull request Oct 28, 2021

Add -no_uuid for hermetic macOS toolchain setup #14190

Closed

keith mentioned this pull request Oct 28, 2021

[5.x] Add -no_uuid for hermetic macOS toolchain setup #14191

Merged

oquenchil self-requested a review November 2, 2021 14:31

oquenchil self-assigned this Nov 2, 2021

oquenchil added platform: apple team-Rules-CPP Issues for C++ rules labels Nov 2, 2021

EdSchouten mentioned this pull request Nov 26, 2021

Release 5.0 - January 2022 #14013

Closed

9 tasks

meteorcloudy approved these changes Nov 29, 2021

View reviewed changes

meteorcloudy added this to the Bazel 5.0 Release Blockers milestone Nov 29, 2021

meteorcloudy added the release blocker label Nov 29, 2021

bazel-io closed this in 76b3c24 Nov 29, 2021

meteorcloudy reopened this Nov 29, 2021

brentleyjones mentioned this pull request Nov 30, 2021

Build xcode-locator as a universal binary #14351

Merged

meteorcloudy closed this Dec 1, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build xcode-locator as a universal binary #14168

Build xcode-locator as a universal binary #14168

EdSchouten commented Oct 26, 2021

brentleyjones commented Oct 26, 2021

keith commented Oct 26, 2021

keith commented Oct 26, 2021

keith commented Oct 26, 2021

keith commented Oct 26, 2021

EdSchouten commented Oct 28, 2021

EdSchouten commented Oct 28, 2021

keith commented Oct 28, 2021

keith commented Oct 28, 2021

keith commented Oct 28, 2021

keith commented Oct 28, 2021

EdSchouten commented Nov 26, 2021 •

edited

Loading

meteorcloudy commented Nov 29, 2021 •

edited

Loading

meteorcloudy commented Nov 29, 2021

meteorcloudy commented Dec 1, 2021

coeuvre commented Jan 24, 2023

keith commented Jan 24, 2023 via email •

edited

Loading

Build xcode-locator as a universal binary #14168

Build xcode-locator as a universal binary #14168

Conversation

EdSchouten commented Oct 26, 2021

brentleyjones commented Oct 26, 2021

keith commented Oct 26, 2021

keith commented Oct 26, 2021

keith commented Oct 26, 2021

keith commented Oct 26, 2021

EdSchouten commented Oct 28, 2021

EdSchouten commented Oct 28, 2021

keith commented Oct 28, 2021

keith commented Oct 28, 2021

keith commented Oct 28, 2021

keith commented Oct 28, 2021

EdSchouten commented Nov 26, 2021 • edited Loading

meteorcloudy commented Nov 29, 2021 • edited Loading

meteorcloudy commented Nov 29, 2021

meteorcloudy commented Dec 1, 2021

coeuvre commented Jan 24, 2023

keith commented Jan 24, 2023 via email • edited Loading

EdSchouten commented Nov 26, 2021 •

edited

Loading

meteorcloudy commented Nov 29, 2021 •

edited

Loading

keith commented Jan 24, 2023 via email •

edited

Loading