Crosscompile NativeLink #1233
Crosscompile NativeLink #1233
Conversation
aaronmondal
assigned adam-singer, allada, blakehatch, caass, MarcusSorealheis and zbirenbaum
There was a problem hiding this comment.
+@adam-singer +@allada +@zbirenbaum +@caass +@blakehatch +@MarcusSorealheis
It's nontrivial to add this change to CI, so I'll defer this to another PR. Until then, it would be very helpful if you could test the following:
# The new container which should now also work on MacOS
nix run .#nativelink-image.copyToDockerDaemon
docker run nativelink:$(nix eval .#nativelink-image.imageTag --raw)The 01_ and 02_ scripts for the K8s example should work now. On mac they should still fail at the bazel invocation, but I'll defer that to a future PR.
You can view the architecture of the crosscompiled executables like so:
nix build .#nativelink-aarch64-darwin
file result/bin/nativelink
nix build .#nativelink-aarch64-linux
file result/bin/nativelink
nix build .#nativelink-x86_64-darwin
file result/bin/nativelink
nix build .#nativelink-x86_64-linux
file result/bin/nativelinkIt's ok if a target gives you a nix error like Unsupported system, but this should occur "immediately" and not try to build in the first place. That is, you should not see compiler or linker errors.
Reviewable status: 0 of 6 LGTMs obtained, and 0 of 3 files reviewed, and pending CI: Analyze (javascript-typescript), Analyze (python), Bazel Dev / ubuntu-22.04, Cargo Dev / macos-13, Cargo Dev / ubuntu-22.04, Installation / macos-13, Installation / macos-14, Installation / ubuntu-22.04, Local / ubuntu-22.04, Publish image, Publish nativelink-worker-init, Publish nativelink-worker-lre-cc, Remote / large-ubuntu-22.04, asan / ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04), macos-13, pre-commit-checks, ubuntu-20.04 / stable, ubuntu-22.04, ubuntu-22.04 / stable, vale, windows-2022 / stable (waiting on @adam-singer, @allada, @blakehatch, @caass, @MarcusSorealheis, and @zbirenbaum)
aaronmondal
force-pushed
the
crosscompile
branch
2 times, most recently
from
debce25 to
e398ebe
Compare
There was a problem hiding this comment.
nit typo in ask above nix run .#nativelink-image.copyToDockerDaemon
Reviewable status: 0 of 6 LGTMs obtained, and 0 of 3 files reviewed, and pending CI: Installation / ubuntu-22.04, Remote / large-ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04) (waiting on @allada, @blakehatch, @caass, @MarcusSorealheis, and @zbirenbaum)
There was a problem hiding this comment.
Reviewable status: 0 of 6 LGTMs obtained, and 0 of 3 files reviewed, and pending CI: Installation / ubuntu-22.04, Remote / large-ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), and 1 discussions need to be resolved (waiting on @adam-singer, @blakehatch, @caass, @MarcusSorealheis, and @zbirenbaum)
native-cli/components/embedded/nix2container-image-info.yaml line 60 at r1 (raw file):
fi if [ "$(uname -m)" = "x86_64" ] && [ "$(uname -s)" = "Linux" ]; then
Can we document here (and in other file) why we only care about doing this on x86 and not arm linux distros?
There was a problem hiding this comment.
Those do work, but on linux I cannot build for mac.
Reviewable status: 0 of 6 LGTMs obtained, and 0 of 3 files reviewed, and pending CI: Installation / ubuntu-22.04, Remote / large-ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), and 1 discussions need to be resolved (waiting on @adam-singer, @blakehatch, @caass, @MarcusSorealheis, and @zbirenbaum)
There was a problem hiding this comment.
Reviewable status: 1 of 6 LGTMs obtained, and 0 of 3 files reviewed, and pending CI: Installation / ubuntu-22.04, Remote / large-ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), and 1 discussions need to be resolved (waiting on @adam-singer, @blakehatch, @caass, @MarcusSorealheis, and @zbirenbaum)
There was a problem hiding this comment.
Reviewed 3 of 3 files at r1, all commit messages.
Reviewable status: 1 of 6 LGTMs obtained, and all files reviewed, and pending CI: Installation / ubuntu-22.04, Remote / large-ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), and 1 discussions need to be resolved (waiting on @adam-singer, @blakehatch, @caass, @MarcusSorealheis, and @zbirenbaum)
aaronmondal
force-pushed
the
crosscompile
branch
from
e398ebe to
a03afe8
Compare
There was a problem hiding this comment.
-@adam-singer -@MarcusSorealheis -@caass -@zbirenbaum -@blakehatch
Reviewed 3 of 3 files at r2, all commit messages.
Reviewable status: 1 of 1 LGTMs obtained, and all files reviewed, and 1 discussions need to be resolved
native-cli/components/embedded/nix2container-image-info.yaml line 60 at r1 (raw file):
Previously, allada (Nathan (Blaise) Bruer) wrote…
Can we document here (and in other file) why we only care about doing this on x86 and not arm linux distros?
Done.
There was a problem hiding this comment.
Dismissed @allada from a discussion.
Reviewable status:complete! 1 of 1 LGTMs obtained, and all files reviewed
This rewrites the logic for the production-grade executables and images. The new setup allows both linux and mac systems to crosscompile NativeLink for `x86_64-linux` and `aarch64-linux`. The Kubernetes example and devcluster are now deployable natively on MacOS. Closes TraceMachina#751
aaronmondal
force-pushed
the
crosscompile
branch
from
a03afe8 to
1391b79
Compare
This rewrites the logic for the production-grade executables and images.
The new setup allows both linux and mac systems to crosscompile
NativeLink for
x86_64-linuxandaarch64-linux.The Kubernetes example and devcluster are now deployable natively on
MacOS.
Closes #751
This change is