WIP: add support for cross-compilation #152
Conversation
|
cc @Ericson2314 |
|
Is anyone going to review this? I'm happy to help weasel around back compat issues if that's what's needed, but I'd like someone to confirm that's indeed problem and to what extent, first. |
|
Is there an example of using this? I tried it out locally and get an error when it tries to link bootstrap. |
|
I have only used this to build some private packages, but I just tried to cross-compile If you remove I'm not sure where exactly it is failing for you. Could you post the command you are using as well as the end of the log? |
|
I realized I have a more complicated setup than I thought: I need a particular nightly, so I'm using the moz-overlay, and I'm compiling statically. It turns out the previous errors were caused by my inconsistent passing of cross packages and not using I tried providing overrides, which worked for dependency crates that have build.rs scripts, but not for the final one defined in the workspace: client = attrs: {
nativeBuildInputs = [
pkgs.buildPackages.glibc.static
];
buildInputs = [
pkgs.glibc.static
];
};Is this the right approach for providing dependencies to build scripts? |
|
I also see this error with crt-static which seems similar to this issue rust-lang/rust#71651. Maybe targetFeatures shouldn't be used for both proc macros and build scripts? |
|
I see:
I think you want a prefixed |
|
I think that's correct because the build script is being built for the x86 host architecture. |
|
Nevermind I think I made an assumption about the name "build_script_build.build_script_build". Not sure I'm right there. |
|
Oh I didn't see the horizontal scrollbar. Perhaps you can make a public reproduction of the issue so we could dive deeper? |
|
Sure. I'll put a repo together. |
|
Here it is https://github.com/joprice/rust-nix-test. You should be able to repro with nix-build. Let me know if you need more info. I also had some othre local workarounds I tried that I can add back in where I attempted to add static libc to the crates that were having issues, but it didn't seem to help much: hasBuildScript = attrs: attrs // {
nativeBuildInputs = (attrs.nativeBuildInputs or [ ]) ++ [ cross.buildPackages.glibc ];
buildInputs = (attrs.buildInputs or [ ]) ++ [ cross.glibc.static ];
extraRustcOpts = attrs.extraRustcOpts ++ extraRustcOpts;
};I then added that to crateOverrides for each dependency that was linking to glibc. Is there a way to do that more generally, or should that not be needed? |
|
Thanks! I'll be able to look tomorrow. |
|
I believe crate2nix is not actively maintained for some time. I just forked this for my own purposes and started addressing problems that matter to me in my fork. |
|
https://github.com/kolloch/crate2nix/commits/master Oh I see, the commits have been fairly rote as of late. |
|
Yeah, I had a conversation with Peter a few days ago and neither of us
are currently really looking after this project. I'll try to pickup some
of the slack in the next month. I'll be looking for a contract /
employer so that leave plenty of time to work on (F)OSS. :)
|
|
Hi @lopsided98, @Ericson2314, @nagisa, @joprice, Sorry, if there is still interest in this, I will find time this week (probably during the weekend). Just ping here. A real-time channel for chatting would be nice to resolve any issues near-time then. E.g. a video conference on Saturday evening (Zurich time). |
|
I can't speak for @lopsided98, but I would really like to see this be finished, and perhaps can do a call then. I'll have to catch up on exactly what's here too! |
|
I don't want to put any undue pressure as I'm currently only using it for hobby purposes, but I'm willing to pitch in if I can. |
|
My priorities shifted elsewhere recently, so I didn't get a chance to finish this. I'm available this weekend to work on this. This is my understanding of the known issues:
|
|
This is the error I'm running into while trying to cross-compile crate2nix: |
|
@lopsided98 Should have sent you contact details via the contact form on your webpage. Got the CI/CD running again and merge the PR that you are dependent on. |
| @@ -7,13 +7,14 @@ | |||
| , pkgs ? import nixpkgs { config = {}; } | |||
| , lib ? pkgs.lib | |||
| , stdenv ? pkgs.stdenv | |||
| , buildRustCrate ? pkgs.buildRustCrate | |||
There was a problem hiding this comment.
Is it necessary to remove this argument? If yes, why?
[I just like to keep top-level interface changes minimally, and for whatever reason I made this a top-level argument.]
There was a problem hiding this comment.
@kolloch see what @lopsided98 said in the PR description:
Unfortunately, I haven't figured out a way to do this without potentially breaking existing users, as well as the debugging tools. This is because there is no longer a single version of
buildRustCrateused by all crates, and instead there are separate ones for each platform. This makes it difficult to pass aroundbuildRustCrateFunc.
I can attest that this makes sense from the usual cross perspective.
There was a problem hiding this comment.
There was a problem hiding this comment.
and buildRustCrate could be deprecated and given a null default, so we can migrate with
| , buildRustCrate ? pkgs.buildRustCrate | |
| , buildRustCrate ? null | |
| , buildRustCrateFromPkgs ? if buildRustCrate != null then _: buildRustCrate else traceDeprecatedMessage (pkgs: pkgs.buildRustCrate) |
With traceDeprecatedMessage left as a bikeshed to the reader :).
| let | ||
| # proc_macro crates must be compiled for the build architecture |
There was a problem hiding this comment.
I love that you specify the reason here in a comment.
|
@kolloch ping me on IRC if you still want to work on this real time. |
|
The |
|
@joprice I was able to make a modified version of your repo build, albeit with stable Rust and musl. See below for the diff. rust-nix-test diffdiff --git a/common.nix b/common.nix
index a119809..750d9ae 100644
--- a/common.nix
+++ b/common.nix
@@ -1,46 +1,21 @@
let
mozOverlay = import (builtins.fetchTarball https://github.com/mozilla/nixpkgs-mozilla/archive/master.tar.gz);
- pkgs = (
- import <nixpkgs>
- {
- overlays = [
- mozOverlay
- (
- self: super: {
- rustChannel = self.rustChannelOf {
- rustToolchain = ./rust-toolchain;
- };
- cargo = self.rustChannel.rust;
- rustc = self.rustChannel.rust;
- }
- )
- ];
- }
- ).pkgsCross.armv7l-hf-multiplatform;
- alsaLibStatic = (pkgs.alsaLib.overrideAttrs
- (o: {
- configureFlags = [
- "--enable-shared=no"
- "--enable-static=yes"
- ];
- }));
+ pkgs = (import <nixpkgs> {}).pkgsCross.armv7l-hf-multiplatform.pkgsStatic;
crateOverrides = pkgs.defaultCrateOverrides // {
alsa-sys = attrs: attrs // {
nativeBuildInputs = (attrs.nativeBuildInputs or [ ]) ++ [
pkgs.buildPackages.pkg-config
];
- buildInputs = (attrs.buildInputs or [ ]) ++ [ alsaLibStatic ];
+ buildInputs = (attrs.buildInputs or [ ]) ++ [ pkgs.alsaLib ];
};
prost-build = attrs: attrs // {
PROTOC = "${pkgs.buildPackages.protobuf}/bin/protoc";
};
client = attrs: attrs // {
PROTOS_DIR = ./proto;
- buildInputs = [
+ nativeBuildInputs = [
# this is needed to format the generated grpc code
pkgs.buildPackages.rustfmt
- # TODO: add to all packages (expect proc macro and build scripts)?
- # pkgs.glibc.static
];
};
};
@@ -48,10 +23,6 @@ let
./Cargo.nix
{
release = false;
- # NOTE: cross building works with this disabled
- targetFeatures = [
- "crt-static"
- ];
defaultCrateOverrides = crateOverrides;
};
in
@@ -59,6 +30,6 @@ in
pkgs = pkgs;
build = cargo_nix.allWorkspaceMembers;
shell = pkgs.mkShell {
- nativeBuildInputs = [ pkgs.buildPackages.rustChannel.rust ];
+ nativeBuildInputs = [ pkgs.buildPackages.rustc ];
};
}I looked into using the Mozilla overlay, but I couldn't see how to make it use musl on armv7l. It looks like I don't think we really should be passing On the other hand, if we want to try to make |
|
I think that leaves only the backwards compatibility and debug tool breakage issues, which I still haven't figured out a good way to solve. |
|
I think I only had to use a specific nightly because it had some improved handling for static compilation. I'll try your changes on the original code and see if the remaining deps compile, and add them to the test repo if they don't. |
| @@ -2368,13 +2358,10 @@ rec { | |||
| target = target // { test = runTests; }; | |||
| } | |||
| ); | |||
| buildByPackageId = packageId: buildByPackageIdImpl packageId; | |||
|
|
|||
| # Memoize built packages so that reappearing packages are only built once. | |||
There was a problem hiding this comment.
Maybe we can do this again now that the hash thing has been changed in NixOS/nixpkgs#105305?
On the other hand, isn't the comment wrong? I would assume this is just saving eval time, and the derivations will be identical and thus deduplicated either way.
There was a problem hiding this comment.
My understanding was also that this was an eval optimization.
I don't think that PR affects this. packageId doesn't have anything to do with the hash; IIRC packageId is just the name and possibly the version if there are multiple. Even if you were to extend it to include the platform, it is not obvious which packages need to be built for which platforms until you walk through the dependency tree.
There was a problem hiding this comment.
OK fair point, then I think it might be nice to do the same corecursive stuff we do in nixpkgs?
let
buildByPackageIdForPkgsMemo = mkBuildByPackageIdForPkgsMemo pkgs;
mkBuildByPackageIdForPkgsMemo = pkgs: {
crates = lib.mapAttrs (packageId: value: buildByPackageIdForPkgs pkgs packageId) crateConfigs;
build = mkBuildByPackageIdForPkgsMemo pkgs.buildPackages;
};It's fine that this contains too much stuff, again cause lazy eval: a good time for space tradeoff.
You use crates for dependencies, build.crates for dev-dependencies, and a nasty backtracking of switching from crates to build.crates for any proc macro crate. (Shame on them for not making proc macros a different type of dependency! I did mention it when proc macros in Cargo first came out...)
|
@lopsided98 I tried your changes and got this error: I assume this fix relies on a change you did in nixpkgs as well, since using the latest commit of nixpkgs worked. |
|
@joprice Yes, you need NixOS/nixpkgs#105314 |
|
Works great for me (aarch64 -> armv6l). What more needs to be done to merge this? |
|
N.B. I am preparing an additional commit with my suggestions. |
|
OK, PR against the PR at https://github.com/lopsided98/crate2nix/pull/1. |
|
Heyo @Ericson2314, the test failure on linux looks legit. Feel free to ignore the one on Mac OS. |
|
I merged the follow up from @Ericson2314 . Thank you for this :) |
I would love if there was some doc in the README.md -- looks like you already collected some experience on how to use it. |
|
For reference, this was the follow up PR: #160 |
This adds support for cross-compiling using crate2nix to Linux targets. With this PR, passing
pkgs = pkgsCross.<platform>to the generatedCargo.nixwill cross-compile the crate.Unfortunately, I haven't figured out a way to do this without potentially breaking existing users, as well as the debugging tools. This is because there is no longer a single version of
buildRustCrateused by all crates, and instead there are separate ones for each platform. This makes it difficult to pass aroundbuildRustCrateFunc.This PR also includes #142 because I needed it to test some of my crates.