Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reintroducing mkRustCrate #31150

Closed
wants to merge 48 commits into from
Closed
Show file tree
Hide file tree
Changes from 41 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
a20704c
Reintroducing mkRustCrate
P-E-Meunier Nov 2, 2017
7979239
mkRustCrate in all-packages.nix
P-E-Meunier Nov 2, 2017
2d85124
Handling packages that don't have a "build=" in their Cargo.toml
P-E-Meunier Nov 3, 2017
00e93d5
Correction after review
P-E-Meunier Nov 3, 2017
4b5aa82
Handling build dependencies for the latest generate-nix-pkg
P-E-Meunier Nov 4, 2017
08046ce
Adding the tool "carnix", compiled with rust-utils.nix
P-E-Meunier Nov 8, 2017
6088db3
Replacing fetchzip with fetchcrate
P-E-Meunier Nov 10, 2017
235a48e
Carnix 0.4.1
P-E-Meunier Nov 10, 2017
6846126
Factorise regular expression
P-E-Meunier Nov 10, 2017
1917e32
Correction after reviews
P-E-Meunier Nov 11, 2017
655f44a
Carnix 0.4.3 (including calls to buildRustCrate instead of mkRustCrate)
P-E-Meunier Nov 11, 2017
cf4ec94
Documenting buildRustCrate
P-E-Meunier Nov 11, 2017
a694f61
Adding myself in maintainers
P-E-Meunier Nov 11, 2017
d0a68ff
Run `cargo build` again after adding dependencies
P-E-Meunier Nov 12, 2017
ca81cad
Carnix 0.4.4
P-E-Meunier Nov 12, 2017
b4c5585
Carnix 0.4.5 + dependencies on sqlite and pkgconfig
P-E-Meunier Nov 12, 2017
4ce91ee
Replacing a rm $(find) with find | xargs rm
P-E-Meunier Nov 12, 2017
1f338c1
Carnix 0.4.6, which can now compile workspaces
P-E-Meunier Nov 14, 2017
4121d59
Overriding the Rust compiler now works.
P-E-Meunier Nov 15, 2017
428f66a
Carnix 0.4.7, updated for the latest Rust nightly
P-E-Meunier Nov 15, 2017
01645cf
rust-utils.nix: using the generic extension for shared objects (darwi…
P-E-Meunier Nov 15, 2017
2d2490a
Carnix 0.4.8, removing constants verbose and release (now overridable…
P-E-Meunier Nov 16, 2017
0387786
mkRustCrate Move rust libraries into a /rlibs dir
FlorentBecker Nov 17, 2017
6ffd5b4
mkRustCrate: correct transitive dependencies
FlorentBecker Nov 18, 2017
7b212b8
Replacing bash ifs with optionalString
P-E-Meunier Nov 18, 2017
f7bed04
runHook
P-E-Meunier Nov 18, 2017
95ed173
Merge branch 'mkRustCrate' of github.com:FlorentBecker/nixpkgs into m…
P-E-Meunier Nov 18, 2017
099e713
BuildInputs overrides
P-E-Meunier Nov 18, 2017
68b7867
Add CARGO_PKG_VERSION_* variables
P-E-Meunier Nov 20, 2017
50b984f
Merging FlorentBecker's recent refactoring
P-E-Meunier Nov 24, 2017
993d376
defaultCrateOverrides without { pkgs } argument
P-E-Meunier Nov 24, 2017
a99ef12
Crate name cannot contain '-'
P-E-Meunier Nov 27, 2017
7e6e3b5
Fixing the names of binaries containing a -
P-E-Meunier Nov 27, 2017
c82be14
Carnix 0.4.9
P-E-Meunier Nov 29, 2017
ea0cf4b
Correct SHA256 for carnix
P-E-Meunier Nov 29, 2017
4c788cc
Carnix 0.4.10
P-E-Meunier Nov 29, 2017
240fc52
Updates after the latest review
P-E-Meunier Nov 30, 2017
0e8ef4b
Fixing a typo
P-E-Meunier Nov 30, 2017
5555123
Carnix 0.4.13
P-E-Meunier Nov 30, 2017
457f9f5
Fixing the "[ integer" error when building binaries
P-E-Meunier Nov 30, 2017
c21a241
Documentation with examples that compile
P-E-Meunier Nov 30, 2017
59d16ef
Carnix 0.4.14
P-E-Meunier Nov 30, 2017
7d7fe21
Carnix src was wrong
P-E-Meunier Dec 1, 2017
fc84be3
Carnix 0.5, including a new way to resolve features
P-E-Meunier Dec 5, 2017
1cd1fca
Explain argument forwarding
P-E-Meunier Dec 11, 2017
cd0f359
rename rust-utils to build-rust-crate
Mic92 Dec 12, 2017
bf70097
rename defaultCrateOverrides to default-crate-overrides
Mic92 Dec 12, 2017
f84ad34
doc/rust: update documentation to reflect latest carnix
Mic92 Dec 12, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 168 additions & 1 deletion doc/languages-frameworks/rust.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ For daily builds (beta and nightly) use either rustup from
nixpkgs or use the [Rust nightlies
overlay](#using-the-rust-nightlies-overlay).

## Packaging Rust applications
## Compiling Rust applications with Cargo

Rust applications are packaged by using the `buildRustPackage` helper from `rustPlatform`:

Expand Down Expand Up @@ -57,6 +57,173 @@ checksum can be then take from the failed build.
To install crates with nix there is also an experimental project called
[nixcrates](https://github.com/fractalide/nixcrates).

## Compiling Rust crates using Nix instead of Cargo

When run, `cargo build` produces a file called `Cargo.lock`,
containing pinned versions of all dependencies. Nixpkgs contains a
tool called `carnix` (`nix-env -iA nixos.carnix`), which can be used
to turn a `Cargo.lock` into a Nix expression.

That Nix expression calls `rustc` directly (hence bypassing Cargo),
and can be used to compile a crate and all its dependencies. Here is
an example for a minimal `hello` crate:


$ cargo new hello
$ cd hello
$ cargo build
Compiling hello v0.1.0 (file:///tmp/hello)
Finished dev [unoptimized + debuginfo] target(s) in 0.20 secs
$ carnix -o hello.nix --src ./. Cargo.lock --standalone
$ nix-build hello.nix
Copy link
Contributor

@boxofrox boxofrox Nov 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nix-build hello.nix no longer works, since carnix replaced the with import <nixpkgs>{}; preamble.

Also, this command would build all crates in hello.nix which might lead to errors if, for example, a crate like redox_syscall is pulled in as a conditional dependency of the time crate for the Redox-OS platform when building on Linux or any platform that is not Redox-OS.

i.e.

let
  ...
in
  ...
  redox_syscall_0_1_18 = redox_syscall_0_1_18_ {};
  time_0_1_37 = time_0_1_37_ {
    dependencies = [ libc_0_2_24 ]
      ++ (if buildPlatform.parsed.kernel.name == "redox" then [ redox_syscall_0_1_18 ] else [])
      ++ (if buildPlatform.parsed.kernel.name == "windows" then [ kernel32_sys_0_2_2 winapi_0_2_8 ] else []);
  };

I would recommend an example that kicks off the build by specifying the "root" Rust project only.

e.g.

nix-build -E 'let pkgs = import <nixpkgs> {}; in (import ./hello.nix { inherit pkgs }).hello_0_1_0'

I think that example expression will work. Corrections and improvements welcome. :/

Anyway, it'll be subject to change if @Mic92's recommendation to replace {pkgs} to improve composability is applied.


Now, the file produced by the call to `carnix`, called `hello.nix`, looks like:

```
with import <nixpkgs> {};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a reminder to update all with import <nixpkgs>{}; in the docs with the final preamble selected for carnix.

let release = true;
verbose = true;
hello_0_1_0_ = { dependencies?[], buildDependencies?[], features?[] }: buildRustCrate {
crateName = "hello";
version = "0.1.0";
src = ./.;
inherit dependencies buildDependencies features release verbose;
};

in
rec {
hello_0_1_0 = hello_0_1_0_ {};
}
```

In particular, note that the argument given as `--src` is copied
verbatim to the source. If we look at a more complicated
dependencies, for instance by adding a single line `libc="*"` to our
`Cargo.toml`, we first need to run `cargo build` to update the
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to use cargo update instead?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, this might update dependencies, which should not be updated.

`Cargo.lock`. Then, `carnix` needs to be run again, and produces the
following nix file:

```
with import <nixpkgs> {};
let release = true;
verbose = true;
hello_0_1_0_ = { dependencies?[], buildDependencies?[], features?[] }: buildRustCrate {
crateName = "hello";
version = "0.1.0";
src = ./.;
inherit dependencies buildDependencies features release verbose;
};
libc_0_2_33_ = { dependencies?[], buildDependencies?[], features?[] }: buildRustCrate {
crateName = "libc";
version = "0.2.33";
sha256 = "1l7synziccnvarsq2kk22vps720ih6chmn016bhr2bq54hblbnl1";
inherit dependencies buildDependencies features release verbose;
};

in
rec {
hello_0_1_0 = hello_0_1_0_ {
dependencies = [ libc_0_2_33 ];
};
libc_0_2_33 = libc_0_2_33_ {
features = [ "use_std" ];
};
}
```

Here, the `libc` crate has no `src` attribute, so `buildRustCrate`
will fetch it from [crates.io](https://crates.io). A `sha256`
attribute is still needed for Nix purity.

Some crates require external libraries. For crates from
[crates.io](https://crates.io), such libraries can be specified in
`<nixpkgs/pkgs/build-support/rust/defaultCrateOverrides.nix>`.

Starting from that file, one can add more overrides, to add features
or build inputs, as follows:

```
with import <nixpkgs> {};
let kernel = buildPlatform.parsed.kernel.name;
abi = buildPlatform.parsed.abi.name;
helo_0_1_0_ = { dependencies?[], buildDependencies?[], features?[] }: buildRustCrate {
crateName = "helo";
version = "0.1.0";
authors = [ "pe@pijul.org <pe@pijul.org>" ];
src = ./.;
inherit dependencies buildDependencies features;
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example might also help people:

 imaginary-timezone-crate = attrs: lib.optionalAttrs (lib.versionAtLeast attrs.version "1.0")  {
   postPatch = ''
     substituteInPlace lib/zoneinfo.rs \
       --replace "/usr/share/zoneinfo" "${tzdata}/share/zoneinfo"
   '';
 };

in
rec {
helo_0_1_0 = (helo_0_1_0_ {}).override {
crateOverrides = defaultCrateOverrides // {
helo = attrs: { buildInputs = [ openssl ]; };
};
};
}
```

Here, `crateOverrides` is expected to be a attribute set, where the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: "an attribute set"

key is the crate name without version number and the value a function.
The function gets all attributes passed to `buildRustCrate` as first
argument and returns a set that contains all attribute that should be
overwritten.

For more complicated cases, such as when parts of the crate's
derivation depend on the the crate's version, the `attrs` argument of
the override above can be read, as in the following example, which
patches the derivation:

```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would some more information about overrides after the example:

`crateOverrides` is expected to be a attribute set, 
where the key is the crate name without version number and the value a function.
The function gets all attributes passed to `buildRustCrate` as first argument
and returns a set that contains all attribute that should be overwritten.

with import <nixpkgs> {};
let kernel = buildPlatform.parsed.kernel.name;
abi = buildPlatform.parsed.abi.name;
helo_0_1_0_ = { dependencies?[], buildDependencies?[], features?[] }: buildRustCrate {
crateName = "helo";
version = "0.1.0";
authors = [ "pe@pijul.org <pe@pijul.org>" ];
src = ./.;
inherit dependencies buildDependencies features;
};
in
rec {
helo_0_1_0 = (helo_0_1_0_ {}).override {
crateOverrides = defaultCrateOverrides // {
helo = attrs: lib.optionalAttrs (lib.versionAtLeast attrs.version "1.0") {
postPatch = ''
substituteInPlace lib/zoneinfo.rs \
--replace "/usr/share/zoneinfo" "${tzdata}/share/zoneinfo"
'';
};
};
};
}
```


Three more parameters can be overridden:

- The version of rustc used to compile the crate:

```
hello_0_1_0.override { rust = pkgs.rust; };
```

- Whether to build in release mode or debug mode (release mode by
default):

```
hello_0_1_0.override { release = false; };
```

- Whether to print the commands sent to rustc when building
(equivalent to `--verbose` in cargo:

```
hello_0_1_0.override { verbose = false; };
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where should users go to find the full collection of attrs that can be passed?



## Using the Rust nightlies overlay

Mozilla provides an overlay for nixpkgs to bring a nightly version of Rust into scope.
Expand Down
1 change: 1 addition & 0 deletions lib/maintainers.nix
Original file line number Diff line number Diff line change
Expand Up @@ -493,6 +493,7 @@
plcplc = "Philip Lykke Carlsen <plcplc@gmail.com>";
plumps = "Maksim Bronsky <maks.bronsky@web.de";
pmahoney = "Patrick Mahoney <pat@polycrystal.org>";
pmeunier = "Pierre-Étienne Meunier <pierre-etienne.meunier@inria.fr>";
pmiddend = "Philipp Middendorf <pmidden@secure.mailbox.org>";
polyrod = "Maurizio Di Pietro <dc1mdp@gmail.com>";
pradeepchhetri = "Pradeep Chhetri <pradeep.chhetri89@gmail.com>";
Expand Down
Loading