Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: replace everything with dream2nix #56

Merged
merged 83 commits into from
Mar 5, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
3b29e2e
feat: replace everything with dream2nix
yusdacra Feb 4, 2022
45f92f3
fix: cc = cCompiler for stdenv
yusdacra Feb 4, 2022
051cd67
refactor: use runCommandLocal for desktop file link
yusdacra Feb 13, 2022
01d6414
refactor: flatten packagesRaw since it's only an intermediate set
yusdacra Feb 13, 2022
50b807f
tests: run tests seperately
yusdacra Feb 13, 2022
a964597
feat: use crane builder for dream2nix
yusdacra Feb 13, 2022
3f284b7
fix: add rustc and cargo in base drv
yusdacra Feb 15, 2022
ed4a1bf
ci: keep going in tests
yusdacra Feb 15, 2022
e9499b1
feat: use buildRustPackage
yusdacra Feb 15, 2022
e975d1e
chore: update flake deps
yusdacra Feb 15, 2022
05ebf35
fix: make overrides actually work
yusdacra Feb 15, 2022
3723946
tests: ignore broken folder
yusdacra Feb 15, 2022
56d8c76
fix: actually use passed features and release and package flags
yusdacra Feb 15, 2022
fe22281
chore: update flake deps
yusdacra Feb 15, 2022
dbc80a7
refactor: minor changes
yusdacra Feb 16, 2022
94c0eeb
refactor: seperate override collected deps and envs from cargo metada…
yusdacra Feb 16, 2022
704bfc2
refactor: use a shortened name for lib in most places, refactor some …
yusdacra Feb 16, 2022
c6c66a1
refactor: move most attributes under internal in common.nix, remove m…
yusdacra Feb 16, 2022
952f162
fix: use lib from interal in makeOutput
yusdacra Feb 16, 2022
603d1e3
docs(manual): remove mentions of buildRustPackage, mention dream2nix …
yusdacra Feb 16, 2022
d1485a6
fix: remove the infinite recursion error
yusdacra Feb 16, 2022
49d0f07
feat: add dbgPrint func, add more dbg traces
yusdacra Feb 17, 2022
0b73bec
fix: possible infinite recursion?
yusdacra Feb 17, 2022
6160858
feat: make dbg print prettier
yusdacra Feb 17, 2022
35892ce
chore: add cli license to cargo.toml
yusdacra Feb 17, 2022
0fdc437
fix: make override diffs just use toJSOn
yusdacra Feb 17, 2022
b342b9c
feat: add traces to build override diffs
yusdacra Feb 17, 2022
59b8274
feat: use dbgPrint for printing override diffs
yusdacra Feb 17, 2022
9b5c0bc
fix: actual pretty printed traces
yusdacra Feb 17, 2022
c0bf1b9
fix: properly apply override while collecting
yusdacra Feb 17, 2022
dfe0f5e
refactor: add applyOverrides function
yusdacra Feb 17, 2022
92a7b12
docs(manual): remove stale link
yusdacra Feb 17, 2022
8ec3732
refactor: move build overrides to craneOverrides to prepare for more …
yusdacra Feb 17, 2022
1b3ff7b
refactor: dont create our own pkgs sets, properly seperate pkgs-lib a…
yusdacra Feb 18, 2022
5123b75
refactor: minor nits
yusdacra Feb 18, 2022
8241f06
refactor: move create nixpkgs drv to seperate file
yusdacra Feb 18, 2022
e34b15a
docs: fix makeCrateOverrides comment
yusdacra Feb 18, 2022
720f8b9
style: format everything with alejandra
yusdacra Feb 18, 2022
550ccd8
docs: fix example flake
yusdacra Feb 18, 2022
3556f57
fix: properly apply rust overlay
yusdacra Feb 18, 2022
0bc55d8
fix: import filename
yusdacra Feb 18, 2022
e846c01
fix: variable name typo
yusdacra Feb 18, 2022
4ba9f0c
fix: imports in createNixpkgsDrv
yusdacra Feb 18, 2022
5af1089
chore: update flake deps
yusdacra Feb 19, 2022
8d4ca30
tests: fix tests
yusdacra Feb 19, 2022
5bcea54
fix: import devshell in pkgs-set
yusdacra Feb 19, 2022
6e055d0
fix: manually write out extraModulesPath
yusdacra Feb 19, 2022
3618834
fix: pkgs.devshell -> makeDevshell
yusdacra Feb 19, 2022
8b54010
fix: use importTOML from proper import path
yusdacra Feb 19, 2022
e425fa8
chore: update flake deps
yusdacra Feb 19, 2022
bce530e
feat: allow -deps as an override
yusdacra Feb 19, 2022
01c5a4d
chore: update flake deps
yusdacra Feb 19, 2022
94c3071
fix: write build log for crane
yusdacra Feb 19, 2022
2bfe06d
fix: dont write build log on deps only drv
yusdacra Feb 19, 2022
a0124a4
feat: use buildRustPackage dream2nix builder
yusdacra Feb 20, 2022
73c6a0a
chore: use main dream2nix
yusdacra Feb 20, 2022
e4e7b94
feat: add crane overrides back
yusdacra Feb 27, 2022
0302030
chore: make dream2nix nixpkgs follow our nixpkgs
yusdacra Feb 27, 2022
9f94cc4
fix: set dream2nix projectRoot
yusdacra Feb 27, 2022
23be282
fix: set pname for dream2nix makeFlakeOutputs
yusdacra Feb 27, 2022
2dbf921
chore(deps): update flake deps
yusdacra Feb 27, 2022
37849a5
fix: try to overlay cargo hooks so buildRustPackage uses correct tool…
yusdacra Feb 27, 2022
b0da8d4
feat: enable choosing dream2nix builder
yusdacra Feb 27, 2022
ab99cb9
chore(deps): update flake deps
yusdacra Feb 27, 2022
0089d5e
ci: test both builders
yusdacra Feb 27, 2022
bbfb34e
fix: make dream2nix use our pkgs
yusdacra Feb 27, 2022
65075d2
chore: switch to main dream2nix
yusdacra Feb 28, 2022
5c1f7ba
ci: disable fail fast for builder tests matrix
yusdacra Feb 28, 2022
ae4d2ee
chore(deps): update flake deps
yusdacra Mar 5, 2022
3ba219e
feat: replace pkgs override with pkgsOverlays
yusdacra Mar 5, 2022
2a50a8c
refactor: introduce concatAttrsList to replace some code, minor reada…
yusdacra Mar 5, 2022
daec7d0
style: formatting
yusdacra Mar 5, 2022
ef6623a
refactor: introduce thenOr and thenOrNull to make some expressions mo…
yusdacra Mar 5, 2022
8daaefe
feat: use alejandra instead of nixpkgs-fmt in shell
yusdacra Mar 5, 2022
1487625
chore(deps): update flake deps
yusdacra Mar 5, 2022
693d2f1
fix: overlay rust toolchain properly, expose pkgsWithRust
yusdacra Mar 5, 2022
fe39903
fix: prevRoot -> root
yusdacra Mar 5, 2022
6c9a45a
fix: concatAttrsList -> concatAttrLists
yusdacra Mar 5, 2022
8624c54
fix: pass --workspace to crane deps only drv, dont pass --package
yusdacra Mar 5, 2022
8433f5a
docs: update manual and example flake to reflect latest changes
yusdacra Mar 5, 2022
725d739
docs: document builder attribute of makeOutputs
yusdacra Mar 5, 2022
3d9b355
chore(deps): update flake deps
yusdacra Mar 5, 2022
9800c32
Revert "fix: pass --workspace to crane deps only drv, dont pass --pac…
yusdacra Mar 5, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 15 additions & 46 deletions .github/workflows/nix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,15 @@ name: "Nix"
on:
workflow_dispatch:
pull_request:
branches: [ master, release-1.0 ]
branches: [ master ]
push:
branches: [ master, release-1.0 ]
branches: [ master ]
jobs:
brp-tests:
tests:
strategy:
fail-fast: false
matrix:
builder: [crane, brp]
runs-on: ubuntu-latest
steps:
- name: Checkout repo
Expand All @@ -22,48 +26,13 @@ jobs:
with:
name: nix-cargo-integration
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Tests
run: nix build -L --show-trace --no-link --impure --expr 'builtins.removeAttrs (builtins.getFlake (toString ./.)).platformChecks.brp.x86_64-linux [ "preCommitChecks" ]'

naersk-tests:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v2
- name: Install nix
uses: cachix/install-nix-action@v16
with:
extra_nix_config: |
experimental-features = nix-command flakes
nix_path: nixpkgs=channel:nixos-unstable
- name: Setup cachix
uses: cachix/cachix-action@v10
with:
name: nix-cargo-integration
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Tests
run: nix build -L --show-trace --no-link --impure --expr 'builtins.removeAttrs (builtins.getFlake (toString ./.)).platformChecks.naersk.x86_64-linux [ "preCommitChecks" ]'

crate2nix-tests:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v2
- name: Install nix
uses: cachix/install-nix-action@v16
with:
extra_nix_config: |
experimental-features = nix-command flakes
nix_path: nixpkgs=channel:nixos-unstable
- name: Setup cachix
uses: cachix/cachix-action@v10
with:
name: nix-cargo-integration
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Tests
run: nix build -L --show-trace --no-link --impure --expr 'builtins.removeAttrs (builtins.getFlake (toString ./.)).platformChecks.crate2nix.x86_64-linux [ "preCommitChecks" ]'

dream2nix-tests:
- name: Test checks
run: nix build -L --keep-going --show-trace --impure --expr '(builtins.getFlake (toString ./.)).${{ matrix.builder }}Tests.checks.${builtins.currentSystem}'
- name: Test shells
run: nix build -L --keep-going --show-trace --impure --expr '(builtins.getFlake (toString ./.)).${{ matrix.builder }}Tests.shells.${builtins.currentSystem}'
- name: Test packages
run: nix build -L --keep-going --show-trace --impure --expr '(builtins.getFlake (toString ./.)).${{ matrix.builder }}Tests.packages.${builtins.currentSystem}'
build-cli:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
Expand All @@ -80,4 +49,4 @@ jobs:
name: nix-cargo-integration
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Tests
run: nix build -L --show-trace --no-link --impure --expr 'builtins.removeAttrs (builtins.getFlake (toString ./.)).platformChecks.dream2nix.x86_64-linux [ "preCommitChecks" ]'
run: nix build .#nci-cli
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@ tests/**/target
.direnv
.envrc
cli/target
shell.nix
/shell.nix
.vscode
12 changes: 3 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,12 @@

Library to easily and effortlessly integrate Cargo projects with Nix.

- Uses [naersk], [crate2nix] or [buildRustPackage] to build Cargo packages and [devshell] to provide a development shell.
Allows changing between "build platforms" by just changing one attribute (see `buildPlatform`).
- Uses [dream2nix] to build Cargo packages and [devshell] to provide a development shell.
- Allows configuration from `Cargo.toml` file(s) via `package.metadata.nix` and `workspace.metadata.nix` attributes.
- Has sensible defaults, and strives to be compatible with Cargo (autobins, etc.).
- Aims to offload work from the user; comes with useful configuration options (like `renameOutputs`, `defaultOutputs` etc.)
- Can generate nixpkgs-compatible Nix expressions that captures all your packages dependencies / env vars and so on.
You don't need to maintain a seperate derivation for nixpkgs! (see `Generating a nixpkgs-compatible package expression` in manual)
- [naersk] and [crate2nix] `buildPlatform`s allow building packages directly from `Cargo.lock`. [buildRustPackage] uses the `cargoVendorHash` attribute.
- A CLI tool that let's you compile and run arbitrary Rust repositories directly without messing with any files or setting up overlays (see `Using the nci CLI` in manual)

## Usage
Expand Down Expand Up @@ -55,16 +53,12 @@ You can also couple it with [niv](https://github.com/nmattia/niv):
### Examples

- [Basic flake.nix template with commented fields and overrides](./docs/example_flake.nix)
- [crate2nix build platform crate overrides usage](https://gitlab.com/veloren/veloren/-/blob/master/flake.nix)
- [Modifying cargo build options to build a specific feature and changing outputs based on the feature used](https://github.com/yusdacra/bernbot/blob/master/flake.nix)
- [Flake using a bit of everything](https://github.com/harmony-development/Crust/blob/master/flake.nix)
- [Flake using a bit of everything](https://github.com/helix-editor/helix/blob/master/flake.nix)

## Documentation

You can find library API documentation and others at [the manual](./docs/MANUAL.md).

[devshell]: https://github.com/numtide/devshell "devshell"
[naersk]: https://github.com/nmattia/naersk "naersk"
[crate2nix]: https://github.com/kolloch/crate2nix "crate2nix"
[flake-compat]: https://github.com/edolstra/flake-compat "flake-compat"
[buildRustPackage]: https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/rust.section.md#compiling-rust-applications-with-cargo-compiling-rust-applications-with-cargo "buildRustPackage"
[dream2nix]: https://github.com/nix-community/dream2nix "dream2nix"
1 change: 1 addition & 0 deletions cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ name = "nci-cli"
version = "0.1.0"
edition = "2018"
authors = ["Yusuf Bera Ertan <y.bera003.06@protonmail.com>"]
license = "MIT"

[package.metadata.nix]
build = true
Expand Down
6 changes: 5 additions & 1 deletion cli/src/flk.nix
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,11 @@
nci.url = "path:nci_source";
};

outputs = { source, nci, ... }@inputs:
outputs = {
source,
nci,
...
} @ inputs:
nci.lib.makeOutputs {
root = source;
overrides = {
Expand Down
2 changes: 1 addition & 1 deletion default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ let
sha256 = "sha256:0x2jn3vrawwv9xp15674wjz9pixwjyj3j771izayl962zziivbx2";
};
in
(import compat { src = ./.; }).defaultNix.lib
(import compat {src = ./.;}).defaultNix.lib
48 changes: 13 additions & 35 deletions docs/MANUAL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Index

- [Library documentation](#library-documentation)
- [Build platform specific usage](#build-platform-specific-usage)
- [Generating a nixpkgs-compatible package expression](#generating-a-nixpkgs-compatible-package-expression)
- [Using the `nci` CLI](#using-the-nci-cli)
- [Enabling trace](#enabling-trace)
Expand All @@ -12,49 +11,36 @@
**IMPORTANT** public API promises: Any API that is not documented here **IS NOT** counted
as "public" and therefore can be changed without breaking SemVer. This does not mean that
changes will be done without any notice. You are still welcome to create issues / discussions
about them. Upstream projects such as [devshell], `nixpkgs` ([buildRustPackage]) etc. can have
breaking changes, these breakages will be limited to `overrides`' `shell` and `build` in the
public API, since these directly modify [devshell] / `buildPlatform` configs.

NOTE: `nix-cargo-integration` does not directly use upstream [naersk] / [crate2nix]. It uses
forks of them which contain additional fixes and features that aren't upstream.
about them. Upstream projects such as [devshell], [dream2nix] etc. can have breaking changes,
these breakages will be limited to `overrides.shell` and `overrides.build` in the public API,
since these directly modify [devshell] / [dream2nix] builder configs.

### The `common` attribute set

This attribute set is passed to (almost) all overrides. It contains everything you may
need for adding stuff, such as the nixpkgs package set (`common.pkgs`), library with
`nix-cargo-integration` utilities (`common.lib`) and other shared data between build
and development shell (`common.buildInputs`, `common.env` etc.). For more information
on what `common` actually exports, please check the bottom of [common.nix](./src/common.nix).
need for adding stuff, such as the nixpkgs package set (`common.pkgs`) and other shared
data between build and development shell (`common.buildInputs`, `common.env` etc.). For
more information on what `common` actually exports, please check the bottom of [common.nix](./src/common.nix).

### `makeOutputs`

Generates outputs for all systems specified in `Cargo.toml` (defaults to `defaultSystems` of `nixpkgs`).

#### Arguments

- `builder`: which dream2nix builder to use (type: string) (default: `"crane"`)
- `enablePreCommitHooks`: whether to enable pre-commit hooks (type: boolean) (default: `false`)
- `buildPlatform`: platform to build crates with (type: `"naersk", "crate2nix" or "buildRustPackage"`) (default: `"naersk"`)
- `root`: directory where `Cargo.lock` and `Cargo.toml` (workspace or pacakge manifest) is in (type: path)
- `cargoVendorHash`: vendor hash feeded into [buildRustPackage]'s `cargoSha256` (type: string) (default: `lib.fakeHash`)
- `useCrate2NixFromPkgs`: toggles using the `crate2nix` package from `nixpkgs` instead of the package in `crate2nix` source (type: boolean) (default: `false`)
- this can be helpful if you are facing frequent rebuilds of `crate2nix` (see https://github.com/yusdacra/nix-cargo-integration/issues/38)
- `root`: directory where `Cargo.lock` and `Cargo.toml` (workspace or package manifest) is in (type: path)
- `overrides`: overrides for devshell, build and common (type: attrset) (default: `{ }`)
- `overrides.systems`: mutate the list of systems to generate for (type: `def: [ ]`)
- `overrides.sources`: override for the sources used by common (type: `common: prev: { }`)
- `overrides.pkgs`: override for the configuration while importing nixpkgs in common (type: `common: prev: { }`)
- `overrides.pkgsOverlays`: overlays to apply to the nixpkgs package set (type: list of nixpkgs overlays)
- `overrides.crateOverrides`: override for crate overrides (type: `common: prev: { }`)
- with [crate2nix], this will allow you to override per-crate
- with [naersk], the overrides here will be collected and be used for
overriding the dependencies derivation / main derivation.
- with [buildRustPackage], the overrides here will be collected and
be used for overriding the resulting derivation.
- `overrides.common`: override for common (type: `prev: { }`)
- this will override *all* common attribute set(s), refer to [common.nix](./src/common.nix) for more information
- `overrides.shell`: override for devshell (type: `common: prev: { }`)
- this will override *all* [devshell] configuration(s), refer to [devshell] for more information
- `overrides.build`: override for build config (type: `common: prev: { }`)
- this will override [naersk]/[crate2nix]/[buildRustPackage] build config, refer to [naersk]/[crate2nix]/[buildRustPackage] for more information
- `renameOutputs`: which crates to rename in package names and output names (type: attrset) (default: `{ }`)
- `defaultOutputs`: which outputs to set as default (type: attrset) (default: `{ }`)
- `defaultOutputs.app`: app output name to set as default app (`defaultApp`) output (type: string)
Expand Down Expand Up @@ -131,12 +117,6 @@ Refer to [devshell] documentation.

NOTE: Attributes specified here **will not** be used if a top-level `devshell.toml` file exists.

## Build platform specific usage

- Disabling tests:
- for `crate2nix`: add `runTests = false;` to the `build` override.
- for `naersk` and `buildRustPackage`: add `doCheck = false;` to the `build` override.

## Generating a nixpkgs-compatible package expression

`nix-cargo-integration` will generate outputs named `<packageOutputName>-derivation`.
Expand Down Expand Up @@ -172,12 +152,6 @@ variable to `1` and passing `--impure` to `nix`. Example:
NCI_DEBUG=1 nix build --impure .
```

[devshell]: https://github.com/numtide/devshell "devshell"
[naersk]: https://github.com/nmattia/naersk "naersk"
[crate2nix]: https://github.com/kolloch/crate2nix "crate2nix"
[flake-compat]: https://github.com/edolstra/flake-compat "flake-compat"
[buildRustPackage]: https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/rust.section.md#compiling-rust-applications-with-cargo-compiling-rust-applications-with-cargo "buildRustPackage"

## Tips and tricks

### Ignoring `Cargo.lock` in Rust libraries
Expand All @@ -198,3 +172,7 @@ A neat fix for that is to track the path to `Cargo.lock` without staging it
$ git add --intend-to-add Cargo.lock
```
Add `--force` if your `Cargo.lock` is listed in `.gitignore`.

[devshell]: https://github.com/numtide/devshell "devshell"
[flake-compat]: https://github.com/edolstra/flake-compat "flake-compat"
[dream2nix]: https://github.com/nix-community/dream2nix "dream2nix"
Loading