nix/haskell-packages: replace IFD with importing generated files #1220
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
basvandijk
force-pushed
the
basvandijk/get-rid-of-IFD
branch
3 times, most recently
from
3e5a156
to
6c5b5ea
Compare
nmattia
approved these changes
Feb 19, 2020
nomeata
approved these changes
Feb 19, 2020
There was a problem hiding this comment.
Nice solution!
I wonder if it would be nicer to keep the generated files all in
./nix/generated/
(e.g. ./nix/generated/winter.nix), have one command to update all generated files, and only one check, and not require an essentially empty winter/ directory. But probably overkill.
Maybe add a section in Building.md about Updating Haskell dependencies?
I opted for keeping the
Good idea. Done. |
Replace IFD calls to callCabal2nix with importing generated `default.nix` files. The generated `default.nix` files are stored in the repository and contain instructions on how to regenerate them. The generation of the `default.nix` files is done using the `haskellSrc2nixWithDoc` function which generates a directory containing a `default.nix` which is the result of running `cabal2nix` with the `extraCabal2nixOptions` on the provided `src`. A header is then added to the `default.nix` which contains instructions on how to regenerate that file. Finally the new jobs are added which check that the stored `default.nix` matches the expected `default.nix`. So whenever the `.cabal` files are updated and don't match the stored `default.nix` anymore CI will complain.
basvandijk
force-pushed
the
basvandijk/get-rid-of-IFD
branch
from
6b482cc
to
ff94efb
Compare
|
Ugh, now I get this. Any idea? |
|
Ah, it somehow broke the proper use of |
is of course wrong. Will refactor… |
nomeata
added a commit
that referenced
this pull request
Feb 20, 2020
the reiterates on #1220. The motivation for the refactoring was that the generated files were not using `gitSource`, so I would get cache misses and dirty files in the repo. Fixing this was easier if all generated nix files are in one place, so I did the following changes: * All generated nix files are in `nix/generated`. This de-pollutes the normal working area with generated files, and keeps them out from sources (more precise derivations). * The local generated files now use `gitSource`. * The generator is now `nix/generate.nix`. * I moved complex logic from `nix/default.nix` to `nix/generate.nix`, this means everything is in one place, instead of spread over several nix files. * There is now only a single check to check if the generated files are upto date, and a single command to update the files. * This check is no longer marked `allowSubstitutes = false`. The check isn’t cheap (it requires `cabal2nix`) and we want the check to happen on builders, not hydra itself, so that the cache has `cabal2nix`. * The check is now included in `all-systems-go`.
mergify bot
pushed a commit
that referenced
this pull request
Feb 20, 2020
the reiterates on #1220. The motivation for the refactoring was that the generated files were not using `gitSource`, so I would get cache misses and dirty files in the repo. Fixing this was easier if all generated nix files are in one place, so I did the following changes: * All generated nix files are in `nix/generated`. This de-pollutes the normal working area with generated files, and keeps them out from sources (more precise derivations). * The local generated files now use `gitSource`. * The generator is now `nix/generate.nix`. * I moved complex logic from `nix/default.nix` to `nix/generate.nix`, this means everything is in one place, instead of spread over several nix files. * There is now only a single check to check if the generated files are upto date, and a single command to update the files. * This check is no longer marked `allowSubstitutes = false`. The check isn’t cheap (it requires `cabal2nix`) and we want the check to happen on builders, not hydra itself, so that the cache has `cabal2nix`. * The check is now included in `all-systems-go`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://dfinity.atlassian.net/browse/INF-1002
Replace IFD calls to callCabal2nix with importing generated
default.nixfiles.The generated
default.nixfiles are stored in the repository and contain instructions on how to regenerate them.The generation of the
default.nixfiles is done using thehaskellSrc2nixWithDocfunction which generates a directory containing adefault.nixwhich is the result of runningcabal2nixwith the
extraCabal2nixOptionson the providedsrc. A header is then added to thedefault.nixwhich contains instructions on how to regenerate that file.Finally the new jobs are added which check that the stored
default.nixmatches the expecteddefault.nix. So whenever the.cabalfiles are updated and don't match the storeddefault.nixanymore CI will complain.