Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

default.nix: dep: upd hnix-store 0.2 #580

Merged
merged 5 commits into from
May 26, 2020

Conversation

Anton-Latukha
Copy link
Collaborator

@Anton-Latukha Anton-Latukha commented May 23, 2020

Report: Use hnix-store 0.2 #561


This PR depends on the: default.nix: clean-up old overrides #579 please merge that first!


*this gives us ground to make New release #560

, which would close: Compatibility woes #551

  • freed the semialign from being 1.1 - there are no 1.1 in new nixpkgs revisions
  • updated nixpkgs revision
  • hnix-store 0.2 its overlay is empty. hnix-store 0.2 now works fine from the main repository, no load of its overrides or from source is needed.

New nixpkgs revision has a lot to compile. Travis probably would fail to compile it in the free tier timeout (~45-60m).

... current Travis build fail (usage of dependent-sum-0.7.1.0 but dependent-sum < 0.7 is addressed in: dep: upd dependent-sum (0.6->0.7) #559.


As always:

  • everything compiles
  • passes tests
  • binary executes and evaluates

@Anton-Latukha Anton-Latukha added the refactoring No API breakages. Work that only makes code to be nicer or allows for future functionality label May 23, 2020
@Anton-Latukha Anton-Latukha mentioned this pull request May 23, 2020
@Anton-Latukha Anton-Latukha added the dependency update Updating and changing dependencies label May 23, 2020
@Anton-Latukha Anton-Latukha added this to the 0.7.2 minor release milestone May 23, 2020
This was referenced May 25, 2020
@Anton-Latukha Anton-Latukha requested a review from sorki May 25, 2020 10:59
sorki
sorki previously approved these changes May 25, 2020
Copy link
Member

@sorki sorki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. We will need hnix-store overlay again quite soon so it's good to keep it around.

Copy link
Member

@sjakobi sjakobi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know too little about Nix to tell whether this is a good change! :)

I asked questions instead, to learn more. I would appreciate answers but don't feel obliged!

@@ -8,7 +8,7 @@

, withHoogle ? true

, rev ? "8da81465c19fca393a3b17004c743e4d82a98e4f"
, rev ? "29d57de30101b51b016310ee51c2c4ec762f88db" # 2020-05-23: NOTE: UTC 17:00
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this change other dependency versions too? Is there something like a diff-tool that would reveal the changes relevant to this project?

Copy link
Collaborator Author

@Anton-Latukha Anton-Latukha May 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question.

Back in a day I coudn't figure-out such a question.

In short:

nix-store --query --references  $(nix-instantiate 'https://github.com/nixos/nixpkgs/archive/29d57de30101b51b016310ee51c2c4ec762f88db.tar.gz' -A haskellPackages.hnix)

You and I probably should make a function from it.

Output is:

warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh
/nix/store/8a1fcpb7hv65b1krcfhmp26f9imvprd9-coreutils-8.31.drv
/nix/store/a7hrhw1zxjbyxdxngh73dpnf3izzp1m2-bash-4.4-p23.drv
/nix/store/j95ipv9gwq63d5f8923fk7zax0y7plcg-gnugrep-3.4.drv
/nix/store/32ad3n4cvyp1gm6gikclhkija3sinn6s-stdenv-linux.drv
/nix/store/3w099kjh1m33zxfxmsxg8kn8mcry2px6-remove-references-to.drv
/nix/store/4mdp8nhyfddh7bllbi7xszz7k9955n79-Setup.hs
/nix/store/907d7wf2d113vlv8dlhi1i3d683gc1rs-unpretty-cabal-conf.awk
/nix/store/nmngrb0pda8kap2h9rgzda1yv11mhjd2-ghc-8.8.3.drv
/nix/store/zhvngnx9477pb2m117ygnni2grnj2fhr-glibc-locales-2.30.drv
/nix/store/n6fxxjm913jljhir19iwj8bvsmr89vva-hscolour-1.24.4.drv
/nix/store/0d2qfv79s1y6i31fl64ncji5myfj3a4r-http-types-0.12.3.drv
/nix/store/3kfb6s2pp0h9gp8lry4jab9k3a2dq0sm-comonad-5.0.6.drv
/nix/store/hhq0isqk0bfyfczqpgv09znyr3p3nm6f-transformers-base-0.4.5.2.drv
/nix/store/wqx06l77hbryf6z4vvymg90bmbsbg2l4-exceptions-0.10.4.drv
/nix/store/0ddlgamp32gi1sbqjgqvg2gvz6mjifrs-free-5.1.3.drv
/nix/store/0g0n6xc8al09wdr0njp3ipjszrwdmgpl-hashing-0.1.0.1.drv
/nix/store/1zav5vskyhpxwzgpxlsdpp3lg3x4a07p-logict-0.7.0.2.drv
/nix/store/3k1adpwnybpas1mpjxd6mmfczr9xz2n2-optparse-applicative-0.15.1.0.drv
/nix/store/aqvhv0zyq47nw7q866ni8gj15n8g38fd-jailbreak-cabal-1.3.5.drv
/nix/store/kr90md1yhv65cxpndfgxr9b6h10s2bjg-base16-bytestring-0.1.1.6.drv
/nix/store/467j9zp292y05wh68l9gswphhq11z7pk-cryptohash-sha1-0.11.100.1.drv
/nix/store/6231z9by4h84qmnfscjdqfg39pxcslyi-data-fix-0.2.1.drv
/nix/store/6mdjxvkfk5lid38dqgcl1r1w69rj160k-lens-family-core-2.0.0.drv
/nix/store/6wkifsq03gpqjziznmd8czfnqnp69wpg-hashable-1.3.0.0.drv
/nix/store/iqwalk1s48g1x55f9n4df7lsp0zjb5cx-vector-0.12.1.2.drv
/nix/store/khyb5v16bdlvfivxy3pxwyc9wsviraz6-scientific-0.3.6.2.drv
/nix/store/qdkaaqwi18j7jvcgw3apmln2yxd2hm81-unordered-containers-0.2.10.0.drv
/nix/store/812ph4x75x09z4d8b1bfq20pgmbiydgm-aeson-1.4.7.1.drv
/nix/store/8shbi5v89w3rfbppgi0hqzw62216m4kx-lens-family-2.0.0.drv
/nix/store/9as81rdl4fc9rj9mkxnssv8gp6xcrw9g-deriving-compat-0.5.8.drv
/nix/store/h7n6fnw5gfw58v03yqzbi8hhvhizw80z-cryptohash-sha256-0.11.101.0.drv
/nix/store/y5sb0pz39692adrsjc3fk6jsxyshr7cd-cryptohash-md5-0.11.100.1.drv
/nix/store/ywfdqcp45ls1214qxkf0nxgfl8xj68vd-regex-tdfa-1.3.1.0.drv
/nix/store/9yzqn7s76c1x02a7645s9hq5njx18w3y-hnix-store-core-0.2.0.0.drv
/nix/store/cpj205asqdr77mwhwmb6524d75m230y2-prettyprinter-1.6.1.drv
/nix/store/dg2r82gax005w0h6v6v0k8ig0kkqymqy-contravariant-1.5.2.drv
/nix/store/dpj50iczy4d7sv26h05p39xjmwhdi5mj-dependent-sum-0.7.1.0.drv
/nix/store/drhahmknjkc6mavspyijqpmxpk4czwxw-these-0.7.6.drv
/nix/store/f8pwqjcw90pp347mwfb6k47lz4sb6bfj-interpolate-0.2.0.drv
/nix/store/xzyqc5gvpi305gbis33xibd5yv1xrw5k-http-client-0.6.4.1.drv
/nix/store/fg1fg7gjzn2i7ny0vnhfby30zn66jh4z-http-client-tls-0.3.5.3.drv
/nix/store/gwmvgyjk24gwzfz19xf62jdl2w1pkfa8-cryptohash-sha512-0.11.100.1.drv
/nix/store/hxdc2q670zzijf99lylj3p48v670h5bw-ref-tf-0.4.0.2.drv
/nix/store/ik89d1jiivc7dfq616bfryydh7gdifh4-pretty-show-1.10.drv
/nix/store/jjpz1wknicpi8kyjlrmp4i3drz9h36gd-repline-0.2.2.0.drv
/nix/store/knhdz2f6r3wgv4x0gf3k3jf7c4dnxyam-monad-control-1.0.2.3.drv
/nix/store/lp0kwyncyfr348a9jkgjq2186q55wz4k-monadlist-0.0.2.drv
/nix/store/pdk4qvwfgims85f6vsqklv6n5lczl67f-semialign-1.1.drv
/nix/store/m0rlzy7h3a33d1x3cbqj5ja7534y5qyn-semialign-indexed-1.1.drv
/nix/store/pn377lfinr8lnrcv52sa94936hbl6pyq-split-0.2.3.4.drv
/nix/store/sp6fbrvnbrjc42ag0npiwx33d7kfzfky-lens-family-th-0.5.0.2.drv
/nix/store/v7rqa79jmy9q83hr0bqyg2wb3brbii93-xml-1.3.14.drv
/nix/store/zcagmivvxvqnn30ffr7jdqm2r93vn38j-parser-combinators-1.2.1.drv
/nix/store/vh9ncc6v9r18h1022ckivj3d9ylxh998-megaparsec-8.0.0.drv
/nix/store/wcgjy4kg82f0yf1n3zm9x9rkqd5mbgmw-semigroups-0.19.1.drv
/nix/store/wr5icn7wkycnq51fnsfaqqmjwb34kb6d-hnix-0.7.1.tar.gz.drv
/nix/store/wxqij2grx351r0z43c46kxrajcp5n66b-serialise-0.2.3.0.drv
/nix/store/zpzykfbjcdn32qamrzh9qnmrm54762gx-syb-0.7.1.drv

We also of-course can cut-out the /nix/store/hash part.

Copy link
Collaborator Author

@Anton-Latukha Anton-Latukha May 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe in the future we would just need to put master there, or nixos-unstable, so the dependencies would roll themselves.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the extensive response! When I tried to run the same command:

$ nix-store --query --references  $(nix-instantiate 'https://github.com/nixos/nixpkgs/archive/29d57de30101b51b016310ee51c2c4ec762f88db.tar.gz' -A haskellPackages.hnix)
error: Package ‘hnix-0.7.1’ in /nix/store/45h4ksscq6id5m39db6vfa3fa8s1s45w-29d57de30101b51b016310ee51c2c4ec762f88db.tar.gz/pkgs/development/haskell-modules/hackage-packages.nix:125315 is marked as broken, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

(use '--show-trace' to show detailed location information)

I went with adding a ~/.config/nixpkgs/config.nix. I hope that was the right call! :)

Copy link
Collaborator Author

@Anton-Latukha Anton-Latukha May 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, so we trying to fix it.

We always can contribute to Nixpkgs.

Interesting why it marked broken.

Probably it is HNix at blame, created a cross-dependency issues in Nixpkgs.

But the "Broken" mechanism is broken in Nixpkgs in itself. It is like a kind-of must to propagate Nothing in Haskell, if Nothing demanded the propagation to dependent packages.

Propagation is: broken status breaks dependent packages, so they also become marked broken. But there is no mechanism currently to unbreak the packages or see their tree hierarchies, except to someone checking manually if they work..

Comment on lines +64 to +65
# 2020-05-23: NOTE: Currently HNix-store needs no overlay
# hnix-store-src = pkgs.fetchFromGitHub {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the motivation for keeping the overlay as a comment instead of simply removing it?

@sorki mentioned we'd soon need it again. What for?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nixpkgs rolls differently to Hackage rolls.

Nixpkgs branches tend to have the build descriptions that work only for a set of versions that build the same as the latest version, or those old important versions should receive a different name for a package description.

We do not know how things going inside hnix-store, until now I still yet to read at least some of their source code.

But we know that if some issues happen between Hackage and Nixpkgs for them - they would put it in hierarchy that we can load from that overlay.nix package, we effectively would build a chain hackage -> nixpkgs -> hnix-store overlay -> hnix-store build to be able to build it "their current way".

Because Nix language is hard to remember, and we may need that line in the future from time to time - better to live it as a comment, or even compose it into a switch in the header of the file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I clicked "unresolved" just to to ensure that you see my response)

Thanks for the explanation! Generally, I'm not a huge fan of commented-out code. If we need this again, we can also simply look at the Git history. But you have more experience with Nix so I defer to your judgement. :)

@Anton-Latukha
Copy link
Collaborator Author

Anton-Latukha commented May 25, 2020

@sjakobi

Nix has such complex tooling... Complex UI of the CI. About tooling: There are 20-30 tools in different languages written by completely different people for different things and have a different style of data output, and have very different keys, often with long key names.

Basically - the only way to know Nix - is to use NixOS daily, read a bunch of documentation, play in the REPL, and keep notes of those commands you encounter/create your own functions for them, and participating in the Nixpkgs writing/reviewing a bunch of packages, and constantly refer Manuals, but not everything is there, and what there are not always put understandably and often quite open to interpretations.

When I started 2015-2017 - a lot of Nix stuff was just a secret knowledge that was not on the Internet except reading code sources directly. And people in the community loved a rewriting paradigm, so your dev could be a Boolean value, string, or a set, depending on who were in the structure overshadowed the global thing to shorten, "simplify" what. Good that it is not a style anymore because it even was considered a "good style" by a large subset of old contributors. It all basically was a "secret knowledge" on how to navigate all this, and it is still pretty much is, because of its ad-hockness.

So we have a chance to resolve into better tooling in the end of this.

@Anton-Latukha
Copy link
Collaborator Author

Anton-Latukha commented May 26, 2020

I do not know what the issue is, it keeps blocking PR because it requires branch update, when it is already at master. And that placebo branch update produces an empty merge commit which invalidates the review. Sorry guys, IDK what is going on here. It seems not the only PR I got this way today. It seems not a browser cache, I've resetted cache a bunch of times.

...

Ok, I thought I updated my local master branch... 🤦

It no longer needed, and 1.1 now does not exist in Nixpkgs - and this override
prevents building HNix with HNix-store 0.2

M  default.nix
@Anton-Latukha Anton-Latukha merged commit 389e225 into haskell-nix:master May 26, 2020
@Anton-Latukha Anton-Latukha deleted the upd-HNix-store-0.2 branch May 31, 2020 21:09
@Anton-Latukha Anton-Latukha restored the upd-HNix-store-0.2 branch May 31, 2020 21:11
Anton-Latukha added a commit that referenced this pull request Jul 31, 2020
default.nix: dep: upd hnix-store 0.2

We cleaned-up the `default.nix` from most of overrides.
Anton-Latukha added a commit that referenced this pull request Aug 14, 2020
default.nix: dep: upd hnix-store 0.2

We cleaned-up the `default.nix` from most of overrides.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependency update Updating and changing dependencies refactoring No API breakages. Work that only makes code to be nicer or allows for future functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Compatibility woes
3 participants