nixpkgs is increasing with each Haskell LTS release for 400+ Kb

[danbst]

nixpkgs is increasing with each Haskell LTS release for 400+Kb and now takes 50Mb out of 128 Mb of checkout of nixpkgs. Are people aware of this? What are the policies on repo size?

Issue should be tagged as closure-size, I think (nixpkgs checkout resides in my /nix/store)

[abbradar]

I remember there was a discussion about having only revision and checksum of a Hackage snapshot inside Nixpkgs and building hackage-packages.nix on Hydra instead (and using Nix's import-from-derivation). Was there some sort of conclusion reached?

[danbst]

My concern is not about hackage, but files like https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/configuration-lts-1.5.nix
That's around of 9k lines "library" = dontDistribute super."library"

1. Maybe dontDistribute should be default and LTS configuration should include only those to distribute?
2. LTS releases should be based on on some basis LTS config using Nix merge capabilities

[copumpkin]

@abbradar I think we need one more nix feature for that to work well (NixOS/nix#52), but that feature needs work and someone to work on it.

[ericsagnes]

Related ML thread: Publish All of Hackage

[vcunat]

I thought importing e.g. from fetchurl works fine, only you have that consequence of building the imported stuff during evaluation already.

[domenkozar]

This is becoming more and more annoying, nixpkgs is 100MB while haskell is 10MB and LTS packages are 40MB.

[domenkozar]

Could builtins.fetchTarball be used here? Then we could have a separate job for haskell packages and people could still get LTS packages.

We would need to be very careful introducing new fetchTarball calls, but I think this could be an exception.

cc @peti

[copumpkin]

I don't think we need fetchTarball here. It doesn't seem terrible for a particular nixpkgs version to be tied to a particular haskellPackages version, so updating the ref and sha256 every so often (with e.g., import (fetchFromGitHub { owner = "peti"; repo = "haskellPackages"; ... })) doesn't seem too bad.

Ideally, I'd like there to be a hard rule that fetchTarball appear nowhere in nixpkgs.

[vcunat]

We would want to use immutable references here. Apart from that, AFAIK the main difference is whether you need to compile curl, unzip, etc. during evalutation (e.g. --dry-run).

[peti]

I'll remove almost all Stackage-related package sets from Nixpkgs soon. Give me a few more days, then I'll have a posting for ready nix-dev that explains my plans.

[peti]

http://permalink.gmane.org/gmane.linux.distributions.nixos/20505

tl;dr: once the LTS 7.x package set comes out, I intend to make the following changes in "master":

• All haskell.packages.lts.* package sets will disappear.
• haskellPackages will loosely follow the most recent LTS release,

[domenkozar]

I'd expect tarball size to go down.

http://nixos.org/releases/nixos/16.03/nixos-16.03.1169.25e68f7/nixexprs.tar.xz is 6.8MB
http://nixos.org/releases/nixos/unstable/nixos-16.09pre87916.8983df7/nixexprs.tar.xz is 7.3MB

[domenkozar]

@peti can I ask you to write a changelog entry for this PR?

[peti]

Yeah, you are right. The Haskell situation needs some documentation. I'll see what I can do. I'll be traveling next week, so I might not get to it until the week after that.

[domenkozar]

@peti could we document these changes today before the release? :)

[domenkozar]

@peti I've been talking to various people in Haskell community and this brings major pain to them. Could we keep more than just latest LTS package set?

This would give them a timespan of for example one year to migrate. Currently when they update nixpkgs, they get a new TLS set and they're forced to also upgrade at the same time.

I think keeping latest or a few latest sets would yield very little change to nixpkgs size.

[peti]

@domenkozar, I'll add an entry to the release notes in a few minutes.

About offering LTS package sets again: I don't want to do it. My life as a maintainer for Haskell packages has become much easier since we're following the latest LTS release only. I already threw all the obsolete code out of my generator tools and everything is so much simpler and nicer now! I realize it's inconvenient for some, but I just don't want to bother worrying about ancient LTS package sets anymore, especially not in a situation where we don't have the bandwidth to do test builds for them. If someone else wants to maintain that stuff, I am all for it, but I don't want to any more.