Haskell deps that use backpack get mangled

[andrewthad]

Issue description

I have a repository on github that illustrates this issue. Using nix to build a haskell project with a dependency that uses backpack internally does not work correctly.

Steps to reproduce

The steps to reproduce are provided in the repository linked above, but a copy is provided here:

Run the following commands, which results in the error shown afterwards:

bash> git clone https://github.com/andrewthad/nix-backpack-problem
bash> nix-build --pure release.nix
... elided for brevity ...
Preprocessing executable 'example' for nix-backpack-problem-1.0.0..
Building executable 'example' for nix-backpack-problem-1.0.0..
[1 of 1] Compiling Main             ( Main.hs, dist/build/example/example-tmp/Main.o )
/nix/store/ah79awgrjc75c68nn6gp32v4a9qli7mn-primitive-containers-0.1.0.0/lib/ghc-8.4.1/primitive-containers-0.1.0.0/Data/Map/Unboxed/Lifted.hi
Declaration for primitive-containers-0.1.0.0-Dh702QUZ6mrBrQ6cfYPU3V:Data.Map.Unboxed.Lifted.$fIsListMap2{v r8x}:
  Bad interface file: /nix/store/ah79awgrjc75c68nn6gp32v4a9qli7mn-primitive-containers-0.1.0.0/lib/ghc-8.4.1/primitive-containers-0.1.0.0/Map.hi
      Something is amiss; requested module  primitive-containers-0.1.0.0-JNYz44HuDdtIDtASnNcM9F-map-indef+2bbwYj3qpR1AhwaWbfmWe2:Map differs from name found in the interface file primitive-containers-0.1.0.0-JNYz44HuDdtIDtASnNcM9F-map-indef+K2XRbKjbYZpLEn7VFxOfUn:Map (if these names look the same, try again with -dppr-debug)

<no location info>: error:
    Cannot continue after interface file error
builder for '/nix/store/0bkrbljm2kyph09waf7lw9ksmqfkrckf-nix-backpack-problem-1.0.0.drv' failed with exit code 1
error: build of '/nix/store/0bkrbljm2kyph09waf7lw9ksmqfkrckf-nix-backpack-problem-1.0.0.drv' failed

Make sure you do not have a dist or dist-newstyle directory in here since
that confuses nix-build and can make it claim that it has successfully built
the project. I am able to successfully build this project, and projects like it,
with cabal new-build when I am not using nix. I suspect that the problem is
that the different instantiations of the same indefinite module are somehow
clobbering one another.

cc @chessai @taktoa

[andrewthad]

cc @ezyang

[ezyang]

Yes, the path /nix/store/ah79awgrjc75c68nn6gp32v4a9qli7mn-primitive-containers-0.1.0.0/lib/ghc-8.4.1/primitive-containers-0.1.0.0/Map.hi definitely looks like clobbering; it really needs to have the Backpack ID (which Cabal should have generated in the folder structure) here.

I am not too familiar with Nix. Does anyone here know / know who would know how Nix supports convenience libraries (without Backpack) and, if it does, how it manages installing them into the nix store?

[cocreature]

@ezyang Nix uses Setup.hs directly, see https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/generic-builder.nix#L333. I’m not sure it does anything special for convenience libs.

[ezyang]

The registration code looks fine, because it is using ‘id’ to determine what name it should register as, and id does contain the instantiation ID (that’s the hash that is after the plus sign in the ID). Also, registration code being a problem is inconsistent with the symptoms, which is that the build products themselves are clobbering each other.

So, what I expect, is that Nix is feeding Cabal information about where it should “install” the files, and that’s the crux of the matter. In particular, these lines don’t look too good: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/generic-builder.nix#L141

Libsubdir is using $pkgid only, which looks bad to me. The default setting in Cabal is $abi/$libname; note the use of $libname here: $libname will record instantiation information; $pkgid does not. But I imagine that if we change this, there are knock on effects in the Nix file. "--ghc-option=-optl=-Wl,-rpath=$out/lib/${ghc.name}/${pname}-${version}" also looks questionable.

[chessai]

@peti this seems more like a bug than a request for an enhancement, if i'm not mistaken

[quasicomputational]

Packages that only use mixins for hiding/renaming modules are also broken, even if they don't use backpack itself (i.e., no signatures).

[cartazio]

Theres a reasonably decent probability that (should all the details work out) the next minor / major releases of vector will use backpack to simplify the internal of vector while providing the same apis with improved extensibility. (To be clear, I’m still futzing around with some prototypes to understand if there’s any blockers I’ve not anticipated)

I’m hoping to have some clarity on how effective this design direction is over the coming month or so and be able to do a release for everyone sometime in the next 6 months if this all works out as I hope.

Since I use cabal I’m not impacted by this and other build systems having buggy support for ghc Haskell. But id like to still be friends with them after the next release of vector. So I’d really appreciate it if this bug could be taken seriously.

Also it’s a bummer that a really cool modularity feature isn’t supported.

But seriously: please fix this. Vector is widely used and I want to make sure it keeps on getting better. Not supporting backpack hurts the entire ecosystem and will suck for everyone.

[expipiplus1]

The testcase @andrewthad posted builds successfully now. Can this issue be closed and backpack marked as supported?

CC @shlevy, I think you are using backpack with nix, right?

[chessai]

Unfortunately this is not the only place where backpack is still an issue. @andrewthad has another one that we ran into at work awhile back. Perhaps he can provide a minimal example, as I no longer work there.

[andrewthad]

There isn't any nix file in the repo for building it, but https://github.com/andrewthad/vex-instantiate is another super minimal example of a library that cabal build (3.0) can build, but as of November 15, 2019, I don't think nix was able to.