Add Backpack support. #1806

ghost · 2022-12-06T20:58:31Z

This PR adds support for Backpack libraries. Should fix #244.

TODO:

Add tests
Refactor the code
Add comments

angerman · 2022-12-07T01:54:53Z

builder/comp-builder.nix

+    if lib.strings.hasInfix "inplace" instLine then
+      "$(read packageId < <(read sedline < <(echo ${instLine} | sed -n -e 's/.*=\\(.*\\)-inplace-\\(.*\\):.*/\\1-\\\\\\\\([^\"]*\\\\\\\\)-\\2/p') ; sed -n -e \"s/.*\$\{sedline\}/\\1/p\" ${configFiles}/configure-flags | head -1) ; echo ${instLine} | sed -n -e \"s/\\(.*\\)-inplace-\\(.*\\)/\\1-\$\{packageId\}-\\2/p\")"
+    else instLine;


This needs a comment, explaining what we try to rewrite, from what to what and how.

builder/comp-builder.nix

angerman · 2022-12-07T01:56:30Z

builder/comp-builder.nix

@@ -162,8 +167,12 @@ let
      if configureAllComponents
        then ["--enable-tests" "--enable-benchmarks"]
        else ["${haskellLib.componentTarget componentId}"]
-    ) ++ [ "$(cat ${configFiles}/configure-flags)"
-    ] ++ commonConfigureFlags);
+    ) ++ [ "$(sed 's/--dependency=\\(.*\\)+\\(.*\\)/--dependency=\\1/' ${configFiles}/configure-flags)"


Needs a comment, that explains what (and why) we are running sed on the configure-flags. E.g. why are they wrong by construction, and what do we need them to look like, and why?

nix-tools/plan2nix/Plan2Nix.hs

angerman

This is great!

@andreabedini can I get your input on this?

andreabedini · 2022-12-07T04:06:11Z

@ak3n If I understand correctly, the logic is the following

patch cabal to add to plan.json the required information (instantiated-with)
pass that information all the way down to the component builder (which at the moment does not see the instantiated components at all?)

Is this correct?

edit: I am not sure what patchDeps does

L-as · 2022-12-07T04:48:31Z

Side note: Isn't Backpack also unmaintained in GHC? (Though I'd love to use it)

L-as · 2022-12-07T04:49:29Z

Did you test this in the dev shell with exactDeps = true?

angerman · 2022-12-07T07:42:07Z

Side note: Isn't Backpack also unmaintained in GHC? (Though I'd love to use it)

I'm afraid I too think it mostly is, but I'm in the same boat as you. I want to use it ;)

ghost · 2022-12-20T20:13:54Z

@angerman thanks for the review! Fixed some moments. Will add comments a bit later after refacting the code. Added the passing test for now.

@andreabedini yep, that's it. The problem at the moment is that we do not have instantiated libraries — cabal instantiates them itself while building with cabal build but haskell.nix uses plan.json and ./Setup.

andreabedini · 2022-12-20T23:44:00Z

The problem at the moment is that we do not have instantiated libraries — cabal instantiates them itself while building with cabal build but haskell.nix uses plan.json and ./Setup.

In #1775 we started moving away from reading plan.json and using cabal-install internall library to extract the information we need (which in that PR is the revised cabal files). Here you are doing something very similar by patching cabal-install 👍

ghost · 2022-12-21T19:42:44Z

@L-as nope, testing right now, thanks!

@angerman, @andreabedini, I have updated the PR, refactored the code, added tests and comments. Please take a look. It should be more clear now.

UPD: @L-as how exactly to test this? Is starting a nix-shell with exactDeps is enough?

L-as · 2022-12-22T00:33:57Z

@L-as nope, testing right now, thanks!

@angerman, @andreabedini, I have updated the PR, refactored the code, added tests and comments. Please take a look. It should be more clear now.

UPD: @L-as how exactly to test this? Is starting a nix-shell with exactDeps is enough?

Yep, but do note it doesn't work with source-repository-package well. To make it work with source-repository-package, those stanzas must only be visible to haskell.nix, so you must put them into the cabalProjectLocal argument to cabalProject'.

ghost · 2022-12-23T16:26:46Z

@L-as yeah, that works for me.

For some reason nix run .#hix --accept-flake-config -- run github:haskell/cabal/3.8#cabal-install:exe:cabal --accept-flake-config --override-input haskellNix . -- --version fails.

It tries to patch cabal-install but there is no src folder in unpacked source archive path, only main for the executable.

andreabedini · 2023-02-09T04:30:17Z

Now that #1775 is merged I can integrate this. Sorry for the wait @ak3n!

ghost · 2023-02-09T07:37:20Z

@andreabedini thanks! but actually I paused this a bit — I have started porting backpack tests from cabal and discovered new situations which are not processed properly yet, marked as draft for now.

ghost · 2023-02-16T16:22:08Z

@andreabedini there is something really wrong with my CI checks...

*** Running the nix-build tests...
nix: /nix/store/5lp5v0sa76mhvc42bdc4z8vxwysp18r9-gcc-11.3.0-lib/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /nix/store/4d5r7ibhfyjajbr853b6hqy2x1nhn4pz-nix-2.13.2/lib/libnixstore.so)
nix: /nix/store/5lp5v0sa76mhvc42bdc4z8vxwysp18r9-gcc-11.3.0-lib/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /nix/store/4d5r7ibhfyjajbr853b6hqy2x1nhn4pz-nix-2.13.2/lib/libnixutil.so)
nix: /nix/store/5lp5v0sa76mhvc42bdc4z8vxwysp18r9-gcc-11.3.0-lib/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /nix/store/5qgdg3nja42qbv7dk2a8p9dd5m226fzb-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-transfer.so)
nix: /nix/store/5lp5v0sa76mhvc42bdc4z8vxwysp18r9-gcc-11.3.0-lib/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /nix/store/5qgdg3nja42qbv7dk2a8p9dd5m226fzb-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-core.so)

is it a known issue?

andreabedini · 2023-02-17T08:18:38Z

@ak3n yes, @hamishmack should have fixed that in #1846.

Add test with 2 sigs instantiated together. Add comments. Refactor cabal2nix.

ghost · 2023-02-22T17:50:50Z

Ok, this time I'm blocked by something strange (well, all previous blockers looked so).

I have added the Includes3 test from the cabal-testsuite which tests that we can instantiate the signature with a module from external package (nice test!). I have fixed my solution to figure out and provide the external package for instantiated package but it doesn't work.

Configure flags:
--prefix=/nix/store/9k48cbvw20wmrm3khfshc62ncfcd9svk-Includes3-lib-indef+6YmfnARghEC3uazvHVLJ9b-0.1.0.0 lib:indef --package-db=clear --package-db=/nix/store/zp01x9xyhg2wzaqixwr1cqvrp4mrfryg-Includes3-lib-indef+6YmfnARghEC3uazvHVLJ9b-0.1.0.0-config/lib/ghc-9.2.5/package.conf.d --exact-configuration --dependency=sigs=Includes3-0.1.0.0-1Ga82QqVbapC7ye3zCsNVv-sigs --dependency=Includes3:sigs=Includes3-0.1.0.0-1Ga82QqVbapC7ye3zCsNVv-sigs --dependency=containers=containers-0.6.5.1-LoBk5T9QwNLBqnqOQVtDCh --dependency=indef=Includes3-0.1.0.0-Co7UX9U53rW27g504uor8I-indef --dependency=Includes3:indef=Includes3-0.1.0.0-Co7UX9U53rW27g504uor8I-indef --dependency=array=array-0.5.4.0 --dependency=base=base-4.16.4.0 --dependency=deepseq=deepseq-1.4.6.1 --dependency=ghc-bignum=ghc-bignum-1.2 --dependency=ghc-boot-th=ghc-boot-th-9.2.5 --dependency=ghc-prim=ghc-prim-0.8.0 --dependency=integer-gmp=integer-gmp-1.1 --dependency=pretty=pretty-1.1.3.6 --dependency=rts=rts --dependency=template-haskell=template-haskell-2.18.0.0 --with-ghc=ghc --with-ghc-pkg=ghc-pkg --with-hsc2hs=hsc2hs --with-gcc=cc --with-ld=ld --with-ar=ar --with-strip=strip --disable-executable-stripping --disable-library-stripping --disable-library-profiling --disable-profiling --enable-static --enable-shared --disable-coverage --enable-library-for-ghci --datadir=/nix/store/3ssmnhbjlrpvfddkvmbs1hj64gwz0r0z-Includes3-lib-indef+6YmfnARghEC3uazvHVLJ9b-0.1.0.0-data/share/ghc-9.2.5 --instantiate-with=Data.Map=containers-0.6.5.1:Data.Map
Configuring library 'indef' instantiated with
  Data.Map = containers-0.6.5.1:Data.Map
for Includes3-0.1.0.0..
Warning: 'hs-source-dirs: repo/exe-0.1.0.0' specifies a directory which does
not exist.
Warning: 'hs-source-dirs: repo/sigs-0.1.0.0' specifies a directory which does
not exist.
Error:
    The following packages are broken because other packages they depend on are missing. These broken packages must be rebuilt before they can be used.
planned package Includes3-0.1.0.0 is broken due to missing package containers-0.6.5.1

error: builder for '/nix/store/56vmgj89wl30mvbi21k0d961aqzjs43q-Includes3-lib-indef+6YmfnARghEC3uazvHVLJ9b-0.1.0.0.drv' failed with exit code 1;
       last 10 log lines:
       >   Data.Map = containers-0.6.5.1:Data.Map
       > for Includes3-0.1.0.0..
       > Warning: 'hs-source-dirs: repo/exe-0.1.0.0' specifies a directory which does
       > not exist.
       > Warning: 'hs-source-dirs: repo/sigs-0.1.0.0' specifies a directory which does
       > not exist.
       > Error:
       >     The following packages are broken because other packages they depend on are missing. These broken packages must be rebuilt before they can be used.
       > planned package Includes3-0.1.0.0 is broken due to missing package containers-0.6.5.1
       >
       For full logs, run 'nix log /nix/store/56vmgj89wl30mvbi21k0d961aqzjs43q-Includes3-lib-indef+6YmfnARghEC3uazvHVLJ9b-0.1.0.0.drv'.
error: 1 dependencies of derivation '/nix/store/wawlp4bv5rx5rhihmflpcdyc539hcdnn-Includes3-exe-exe-0.1.0.0-check.drv' failed to build
error: build of '/nix/store/wawlp4bv5rx5rhihmflpcdyc539hcdnn-Includes3-exe-exe-0.1.0.0-check.drv', '/nix/store/z7phz0qjpb71ffln1r9mpn4mp1crir9d-Includes3-exe-exe-0.1.0.0.drv' failed

It says that there is a missing package containers anyway (generated nix plan).

Need some time again to reset the focus I suppose.

michaelpj · 2023-04-13T09:27:56Z

I'm going to close this because for some reason the update bot keeps pushing commits to it and it's probably pointlessly burning CI resources.

angerman reviewed Dec 7, 2022

View reviewed changes

builder/comp-builder.nix Outdated Show resolved Hide resolved

angerman reviewed Dec 7, 2022

View reviewed changes

nix-tools/plan2nix/Plan2Nix.hs Outdated Show resolved Hide resolved

angerman requested changes Dec 7, 2022

View reviewed changes

ghost marked this pull request as ready for review December 21, 2022 19:38

ghost requested a review from angerman December 21, 2022 19:38

ghost marked this pull request as draft January 9, 2023 05:31

andreabedini self-assigned this Feb 9, 2023

Evgenii Akentev added 7 commits February 17, 2023 15:57

Add Backpack support.

b68a8a1

Add test. Fix review comments.

e50d1bd

Add second test with two instantiations.

43a9cb8

Get buildable test back.

cfab953

Add test with 2 sigs instantiated together. Add comments. Refactor cabal2nix.

Add bkpcabal01 test from cabal testsuite.

87b40ff

Add Includes2 test from cabal testsuite.

9d1cf22

Add Includes3

601f688

IOHK added 28 commits March 16, 2023 00:49

Update Hackage and Stackage

8ee2dec

Update Hackage and Stackage

fca642a

Update Hackage and Stackage

f9b5a4f

Update Hackage and Stackage

1f1876f

Update Hackage and Stackage

0303d65

Update Hackage and Stackage

a53d27c

Update Hackage and Stackage

2badee2

Update Hackage and Stackage

29b8b5f

Update Hackage and Stackage

f4d349a

Update Hackage and Stackage

59a2d70

Update Hackage and Stackage

c728663

Update Hackage and Stackage

b933ec7

Update Hackage and Stackage

40abff8

Update Hackage and Stackage

a8eded8

Update Hackage and Stackage

80c52c6

Update Hackage and Stackage

00a2429

Update Hackage and Stackage

2ed4b72

Update Hackage and Stackage

4e4a900

Update Hackage and Stackage

55ef36e

Update Hackage and Stackage

7417b7f

Update Hackage and Stackage

2e2114f

Update Hackage and Stackage

2b1967b

Update Hackage and Stackage

164d22f

Update Hackage and Stackage

875b265

Update Hackage and Stackage

9aa2f10

Update Hackage and Stackage

a29642c

Update Hackage and Stackage

b038211

Update Hackage and Stackage

c1a2855

michaelpj closed this Apr 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Backpack support. #1806

Add Backpack support. #1806

ghost commented Dec 6, 2022 •

edited by ghost

Loading

angerman Dec 7, 2022

angerman Dec 7, 2022

angerman left a comment

andreabedini commented Dec 7, 2022 •

edited

Loading

L-as commented Dec 7, 2022 •

edited

Loading

L-as commented Dec 7, 2022

angerman commented Dec 7, 2022

ghost commented Dec 20, 2022

andreabedini commented Dec 20, 2022

ghost commented Dec 21, 2022 •

edited by ghost

Loading

L-as commented Dec 22, 2022

ghost commented Dec 23, 2022

andreabedini commented Feb 9, 2023

ghost commented Feb 9, 2023

ghost commented Feb 16, 2023

andreabedini commented Feb 17, 2023

ghost commented Feb 22, 2023

michaelpj commented Apr 13, 2023

Add Backpack support. #1806

Add Backpack support. #1806

Conversation

ghost commented Dec 6, 2022 • edited by ghost Loading

angerman Dec 7, 2022

Choose a reason for hiding this comment

angerman Dec 7, 2022

Choose a reason for hiding this comment

angerman left a comment

Choose a reason for hiding this comment

andreabedini commented Dec 7, 2022 • edited Loading

L-as commented Dec 7, 2022 • edited Loading

L-as commented Dec 7, 2022

angerman commented Dec 7, 2022

ghost commented Dec 20, 2022

andreabedini commented Dec 20, 2022

ghost commented Dec 21, 2022 • edited by ghost Loading

L-as commented Dec 22, 2022

ghost commented Dec 23, 2022

andreabedini commented Feb 9, 2023

ghost commented Feb 9, 2023

ghost commented Feb 16, 2023

andreabedini commented Feb 17, 2023

ghost commented Feb 22, 2023

michaelpj commented Apr 13, 2023

ghost commented Dec 6, 2022 •

edited by ghost

Loading

andreabedini commented Dec 7, 2022 •

edited

Loading

L-as commented Dec 7, 2022 •

edited

Loading

ghost commented Dec 21, 2022 •

edited by ghost

Loading