cabal-install HEAD trips assert in XDG paths mode

[geekosaur]

Describe the bug
When attempting to use the new support for XDG paths, I got the following assertion fail:

Assertion failed
CallStack (from HasCallStack):
  assert, called at src/Distribution/Client/ProjectPlanning.hs:246:5 in cabal-install-3.9.0.0-inplace:Distribution.Client.ProjectPlanning

To Reproduce
Move ~/.cabal out of the way and attempt to build a project with a significant number of transitive dependencies.

$ cabal v2-install exe:xmonad-bsa --enable-executable-stripping --enable-optimization=2 --installdir=$(pwd) --overwrite-policy=always -v3

Expected behavior
Successful build and install, as with ~/.cabal instead of XDG paths. (I forced a full build/install of that as well; I had been using ghc 8.10.7 so I upgraded it to match the version used with cabal HEAD as of last week.)

System information

• Ubuntu 20.04.1 x86_64

• cabal-install version 3.9
  compiled using version 3.9.0.0 of the Cabal library 
  

• The Glorious Glasgow Haskell Compilation System, version 9.2.4

Additional context
See the attached typescript taken with cabal -v3 as shown above.

[geekosaur]

Github won't let me attach the typescript, even as a tar.gz. It is available as https://www.dropbox.com/s/qd2zk9p3omj6lbc/typescript.txt.tar.gz?dl=0.

[Mikolaj]

Thank you for the report. Does it fail the same with cabal build?

@athas: the error message seems similar to what we are already getting in other tickets unrelated to XDG. Still, if you could have a look, experiment, diagnose a bit, that would be awesome.

[geekosaur]

Same failure with cabal v2-build.

Interestingly, I get a different failure mode if I start out with an unpopulated ~/.cabal, do a cabal v2-update, then attempt the same cabal v2-build. Github still isn't letting me do attachments so see https://www.dropbox.com/s/1bque2ra6ke9w9r/cabal-debug.tgz?dl=0. The failure in typescript-dotcabal is not obvious; search for CallStack.

[Mikolaj]

That's scary. Does cabal clean help? Is there anything written to the XDG location as well (e.g., after you wipe out ~/.cabal? Did you, by any chance, use cabal install --lib ever (if so, GHC env files may be laying around)? Am I asking for too much when I wonder alound if this fails the same with cabal-install 3.8.1 (easy to obtain from ghcup tui)?

I'm guessing blindly, but perhaps cabal compiled using ~./cabal, which got removed, so it created the XDG dirs and tried to continue (recompile), perhaps not noticing anything has changed. It certainly should not crash.

[geekosaur]

It works if I use my existing ~/.cabal (I have it bind-mounted so I can make it go away and come back easily; unfortunately xmonad is running from it so I have to log out of my GUI session to do these tests).

There are no environment files anywhere except in a testbed that is sandboxed so it's not visible unless I use my sandbox script that alters $HOME etc.

I got this cabal binary from https://mail.haskell.org/pipermail/haskell-cafe/2022-October/135590.html (more specifically https://github.com/haskell/cabal/suites/8566596132/artifacts/383133509) so it should have no links to anything on my system, although I can't promise that it didn't get them from CI.

I haven't tried clearing out the XDG locations with a cleared ~/.cabal but can try that and cabal 3.8.1.0 tomorrow. I do know that if ~/.cabal doesn't exist when I start out, it still doesn't exist afterward.

[Mikolaj]

Please try cabal clean, too. One more keyword in the typescript is that caught my attention is "secure repo". I wonder if perhaps hackage-security hardwires that keys are stored in ~/.cabal. Or perhaps cabal can recover when ~/.cabal is wiped out, but not when it's partially wiped out or overwritten with an older version, which may be what it sees when some files are present in the corresponding XDG location after ~/.cabal is gone.

The more bugs you find, the better. :) But if a smaller repro can be distilled, even exhibiting only one of the bugs, that's surely going to help.

[athas]

If a ~/.cabal directory appears at any point (for example created by other tooling), the XDG-enabled cabal will certainly become horribly confused, as this will enable the backwards compatibility code path. But it doesn't sound like this is happening.

[geekosaur]

Were I doing it, I'd've probably looked for ~/.config/cabal/config before ~/.cabal/config to enable the new code path instead of letting existence of ~/.cabal control it; this after some experience handling backward compatibility in xmonad, where things floated around a bit before we found a setup that was reliable and mostly pain-free. "Other tooling" (which in our case includes contrib modules) figured into this, since some modules weren't using the documented ways to determine config and data directories.

I mention this in part because at some point I'll want to test to see if https://github.com/phadej/cabal-extras works with the new cabal.

[athas]

Is there a public project that reproduces this behaviour?

[geekosaur]

None that I know of. I may test it with a smaller project later, but as I have to knock the window manager down to make ~/.cabal go away (it's bind mounted, and xmonad is running from ~/.cabal/bin) I've been testing with that. I did include the files necessary for it in the second tarball, or you can get them from https://github.com/geekosaur/xmonad.hs/tree/skkukuk, but I admit the dbus dependency makes for a very large footprint given that it drags in both conduit and lens.

[geekosaur]

Sorry for the delay.

I have reproduced the behavior after a cabal clean, and additionally minus the dbus (and therefore conduit and lens) dependency: https://www.dropbox.com/s/500obdhcyphwf52/typescripts-clean.tgz?dl=0.

[athas]

In a checkout of https://github.com/geekosaur/xmonad.hs/tree/skkukuk I am trying the command

cabal v2-build exe:xmonad-bsa --enable-executable-stripping --enable-optimization=2 -v3

but that gives me another internal error:

Error: cabal: Internal error in target matching. It should always be possible
to find a syntax that's sufficiently qualified to give an unambiguous match.
However when matching 'exe:xmonad-bsa' we found exe:xmonad-bsa
(unknown-component) which does not have an unambiguous syntax. The possible
syntax and the targets they match are as follows:
'exe:xmonad-bsa' which matches exe:xmonad-bsa (unknown-component),
:pkg:exe:lib:exe:file:xmonad-bsa (unknown-file)

Using the command

cabal v2-install exe:xmonad-bsa --enable-executable-stripping --enable-optimization=2 --installdir=$(pwd) --overwrite-policy=always -v3

gives me the following non-internal error:

Error: cabal: Unknown package "exe".

[athas]

I hadn't noticed that it was using a non-master branch. Something is happening now. I'll see if I can reproduce the issue.

[geekosaur]

Sorry, yes, master is just the shared components and there's a branch for each machine I have configs for (most now historical, sadly).

[athas]

Well, the repository builds for me, although I'm not using exactly the same binary you are. The binary linked from the mailing list post is produced from essentially a random commit (I think the newest master at the time), and who knows how broken it may be. Could you a freshly built cabal instead?

[geekosaur]

Building now (with cabal 3.8.1.0 just in case).

[geekosaur]

Well, it got significantly further but eventually failed with the same assertion. It also complained about the haddock documentation for dbus not building. https://www.dropbox.com/s/bon17ewlyq9nl7o/typescript-1019.tgz?dl=0

[ulysses4ever]

@geekosaur would it be possible to post listings of output in a more accessible format? Like github gists or something. GitHub itself has a spoiler tag… I prefer to browse this bug tracker from mobile devices, and I see no way to access your links there.

[geekosaur]

Github keeps insisting it doesn't support the file type I'm attaching, even if I change the extension etc. I'm still trying to figure out what it will actually let me attach. Also it's 4MB of output in the latest one (so after editing out control characters and renaming I now get "Something went really wrong"). I don'[t see a reasonable way to do it as a gist, either.

[ulysses4ever]

I know the attaching problem: I hit it myself regularly. But I'm asking to avoid attaching in the first place, and simply to copy and paste plain text. Is this too painful?

[geekosaur]

Have you tried to copy-paste 4MB of text before? (I think I just broke paste.tomsmeding.com.) Granting that's the most recent one, but earlier ones had other files along. In any case, yes I am trying to copy-paste if something will actually let me do it.

[geekosaur]

Okay, gist seems to have taken after paste.comsmeding.com wedged on it: https://gist.github.com/geekosaur/c8d999ec2106bf88298b68ba94005031

[ulysses4ever]

I'm sorry I didn't know that it's that much text. I'm happy gist worked.

[athas]

I'm a bit stumped on this one as I cannot reproduce it. The assert that fires is this, but I don't have a real clear grasp of what it's checking or or when it might fail (or what it could possibly have to do with XDG). It's using this complex-looking function, but these are parts of cabal I've never had to touch.

If the hash of a package (or its inputs) changes during compilation, I really wonder what could cause that. What does the generated ~/.config/cabal/config look like for you?

[geekosaur]

I used the originally generated config, with one change: I set Documentation: True. See https://gist.github.com/geekosaur/b35980c366118f6663d0ee2e6bb51760.

[ulysses4ever]

Fascinating thread... @geekosaur, one curious experiment that you may try is to build cabal master in the release mode, where asserts are no-ops (there's a project file for that on the root). Also, note that documentation: True is known to fail some asserts #8313.

[geekosaur]

I used that project file to build, per the instructions in README.md. I can attempt the build again with the documentation option disabled.

[ulysses4ever]

Are you sure you used the release project file? because that way built cabal wouldn't ever fail with an assertion, because assertions are disabled in the release mode.

Documentation option shouldn't matter much when you build cabal itself. It can matter when you build something else. At least, that's what I saw in practice.

[athas]

I can reproduce the issue when I set documentation: True in the config file. This doesn't bring me much closer to understanding what is going on, but perhaps someone more familiar with those corners of Cabal can eventually help out.

[athas]

It fails even without using XDG paths as soon as I set documentation: True. I think this is unrelated to XDG, but related to some other change in Cabal. It works with the last release.

[athas]

Note for anyone wanting to try this out at home: if you install a new cabal with cabal install, it will apparently not be a debug build, so the assertions will not fail. You need to use cabal build cabal-install and then use cabal list-bin cabal-install to find the cabal binary. Funny how cabal build instructions tend to end up as tongue twisters.

[athas]

It seems to also fail with 67b3c8d, which is the last commit before XDG was merged. I'll try a bisection to find the error.

[athas]

It fails with 3.8.1.0 in debug mode. I'm having trouble compiling 3.6.3.0, so I don't know how much further back I can go.

[geekosaur]

From my shell history: cabal-3.8.1.0 install --project-file=cabal.project.release cabal-install. I then copied the binary using cabal list-bin to ~/.local/bin/cabal-head-20221018 and symlinked it to cabal, since I'm also using ghcup in XDG mode and it gets confused if you don't follow its expectations.

[Mikolaj]

Note for anyone wanting to try this out at home: if you install a new cabal with cabal install, it will apparently not be a debug build, so the assertions will not fail. You need to use cabal build cabal-install and then use cabal list-bin cabal-install to find the cabal binary. Funny how cabal build instructions tend to end up as tongue twisters.

That's probably because cabal install cabal-install installs from Hackage, which ignores project files, while cabal build in the cabal repo use the project files which are configured for dev (debug) build.

[geekosaur]

That's probably because cabal install cabal-install installs from Hackage, which ignores project files, while cabal build in the cabal repo use the project files which are configured for dev (debug) build.

This means the "from git" build instructions in https://github.com/haskell/cabal/README.md are incorrect, since they say to use cabal install --project-file=cabal.project.release cabal-install.

[Mikolaj]

Oh, they do? That would indicate README is not aimed at devs (cabal.project.release disables assertions). Perhaps that's fine? However, we could use some clarification there and some instructions for devs and also some for people that just want to reproduce their bug report with cabal's HEAD and would benefit from extra debug info.

[geekosaur]

That wasn't the thing I was pointing to, but instead that it explicitly specifies cabal-install as a target which means that instead of "from git" it's getting the package from Hackage.

[Mikolaj]

I thought if you are in a repo directory, the package is taken from the repo, not from Hackage. I thought that's the case always: both with single package repos when you specify the component and with multi-package repos, where you specify a package (or a package and component). Am I wrong?

[geekosaur]

I have managed to build using a recent HEAD and Documentation: False, so it appears this ticket can be closed.