Understanding new-style cabal

[yaxu]

I'm having a hard time using the new cabal install.

For example if I do cabal install in the tidal folder, it'll work the first time, but if I make an edit and do that again, I end up with two versions of tidal with the same version number, and ghci will no longer load it. I have to edit ~/.ghc/x86_64-linux-8.6.5/environments/default to remove tidal and try again, then it works.

However this morning, it's not installing at all:

alex@stanage:~/src/tidal$ cabal install
Wrote tarball sdist to
/home/alex/src/tidal/dist-newstyle/sdist/tidal-1.4.4.tar.gz
Wrote tarball sdist to
/home/alex/src/tidal/dist-newstyle/sdist/tidal-parse-0.0.1.tar.gz
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: base-4.12.0.0/installed-4.1... (user goal)
[__1] trying: ghc-8.6.5/installed-8.6... (user goal)
[__2] next goal: process (user goal)
[__2] rejecting: process-1.6.6.0 (conflict: ghc =>
process==1.6.5.0/installed-1.6...)
[__2] rejecting: process-1.6.5.1, process-1.6.5.0/installed-1.6...,
process-1.6.5.0, process-1.6.4.0, process-1.6.3.0, process-1.6.2.0,
process-1.6.1.0, process-1.6.0.0, process-1.5.0.0, process-1.4.3.0,
process-1.4.2.0, process-1.4.1.0, process-1.4.0.0, process-1.3.0.0,
process-1.2.3.0, process-1.2.2.0, process-1.2.1.0, process-1.2.0.0,
process-1.1.0.2, process-1.1.0.1, process-1.1.0.0, process-1.0.1.5,
process-1.0.1.4, process-1.0.1.3, process-1.0.1.2, process-1.0.1.1,
process-1.0.0.0 (constraint from user target requires ==1.6.6.0)
[__2] fail (backjumping, conflict set: ghc, process)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: process, base, ghc, ghc:buildable
Try running with --minimize-conflict-set to improve the error message.

I'm not sure why this is, the only thing I can think is that I installed some additional packages last night, including a binary that depends on tidal (https://github.com/yaxu/feedforward).

My feeling is that installing global packages isn't working too well. So what should we do instead? Probably if we embraced the new way of doing things we'd have a lot fewer install problems, but I'm not sure what that is..

[yaxu]

@hvr your input would be very much appreciated!

[yaxu]

Here's my ~/.ghc/x86_64-linux-8.6.5/environments/default below.

If I remove package-id process-1.6.6.0-c7b8095b02cad2c23ef06090563d5fc53efc7bc1f0ab4705e0c2cb708e07f5c0 from the file, then tidal builds again, but in the process (no pun intended) the same process-1.6.6.0 line gets added back into the file. So each time I rebuild tidal with changes it seems I need to remove both the process and tidal lines.

clear-package-db
global-package-db
package-db /home/alex/.cabal/store/ghc-8.6.5/package.db
package-id ghc-8.6.5
package-id bytestring-0.10.8.2
package-id unix-2.7.2.2
package-id base-4.12.0.0
package-id time-1.8.0.2
package-id hpc-0.6.0.3
package-id filepath-1.4.2.1
package-id process-1.6.6.0-c7b8095b02cad2c23ef06090563d5fc53efc7bc1f0ab4705e0c2cb708e07f5c0
package-id array-0.5.3.0
package-id integer-gmp-1.0.2.0
package-id containers-0.6.0.1
package-id ghc-boot-8.6.5
package-id binary-0.8.6.0
package-id ghc-prim-0.5.3
package-id ghci-8.6.5
package-id rts
package-id terminfo-0.4.1.2
package-id transformers-0.5.6.2
package-id deepseq-1.4.4.0
package-id ghc-boot-th-8.6.5
package-id pretty-1.1.3.6
package-id template-haskell-2.14.0.0
package-id directory-1.3.3.0
package-id text-1.2.3.1
package-id Cabal-2.4.0.1
package-id hosc-0.17-50a4f514edbe834400a9ce2d4b102d8f2e7a00015f4d439568a91f0e1f3cb25a
package-id tidal-1.4.4-91c9e0bd17f31bd495b8fe404a1181e823290206a7974aba2ae78e0c2fc8a2dc

[yaxu]

I tried cabal install Cabal cabal-install, that doesn't seem to have helped. In fact the same issue is happening with the containers package as well, so now I have to delete three lines each time..

[yaxu]

It's difficult to find any up to date information about using cabal. For example this page looks nice and authoritative, but links to broken blogs, and v1 documentation. I read some of https://www.haskell.org/cabal/users-guide/ until I got to a cabal sandbox init command that didn't work. (cabal v1-sandbox init presumably does but I'm trying to work out how to use the state of the art..)

[jhw]

yeah this stuff is killing me

package-id tidal-1.4.4-91c9e0bd17f31bd495b8fe404a1181e823290206a7974aba2ae78e0c2fc8a2dc

^^^ this is what I get also; and I think the UUID suffix is what is causing cabal info to report Tidal as [ Not Installed ] and is breaking everything

[gbaz]

tidal is purely a lib right? That then gets used by ghci? Is it possible to just cabal build and ask it to generate an env file in the tidal directory, then use ghci pointing to that specific env file? Rather than having a global env file, that will give you a nice clean fresh one each time...

[jwaldmann]

difficult to find any up to date information about using cabal.

I fully agree. - This seems to be a fall-out of first stack trying to be the better cabal, and then cabal trying to be more like stack. Ah, politics. I do fully agree with the motivation (for both): reproducible build and execution environments.

As a drastic work-around: remove the user-global environment file (e.g., "rm ~/.ghc/i386-linux-8.6.5/environments/default") Then "cabal install --lib" works (to build from a source checkout) and Tidal can be used (I am testing with "ghci -ghci-script BootTidal.hs")

This will re-create the user-global environment file, and it will contain an entry for Tidal.

But - this also removes ghc's knowledge of other packages that have been installed by the user meanwhile. And when I try to install some other package (elsewhere) I am again getting incomprehensible cabal errors. ("next goal: tidal (user goal) .. Could not resolve dependencies: ... (constraint from user target requires ==1.4.5)" - that's the version of the source repo, but hackage has 1.4.4. only)

Indeed, a local environment file might solve this, since it isolates the configuration. We need one local file (BootTidal.hs) anyway.

This other problem seems similar: xmonad/xmonad#199 Note that they recommend "stick to v1-install" - or make a "build script" (probably that would include a pointer to an env file)

[yaxu]

Yes, from my perspective I see only confusion from this change. Why do I have to add --lib to a package that presumably knows it's a library? Why is v2- now the default when it's still buggy and the documentation hasn't been updated to match? I'm sure there is some advantage to this change but I haven't been able to find it yet.

[yaxu]

Made an upstream issue here haskell/cabal#6391

[yaxu]

By the way @jwaldmann, it's enough to remove the tidal lines from the environment file, you don't have to delete it completely.

[yaxu]

@gbaz Doesn't that then get in the way of using tidal together with other modules, such as tidal-vis and hstexture?

[jwaldmann]

Why do I have to add --lib

this is an option to "cabal install" so it describes what will be done with result of "cabal build". From the man page:

• "--lib" : publish the library (to the user-global environment, or the env that was specified with "--package-env=ENV")
• default : publish (install) executables (only?) (in $HOME/.cabal/bin)

Ah - now I see: your point is: if the cabal file does not specifiy any executables, then "--lib" should be the default?

[jwaldmann]

It was actually easy to use an environment file: In the top of the source tree,

cabal install  --lib --package-env=tidal

will create $HOME/.ghc/x86_64-linux-8.6.5/environments/tidal, can be used (from anywhere) with

ghci -package-env=tidal -ghci-script software/music/Tidal/BootTidal.hs

@gbaz: this is not "an env file in the tidal directory"? But it's working fine.

[yaxu]

@jwaldmann that still has the same problem for me, e.g. if I do:

cabal install  --lib --package-env=tidal
echo "" >> src/Sound/Tidal/Pattern.hs
cabal install  --lib --package-env=tidal

then ~/.ghc/x86_64-linux-8.6.5/environments/tidal ends up with two tidal's with the same version and ghci won't load it.

[yaxu]

@jwaldmann Well tidal.cabal contains a 'library' section, specifying which modules to expose, etc. If I cabal v1-install it installs them, if I cabal v2-install it doesn't. This seems like a major regression, to me.

[jwaldmann]

Well indeed. For me, re-installing does not work - even when nothing is changed. I reported this in the cabal tracker haskell/cabal#6394

[gbaz]

@gbaz Doesn't that then get in the way of using tidal together with other modules, such as tidal-vis and hstexture?

The thing to do is to follow @jwaldmann 's pattern, and install all the modules you want to use together into a single package environment in a single command line, and then use that environment. Then when you want to use a new version of some things, one-shot install all that stuff together into a new environment.

The issue is really that incremental addition to environments isn't a very nice workflow, since the solver tries to be nice and isolated, unlike before, but this conflicts with the goal of incrementally building a consistent package set.

[yaxu]

@gbaz I see, but what if I'm making continual changes to a couple of packages? Is the expectation that I think of a new environment name each time I build them? I can't see the point in doing this, when I already have a package environment, except cabal keeps on breaking it, due to what seems like a bug.

[jwaldmann]

think of a new environment name each time I build them?

remove the env file, then write it anew?

[yaxu]

@jwaldmann Yep that's fine as a workaround, but when it comes to documenting things so that non-experts can try out new versions, it's unworkable really.
It's also annoying that you used to be able to pass an url to cabal install, but that doesn't seem to work any more.

[jwaldmann]

... pass an url to cabal install

this is haskell/cabal#6393 ?

[xmacex]

Looks like a relevant piece of internet https://gitlab.haskell.org/ghc/ghc/issues/17341

[yaxu]

I've been trying to develop one project that produces a binary and another that produces a library together. It seems to be impossible! I don't understand who v2-install is aimed at, but it doesn't seem to be me.

[gbaz]

So do you mean a library that depends on a binary, or a binary that depends on a library? If you spell out the architecture you can probably get a good answer. One thing to try might be to use a single .project file to tie the two together...

[yaxu]

A binary (the feedforward editor) that depends on a library (tidal).
The current tidal cabal.project already has two packages in it, tidal-parse and tidal itself, so that they are tested together. It's a good suggestion to move other related packages in too. One downside is everything is in the same github repo, sharing the same issue tracker etc. But that isn't necessarily a bad thing.

[yaxu]

I think new-style cabal just isn't useful for us and we should avoid it. cabal-env seems to be a solution but it is not released yet. https://github.com/haskell/cabal/projects/10#card-37408965