gl and cabal new-build don't mix

[RyanGlScott]

I recently attempted to switch the Travis infrastructure in this repo to use cabal new-build, which caused everything to break in unique and mysterious ways.

On GHC 7.8 and 7.10, running cabal sdist simply fails with:

$ (cd "." && cabal sdist)
cabal: The package 'gl' requires Cabal library version >=1.12.0 && >=1.24 but
no suitable version is installed.

I haven't the foggiest idea where that >=1.12.0 constraint is coming from, as gl.cabal requires >=1.24.

On GHC 8.0 and later, the build fails at the custom setup:

$ (cd "." && cabal sdist)
[1 of 6] Compiling Registry         ( Registry.hs, dist/setup/Registry.o )
[2 of 6] Compiling Parser           ( Parser.hs, dist/setup/Parser.o )

Parser.hs:18:1: error:
    Could not find module ‘Text.XML.HXT.Core’
    Use -v to see a list of the files searched for.
   |
18 | import Text.XML.HXT.Core
   | ^^^^^^^^^^^^^^^^^^^^^^^^

This is despite the fact that custom-setup does, in fact, depend on hxt, which exports this module.

I'm way out of my element in debugging this... @hvr, any thoughts?

[RyanGlScott]

That Parser.hs error may not be specific to cabal new-build, as I just noticed that you can trigger it by taking a fresh clone of the gl repo and running cabal clean (with all dependencies installed):

$ cabal clean
[1 of 6] Compiling Registry         ( Registry.hs, dist/setup/Registry.o )
[2 of 6] Compiling Parser           ( Parser.hs, dist/setup/Parser.o )

Parser.hs:18:1: error:
    Could not find module ‘Text.XML.HXT.Core’
    Use -v to see a list of the files searched for.
   |
18 | import Text.XML.HXT.Core
   | ^^^^^^^^^^^^^^^^^^^^^^^^

[hvr]

This sounds a lot like a symptom/manifestation of haskell/cabal#4293

[RyanGlScott]

The first issue, it turns out, isn't unique to gl, as it also popped up in this lens commit where I bumped cabal-version to >=1.18.

[ekmett]

If we smash all the separate modules used by the setup into Setup.hs destroying any attempt at code organization does this issue go away?

[hvr]

Just to make sure we're on the same page: the trouble as I see it, is that we don't have cabal new-sdist yet, and cabal sdist uses the old-build codepaths. So in order for cabal sdist to work, you'd need to do the things the old-build style (i.e. make sure setup-depends are in the pkg-db accessed by cabal sdist; e.g. by running cabal sdist in a sandbox and passing the setup-depends line contents to cabal install .... manually; or you could try with cabal install --deps if the lib components don't needs significantly more deps than the custom-setup component).

[ekmett]

Blech.

[RyanGlScott]

Ah, now I understand why cabal sdist is barfing—it's because it's not installing a recent enough version of Cabal for the custom Setup.hs script! This would explain why any custom Setup.hs breaks on GHC 7.0—it's because GHC 7.0 is bundled with Cabal-1.10.2.0, but apparently Cabal-1.12 is required as a minimum (at least, according to that obscure error message).

@hvr, is there a Cabal ticket for tracking this?

[RyanGlScott]

Never mind, I answered my own question: haskell/cabal#4293 is also apparently the cabal new-sdist ticket. So really, this whole thing hinges on adding support for that.

Alas, there's really nothing more we can do on gl's side, so I'll close this for now.

[RyanGlScott]

In order to make this work, we'll likely need to monkey around a bit with autogen-modules, as suggested in this Cabal-2.4 warning:

[6 of 6] Compiling Main             ( /home/rgscott/Documents/Hacking/Haskell/ghc-8.6/gl/dist-newstyle/build/x86_64-linux/ghc-8.6.1/gl-0.8.0/setup/setup.hs, /home/rgscott/Documents/Hacking/Haskell/ghc-8.6/gl/dist-newstyle/build/x86_64-linux/ghc-8.6.1/gl-0.8.0/setup/Main.o )

/home/rgscott/Documents/Hacking/Haskell/ghc-8.6/gl/dist-newstyle/build/x86_64-linux/ghc-8.6.1/gl-0.8.0/setup/setup.hs:38:5: warning: [-Wdeprecations]
    In the use of ‘sDistHook’
    (imported from Distribution.Simple, but defined in Distribution.Simple.UserHooks):
    Deprecated: "SDist hooks violate the invariants of new-sdist. Use 'autogen-modules' and 'build-tool-depends' instead."
   |
38 |   , sDistHook = \p ml h f -> case ml of
   |     ^^^^^^^^^

/home/rgscott/Documents/Hacking/Haskell/ghc-8.6/gl/dist-newstyle/build/x86_64-linux/ghc-8.6.1/gl-0.8.0/setup/setup.hs:44:25: warning: [-Wdeprecations]
    In the use of ‘sDistHook’
    (imported from Distribution.Simple, but defined in Distribution.Simple.UserHooks):
    Deprecated: "SDist hooks violate the invariants of new-sdist. Use 'autogen-modules' and 'build-tool-depends' instead."
   |
44 |        generateAPI l >> sDistHook simpleUserHooks p' ml h f
   |                         ^^^^^^^^^

[RyanGlScott]

@typedrat, you claimed that you managed to "fix" gl's use of sDistHook in haskell/cabal#5389 (comment). Do you recall how you accomplished this?

[typedrat]

Just the trivial autogen-modules was enough to get it to sdist correctly, but a new way to trigger the code gen will be needed.

[RyanGlScott]

Right. I suppose I'm wondering what the best hook to use in lieu of sDistHook is. We'd likely want something that can automatically populate the list of autogen-modules (since the idea of doing that by hand makes me shudder), although I'm not sure which hook would be best suited for that.

[typedrat]

I’m at a show tonight, but I’ll try and get you a good answer soon. Alexis Williams Sent from Polymail ( https://polymail.io/?utm_source=polymail&utm_medium=referral&utm_campaign=signature )

…

On Mon, Oct 08, 2018 at 6:08 PM Ryan Scott < Ryan Scott ( Ryan Scott ***@***.***> ) > wrote: Right. I suppose I'm wondering what the best hook to use in lieu of sDistHook is. We'd likely want something that can automatically populate the list of autogen-modules (since the idea of doing that by hand makes me shudder), although I'm not sure which hook would be best suited for that. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( #17 (comment) ) , or mute the thread ( https://github.com/notifications/unsubscribe-auth/ABAj_TNk-TIUbk5rWUpz3F3Fmpzw3YGpks5ui_b3gaJpZM4Re0gQ ).

[ekmett]

Maybe it is time to bite the bullet and make the build a preprocess that generates the thing we upload rather than a custom Setup. This would reduce the install footprint for users quite a bit (as they wouldn't need hxt, etc.) at the cost of introducing an extra step in the distribution process. On Mon, Oct 8, 2018 at 9:11 PM Alexis Williams <notifications@github.com> wrote:

…

I’m at a show tonight, but I’ll try and get you a good answer soon. Alexis Williams Sent from Polymail ( https://polymail.io/?utm_source=polymail&utm_medium=referral&utm_campaign=signature ) On Mon, Oct 08, 2018 at 6:08 PM Ryan Scott < Ryan Scott ( Ryan Scott < ***@***.***> ) > wrote: > > > > Right. I suppose I'm wondering what the best hook to use in lieu of sDistHook > is. We'd likely want something that can automatically populate the list of > autogen-modules (since the idea of doing that by hand makes me shudder), > although I'm not sure which hook would be best suited for that. > > > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub ( > #17 (comment) ) , or mute > the thread ( > https://github.com/notifications/unsubscribe-auth/ABAj_TNk-TIUbk5rWUpz3F3Fmpzw3YGpks5ui_b3gaJpZM4Re0gQ > ). > > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#17 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASmEVDZH9Se6AEl-xvYC9fQnOMK_kgmks5ui_ezgaJpZM4Re0gQ> .

[RyanGlScott]

Ping @typedrat. It seems that sDistHook was finally removed in Cabal-3.0.0.0, so as far as I can tell, gl is impossible to build with new-build going forward. What is the alternative?

[RyanGlScott]

While it would be good to know the answer to #17 (comment), it is worth noting that gl has since switched away from using custom Setup scripts as of gl-0.9, so this is no longer a blocker.

[RyanGlScott]

At this point, the only thing preventing gl from using haskell-ci to generate a new-build-based .travis.yml file is haskell-CI/haskell-ci#283.

[RyanGlScott]

haskell-CI/haskell-ci#283 has been fixed. I've submitted #20, which should finally bring gl's CI up to modern standards.