Split up Cabal's Travis buildjobs into stages

[hvr]

#4529 shows a problem we're starting to run into with cabal's heavy Travis jobs:

If a travis job gets killed due to timeout, it misses the opprotunity to write anything to the Travis cache.

However, there's a recently introduced staged jobs feature which should allow us to build & cache dependencies in an early stage, so that if a later stage runs into a timeout, we'd still have at least not lost the cached work, which could then be reused by a restarted job.

Moreover, this could allow us to split work into smaller logical workunits which travis could then schedule accordingly (as our current jobs seem rather unevenly distributed in terms of work-length).

[phadej]

Seems a bit difficult: travis-ci/beta-features#11 (comment)

[23Skidoo]

The announcement says that "build stages work as expected when combined with build matrix expansion".

[23Skidoo]

The announcement says that "build stages work as expected when combined with build matrix expansion".

But see travis-ci/beta-features#11 (comment).

[23Skidoo]

So I sort of got this to work in #4560 (with help from @BanzaiMan), but there are following problems:

1. Travis build stages are a bit coarse-grained for our purposes. In principle, each "test" job depends only on one "precache" job, but with build stages all "precache" jobs must finish before we get to the "test" stage. Ideally we'd want a more expressive way to describe job graphs that'd allow to expose more parallelism.
2. new-build --only-dependencies builds more than is needed by "precache". We don't want to build the Cabal lib and hackage-security in the "precache" stage, since they will not be cached in the store (Cabal is a local package, and hackage-security depends on it), but --only-dependencies can't remove reverse dependencies of a target from the install plan:

$ cabal new-build --only-dependencies --enable-tests cabal-install cabal-tests Cabal --dry-run
cabal: Cannot select only the dependencies (as requested by the
'--only-dependencies' flag), the package Cabal-2.1.0.0 is required by a
dependency of one of the other targets.

[hvr]

new-build --only-dependencies builds more than is needed by "precache".

Btw, I've wanted to have a way to better control what is built (in this case you'd want only non-local deps built). In the case of matrix builder I would need the granularity of adressing to be able to select the build goal(s) by specifying the nix-hash (which I lookup in the plan.json file).

[BardurArantsson]

Could we do away with this nonsense with a bit of sponsorship for a real CI solution? The amount of time wasted on the "free tier" of Travis-CI is absurd. (Not that I expect paid Travis-CI to be much better. Their issue tracker tells quite a tale.)

(FWIW, if it helps the community, I'm willing to donate a bit of money[1] towards it... but I think the Haskell.org might also be willing to donate a bit for a project as critical as Cabal?)

[1] Actually, also time, if need be. If there are existing servers which could be recruited for CI, I'd be quite happy to help sysadmin them.

[23Skidoo]

@BardurArantsson We talked with @gbaz about setting up a build farm for core projects like Cabal using haskell.org machines. Servers seem to be available, what's missing is developer/sysadmin time to make it happen.

[BardurArantsson]

@gbaz Is there some wiki page or something which describes the current infrastructure? I'd like to get an idea of whether it's similar enough to what I'm used to to see if I could meaningfully contribute...

EDIT: E.g. is it running docker containers, etc. etc?

[23Skidoo]

@BardurArantsson I've sent you an e-mail.

[23Skidoo]

Closing the ticket, since enabling build stages doesn't seem to be worth it currently.