Migrate off of travis-ci

[njsmith]

Well, the incredible journey is finally happening: travis's new billing plan only gives OSS projects 1000 minutes/month. Currently our travis builds use ~30 minutes each, so that means we can only do ~30 builds/month, which is obviously inadequate. (Apparently we used ~700 minutes just in the last two days.)

Fortunately we've already moved a bunch of stuff off; unfortunately the stuff that remains is a bit harder to migrate. That is:

• PyPy builds: these are probably pretty easy. Two of them are PyPy nightly branches, which we could probably switch over trivially. One is a PyPy release build, which we currently get from Travis, but the one they offer is outdated anyway, and it would be pretty trivial to pull down an official release from the pypy site. We could probably reuse the same code we currently use for nightly builds, just changing the download URL. GHA also provides PyPy builds.

• CPython nightly builds: used to be really annoying on Github Actions, but now there's an easy solution; just need to switch it on: https://github.com/deadsnakes/action

• A build with an early Python 3.6 version (currently requires SETUPTOOLS_USE_DISTUTILS=1). GHA provides 3.6.7+ and Deadsnakes only provides the latest 3.6. This would be nice to keep as it helped us fix cbcb624 in the past.

• Two builds that use nested virtualization to test against OSes/kernels that aren't otherwise available on GHA. For this... we're kind of screwed. GHA has no plans to support this: Enable nested virtualization  actions/runner-images#183. I'm pretty sure Azure Pipelines doesn't either. We're using SourceHut now: Migrate Alpine, Fedora, FreeBSD CIs to SourceHut #1811

I guess we should start by migrating the easy ones -- if we only run the nested virtualization jobs on Travis, that will cut our usage by ~4x, which will at least buy us some time.

[smurfix]

So instead of nested virtualization, somebody might simply (or in fact not-so-simply) spin up a test runner on Real Hardware?

[njsmith]

@smurfix in theory yes, in practice, maintaining custom CI infrastructure is a massive time sink.

[smurfix]

I know … though if we only need to spin a couple of specific tests instead of "infrastructure", something simple and hand-rolled might actually work with minimal ongoing maintainance.

[pquentin]

GitHub Actions actually support pypy2 and pypy3 on Linux, macOS and Windows: https://github.com/actions/setup-python#usage. They even support the latest version, 7.3.2 right now.

[takluyver]

By my reading, the 1000 minutes are a one-off trial thing, not a monthly allowance. One of my colleagues got an email saying that he would have 1000 credits per month, which is only 100 minutes - which is at odds with the blog post and the docs.

I can't work out exactly what the plan is, but I think there's a good chance that when trio gets 'upgraded' to the new plan, this will become urgent very quickly.

[njsmith]

Random note for future reference, in case sourcehut doesn't work out: It looks like gitlab's CI has some kind of support for running on github repos, and they run jobs on GCP, which in general supports nested virtualization (though I don't know if it's enabled for the specific configuration that gitlab uses).

[njsmith]

Status update: as of right now, it looks like we've used up about half of the credits they gave us on Nov. 30, so at current rate we've got about a month left.

[njsmith]

Once #1842 is done, the only thing the python-trio/trio project will have on Travis is the 3.6.1 build. (And some of our satellite projects are still on travis, but they have much lower usage so it's not as urgent.) I feel like there's some reason @pquentin said we needed to keep testing on 3.6.1, but I forget what it was :-). Does it have unique coverage?

[pquentin]

We don't have to test 3.6.1 at all costs. It's just that early minor versions often have bugs that affect us and testing 3.6.1 helped fix cbcb624.

(Why 3.6.1? Because 3.6.0 isn't available on Travis)

[pquentin]

@njsmith Should we drop 3.6.1 and Travis?

[njsmith]

I don't have a strong opinion either way. I doubt that testing 3.6.1 is providing *much* value, but maybe it's providing some; and running one quick job on Travis doesn't use up *much* of our quota, but I guess it uses up some. So... I'm like, 55% leaning yes, let's drop it?

…

On Mon, Jan 4, 2021 at 11:47 PM Quentin Pradet ***@***.***> wrote: @njsmith <https://github.com/njsmith> Should we drop 3.6.1 and Travis? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1785 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEU42E4ZNQHWPIMBUIOFTTSYK7ZDANCNFSM4THAA5KQ> .

-- Nathaniel J. Smith -- https://vorpus.org <http://vorpus.org>

[pquentin]

OK, +1 for removing then, since the Travis queues tend to be full as they're gradually reducing build capacity. Which means we'll have to wait hours before getting a result for 3.6.1. At least it should not be required.

[njsmith]

Note in case we need it in the future: apparently the GHA macOS runners do support nested virtualization: https://github.com/vmactions/freebsd-vm

[njsmith]

The 3.6.1 build is now failing because of a pip bug: pypa/pip#9879

One more reason to drop it, I guess.

[njsmith]

I've disabled travis running on this repo in the travis-ci.com console, and removed it from the required checks.