The new pricing model for Travis CI #5028
Comments
|
Yeah I think @wiredfool likes GitHub Actions … I'm game. |
See dbhi/qus. It was created and it is used precisely for migrating ARM, PPC64 and S390X jobs to GHA. |
Thanks for that, it works quite well. However, the performance is unsurprisingly an order of magnitude slower; arm64 takes ~30m, ppc64le ~55m, s390x ~45m. Since most of that time is spent installing dependencies, I think it would be reasonable to migrate these jobs to https://github.com/python-pillow/docker-images (since they run in a Docker container anyway). The build and test steps alone take ~10m, which is much more reasonable. I'm not sure if it is feasible to migrate the arm64 wheel builds to GHA as they compile all dependencies from scratch instead of downloading from Ubuntu, so I wouldn't be surprised to see those take multiple hours. |
To quantify this, here is the usage for the past month (6 Oct to 4 Nov): That is ~140,000 credits per month, assuming it is all Linux (it is more if accounting for macOS builds). Looking at just the arm64 wheels on Travis, one build takes ~1h54m total, so 10,000 credits is enough for just 8 builds! They do at least mention the possibility of a renewable amount:
|
Correct. qemu-user is 5-10 times slower than native execution or KVM. At the same time, in qemu-user some features (lower level signals, etc.) might not be complete, compared to qemu-system. Hence, combining qemu-user and containers might not fit all the use cases. However, when it does, it comes quite handy.
Agree. Installing dependencies should be decoupled from building and testing a project. Images with dependencies can be updated weekly or monthly, while the project is expected to be tested more frequently. |
|
I've created python-pillow/docker-images#94 to migrate the docker-images job. |
|
Two weeks after the Travis CI change I'm now out of credits, their UI both says they will be replenished and that they won't be replenished. I've emailed them asking for credits, but nothing yet. So it'd be great to move ahead with this! We now have PRs for all five repos:
Reviews/merge especially welcome on #5029. And @wiredfool please could you add Thanks! |
|
I've added credentials to the docker-images repo. Any existing credentials aren't going to work anymore, so the automated build on travis is going to fail going forward, if it ever has credits to run again. |
|
Thanks! There's no |
|
I just got a sort-of notification (blue dot on my profile icon) on GitHub about "GitHub Container Registry" beta: long link to docs Looks like it is a free alternative to Docker Hub which recently introduced new usage limits of 200 container image requests per 6 hours and is replacing GitHub Packages Docker. It seems to be easier to set up than before (maybe even easier than Docker Hub), so while I don't think this is needed right now, it might be worth looking into at some point in the future if the limit becomes an issue.
I didn't realize there was a cron job as it was not specified in |
|
Yeah, the GitHub registry may be useful in the future, but we don't need it right now as GitHub has been working with Docker Hub so GHA isn't affected by the rate limits: actions/runner-images#1445 (comment) That's not the case with Travis CI, where the limits do apply: https://travis-ci.community/t/docker-hub-you-have-reached-your-pull-rate-limit/10517. So it's good we've moved our Docker stuff off of Travis! Yup, we had cron enabled for the three main repos, you can only set it via the web UI for Travis. The settings:
It would be good to add cron to these three GHA once all merged. GHA is good that you can actually define POSIX cron syntax. We should set them to run at different times/days, would be nice to try and pick "quiet" times, but we're of course a global team so that might not exist :) I'll merge #5029 tomorrow if there's no further feedback. |
Hint: you can also set 'dispatch' event keys, in order to programmatically sequence workflows in the same repo or across repos. For example:
Note: the default GITHUB_TOKEN cannot trigger multiple subsequent workflows. That's a limitation for preventing unexperienced users creating infinite loops. However, you can achieve it if you use a regular token instead. |
|
I see the credits have now begun counting down at https://travis-ci.com/organizations/python-pillow/plan:
I'll email Travis CI support requesting OSS credits for the python-pillow organisation. In the meantime, to save precious credits I've turned off the remaining crons at https://travis-ci.com/github/python-pillow/Pillow/settings and https://travis-ci.com/github/python-pillow/pillow-wheels/settings. Let's also use For reference, for my own account, after being out of credits and unable to test anything on Travis for 16 days, they've added 50k whilst they're waiting (11 days and counting) for "managerial approval": https://twitter.com/hugovk/status/1334054612672901121 So I won't be too surprised if we're out of credits for the January release, which would prevent building aarch64 wheels (the rest are done by GHA). It might be that we can build those on a personal account, or add them to the release later on. Let's see how it goes. |
|
Sent:
Received:
|
|
Seven days later, no reply beyond an autoreply, we're down 950 credits, from 3200 to 2250:
At this rate, we'll be out in 2.5 weeks, roughly Christmas day 🎅 🎄 🎁 I'm not getting my hopes up, someone else was told "credit allocation for OSS projects is on hold as per directives from management". |
|
@hugovk Only loosely following but I'd say we don't need Travis anymore, and by that I mean I'm not waiting around for them to give us credits. I'd rather find an alternative, even if we have to pay a Travis competitor. Can you tell me if we've completely moved to GitHub Actions or if we need to do more work and once we move there are we done? Or do we need to look for additional, possibly paid alternatives? Thanks |
|
We're in quite a good situation now. We've moved pretty much everything we can and everything essential over to GitHub Actions. We've kept testing of three CPU architectures on Travis for this repo, because they're not available elsewhere. These are quite new to Travis anyway. We could probably ditch two and just keep the big-endian one, that's the most useful. Similarly, we've moved all wheel building to GHA except some (aarch64) not available elsewhere. These are additions this summer, so probably not the biggest loss if we need to skip them. All the other repos are migrated 👍 I'm not too worried if these things suddenly stop working and have pretty much given up on Travis. There may be alternatives to fill the gaps, it'll need further investigation (e.g. #5028 (comment)) to find sonething feasible, and see what other projects do. |
|
In python-pillow/pillow-wheels#172 (comment) @nulano suggested temporarily switching off Travis to conserve the last remaining credits. That's a very good idea, as they're going to run out anyway soon. At https://travis-ci.com/github/python-pillow/Pillow/settings and https://travis-ci.com/github/python-pillow/pillow-wheels/settings I've flipped these left two toggles off:
Let's remember to turn the pillow-wheels one back on for the release, and when/if we ever get credits. |
I've created #5088 and python-pillow/docker-images#98 to move the three non-x86 builds in this repo to GHA Docker tests. I haven't investigated whether that approach is reasonable for the Arm64 wheel builds yet, but I'm guessing not. |
|
Update: After triggering a wheel build (#4997 (comment)), gone from 2250(?) to 1100:
Should be good for one more build before we're out. I sent Travis support a ping 3 days ago, nothing yet. |
|
Amazing, after ignoring my ticket ping on 6th January, three days ago Travis sent me this!
https://travis-ci.com/organizations/python-pillow/plan now shows:
My confidence in Travis is low after this six-month episode, and I've not received any updates for other projects I'd contacted them about. Plus the necessity to keep an eye on and request credits again. But I'll re-enable Travis for our repos here, and let's try it for a bit and see how it pans out. |
|
And now re-enabled for pushed branches and PRs at https://travis-ci.com/github/python-pillow/pillow-wheels/settings (our only remaining Travis dependency), and enabled a monthly cron:
|
|
@hugovk Yeah reads more like "OOPS we got called out, and now here's some credits" … but yeah, give it a whirl, while at the same time not relying too heavily on it, I'd say. |
|
Let's close this now, we've moved nearly everything to GitHub Actions, and Travis CI is ticking over okay for where we are using it. https://app.travis-ci.com/organizations/python-pillow/plan currently looks like:
Which is the same as April #5028 (comment) 🤷 OSS-only tab:
|
|
Another ~5 months later, the numbers haven't changed, and the same as April 2021: https://app.travis-ci.com/organizations/python-pillow/plan
|
|
Just to state it in this issue - if Travis CI ever does run out of credits, python-pillow/pillow-wheels#247 can be used to move the aarch64 jobs to GitHub Actions, albeit with a much longer execution time. |
|
In python-pillow/pillow-wheels#367 @radarhere noticed Travis CI is playing up again.
Plan A is to get Travis CI working again. At https://app.travis-ci.com/github/python-pillow/pillow-wheels, it says:
"More options" > "Requests" confirms that recent builds have been cancelled: "Owner python-pillow does not have enough credits." 🤷 OK, so let's go to the Plan page. Nope! 🤷
Trying again makes no difference. 🤷 Same thing for the link to "Consume paid credits for OSS setting". 🤷 Well, I see the same error on my own page at https://app.travis-ci.com/github/hugovk/pillow-wheels, but do have "Credits" although zero "OSS only credits". Neither are negative... 🤷
Anyway, I also see this:
No Pillow listed. 🤷 Maybe I need to "Review and add your authorized organizations". But https://github.com/apps/travis-ci/installations/12696639 shows it's already installed for the wheels repo: 🤷
At this point I give up and will contact Travis CI support... |
|
Good news! Quickly fixed by Travis support:
Pushing a new build now works and the red banner has been removed: However, I can't see the plan page, nor trigger or restart builds. I've replied to ask about that. |
After some back and forth, the response was that only admins can see the plan. And I could trigger and restart builds after logout/login/resync. |
|
Cross post from #7418 (comment): 
|
Travis CI announced a new pricing model today:
It's not entirely clear how this affects open source, but zopefoundation/meta#38 (comment) looks like the best interpretation:
It looks like we should move more stuff from Travis CI to GitHub Actions.
cc @python-pillow/pillow-team
Pillow
This repo, and the most active.
pillow-wheels
The biggest build, less active. Essential for releases.
With great timing, @nulano (thank you!) had already done the work and opened this PR to move all the jobs to GHA, except for Arm64 which need to stay there: python-pillow/pillow-wheels#169.
Let's get that reviewed and merged.
docker-images
A big build, less active.
Let's shift it all over to GHA. We'll need to set credentials as secrets to upload to Docker Hub. Or alternatively upload to GitHub Packages.
Also
Least active, might as well move these over too, should be easy enough.
Checklist
The text was updated successfully, but these errors were encountered: