Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAINT: Replace or remove CirrusCI #11859

Merged
merged 21 commits into from
Aug 3, 2023
Merged

MAINT: Replace or remove CirrusCI #11859

merged 21 commits into from
Aug 3, 2023

Conversation

larsoner
Copy link
Member

@larsoner larsoner commented Aug 2, 2023

Unfortunately on Sept 1st CirrusCI is going to limit us to 40 compute credits. Just to give an idea, this is our usage for the last few months:

Screenshot from 2023-08-02 16-20-15

So we are way over, like by a factor of 10-20. Also:

  1. Cirrus CI jobs were really useful at first to catch some nasty OpenBLAS bugs, and since then (almost a year ago) I can't think of another time it caught something another CI didn't.
  2. The current CirrusCI job is actually one of our flakiest (thanks to PyQt6 or PySide6 segfaulting sometimes) -- actually it happening in BUG: Fix bugs with corrmap computation #11858 is what made me think of making this PR. This is perhaps indicative of some error or tolerance we need to shift, but... again it's more work.
  3. GH has on their roadmap to add GH-hosted arm64 runners in Q4 2023 -- this could change or move a bit, but implementation an our end should be (famous last words!) trivial once they roll out the runners.
  4. Some other devs and I use arm64 machines daily, so at least some non-zero amount of non-CI testing happens somewhat regularly on arm64 hardware anyway.

Alternatives:

  • We could try switch to a cron job (weekly?), but this would require a non-zero amount of work to put in place and monitor, and I'm not convinced our usage would stay low enough anyway given that we're already 3x over for this month two days into it. (There were a lot of PRs in that time, but still...)
  • Another option would be to start paying, but I'd rather not.
  • Yet another option would be to use CircleCI, but this might be a lot more work.

I've taken a stab at CircleCI here just so we can see how it works, but it makes things a bit muddier in terms of our division of labor, but maybe it'll be okay. My "vote-by-PR" is thus:

  1. Remove CirrusCI for now in advance of their "reduction" (for us, probably more of a "shutdown")
  2. If CircleCI works, great, otherwise (i.e., if it doesn't work, is too slow, or uses too many credits)...
  3. ... we can live without arm64 runs for 3-6 months until GH implements the runners, at which point it'll be easy to get up and running.

@larsoner larsoner added this to the 1.5 milestone Aug 2, 2023
@drammock
Copy link
Member

drammock commented Aug 2, 2023

+1 for option 3. If it doesn't "just work" on this first try I'd say give up and wait for GitHub to offer the images. And even if it does work I think I'd still prefer those tests happening on GHA rather than Circle

@hoechenberger
Copy link
Member

Proposal 3 sounds good to me

@larsoner
Copy link
Member Author

larsoner commented Aug 3, 2023

Seems good enough:

Screenshot 2023-08-03 at 6 55 19 PM

Changing merge reqs and marking for merge when green!

@larsoner larsoner enabled auto-merge (squash) August 3, 2023 22:58
@larsoner larsoner merged commit 193e410 into mne-tools:main Aug 3, 2023
@larsoner larsoner deleted the cirrus branch August 3, 2023 23:01
snwnde pushed a commit to snwnde/mne-python that referenced this pull request Mar 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants