Clean up datasets and tables left behind by CI

[j-bennet]

In this repo, there's a problem with macOS CI jobs. They tend to hang and not finish, and when they do, they leave behind datasets in dask-bigquery-testing project.

This is not urgent, but we need to come up with a plan of how to either avoid this problem, or automate the cleanup.

In BigQuery, it is possible to set expiration for tables within a dataset:

https://cloud.google.com/bigquery/docs/updating-datasets#table-expiration

There doesn't seem to be a similar feature for the dataset. Deleting tables would only solve the problem partially, since each CI job also creates a new temporary dataset.

• Xref macOS runners getting stuck on Job is about to start running on the hosted runner actions/runner#2609.

cc @ncclementi @jrbourbeau

[kgantchev]

Hi, sorry if I'm intruding, but a lot of people seem to be having issues with GitHub's macOS runners. Have you considered giving FlyCI a try? FlyCI offers M2 runners for both public and private repos. FlyCI's runners are more reliable, affordable, and faster than GitHub's. There is chat support via Discord for free tier users and on-demand support for those who purchase the support packages.