GitHub rate limit exceeded in tests

[aplowman]

E.g. https://github.com/hpcflow/hpcflow-new/actions/runs/8922555319/job/24504946388

This happens in tests that retrieve demo data from GitHub.

Need to handle this.

[aplowman]

Looks like GitHub's REST API limit for unauthenticated users is 60 per hour.

Setting GITHUB_TOKEN within GHA, we get 1000 per hour, which should be sufficient.

Our problem could be due to a secondary rate limit. E.g. there is a limit on the number of concurrent requests, which we might be hitting from multiple test jobs running at the same time.

So to solve:

• Set GITHUB_TOKEN for test jobs in our GHAs workflows
• Implement a retry loop on HTTP errors when calling fsspec.core.url_to_fs in demo-data related methods; this function is calling GitHub's REST API for us.

[aplowman]

My fixes didn't work; the MacOS tests are still encountering rate limits (after 10 tries with exponential backup plus random jitter). Although now only 4 tests failed on the latest release whereas it was 5 before. It looks like a MacOS specific issue because the Mac runners all use the same IP address: actions/runner-images#602. I don't understand why there is still an issue though.

[aplowman]

Fixed this in #677 by marking the tests as "expected to fail" on MacOS.