Update github actions

[ppigazzini]

Split in server and worker workflows.
Set up the server workflow with the SW versions used on PROD.
Set up the worker workflows to test on Linux with few python versions and on Windows msys2.

See:
https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
https://github.com/marketplace/actions/setup-msys2

[vdbergh]

Would it be possible to test the worker also on MacOSX? There is some Mac specific code in the worker and it would be nice to be able to test it.

[ppigazzini]

That was my intention but I got an error when adding MacOSX in the workflow matrix.

[ppigazzini]

@vdbergh now it works :) https://github.com/ppigazzini/fishtest/actions/runs/3445013827/jobs/5748241705

[Disservin]

What error ? We have macos in the stockfish CI if you are interested take a look there only the avx2 compile doesn’t work for macOS ci runner

[ppigazzini]

My first implementation had python 3.5 too in the matrix (now dropped because I would like to switch to pathlib), the macos action works now for python 3.6-3.11
https://github.com/ppigazzini/fishtest/actions/runs/3445037852

[ppigazzini]

I got the error when using only one workflow for linux and macos, it seems that we are hitting a rate limit for url: https://api.github.com/repos/official-stockfish/books/git/trees/master

https://github.com/ppigazzini/fishtest/actions/runs/3445092768/jobs/5748408359

test_worker_script_with_no_args (test_worker.workerTest.test_worker_script_with_no_args) ... ok

======================================================================
ERROR: test_item_download (test_worker.workerTest.test_item_download)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/runner/work/fishtest/fishtest/worker/games.py", line 146, in requests_get
    result.raise_for_status()  # also catch return codes >= 400
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/runner/work/fishtest/fishtest/worker/.eggs/requests-2.28.1-py3.11.egg/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
uests.exceptions.HTTPError: 403 Client Error: rate limit exceeded for url: https://api.github.com/repos/official-stockfish/books/git/trees/master

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/runner/work/fishtest/fishtest/worker/test_worker.py", line 22, in test_item_download
    games.download_from_github("README.md", ".")
  File "/Users/runner/work/fishtest/fishtest/worker/games.py", line 396, in download_from_github
    tree = requests_get(
           ^^^^^^^^^^^^^
  File "/Users/runner/work/fishtest/fishtest/worker/games.py", line 154, in requests_get
    raise WorkerException("Get request to {} failed".format(remote), e=e)
games.WorkerException: Get request to https://api.github.com/repos/official-stockfish/books/git/trees/master failed

----------------------------------------------------------------------
Ran 6 tests in 20.487s

[ppigazzini]

I don't understand why only the macos workflow is hitting that rate limit...

[ppigazzini]

The issue comes from the way how GitHub counts requests for rate limit. For unauthorized requests, it limits by IP. All macOS VMs have the same IP address because of infrastructure. GitHub complains about rate limit regardless of the source of request because GA workflow can be used to spam API too.

actions/runner-images#602

[vdbergh]

Perhaps we can fake the rate limit request during testing?

[ppigazzini]

The macos workflow still fails when using the github token, even with only one python version.
https://dev.to/github/the-githubtoken-in-github-actions-how-it-works-change-permissions-customizations-3cgp

      - name: Run worker tests
        run: |
          ./test_worker.sh
        env:
          GITHUB_TOKEN: ${{ github.token }}
          # GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

[vdbergh]

@ppigazzini Do you have the link of that action?

[ppigazzini]

https://github.com/ppigazzini/fishtest/actions/

I wonder why we are hitting the link https://api.github.com/repos/official-stockfish/books/git/trees/master with this code:

    def test_item_download(self):
        try:
            games.download_from_github("README.md", ".")
            self.assertTrue(os.path.exists(os.path.join(".", "README.md")))
        except KeyError:
            pass

EDIT_000: because the function is in truth def download_from_books_repo():

REPO_URL = "https://github.com/official-stockfish/books"

def download_from_github(item, testing_dir):
    """Download item from FishCooking to testing_dir"""
    tree = requests_get(
        github_api(REPO_URL) + "/git/trees/master", timeout=HTTP_TIMEOUT
    ).json()
    for blob in tree["tree"]:
        if blob["path"] == item:
            print("Downloading {} ...".format(item))
            blob_json = requests_get(blob["url"], timeout=HTTP_TIMEOUT).json()
            with open(os.path.join(testing_dir, item), "wb+") as f:
                f.write(b64decode(blob_json["content"]))
            break
    else:
        raise WorkerException("Item {} not found".format(item))

[vdbergh]

It is strange that the github token does not work since it is officially supported.

[ppigazzini]

Doubts:

• we are hitting an external repo

the token's permissions are limited to the repository that contains your workflow

• we are using run:

EDIT_000: switching to the fishtest repo that runs the workflow (with the github token passed to the env) failed too...

    def test_item_download(self):
        try:
            games.REPO_URL = "https://github.com/ppigazzini/fishtest"
            games.download_from_github("README.md", ".")
            self.assertTrue(os.path.exists(os.path.join(".", "README.md")))
        except KeyError:
            pass

ERROR: test_item_download (test_worker.workerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/runner/work/fishtest/fishtest/worker/games.py", line 146, in requests_get
    result.raise_for_status()  # also catch return codes >= 400
  File "/Users/runner/work/fishtest/fishtest/worker/.eggs/requests-2.28.1-py3.7.egg/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: rate limit exceeded for url: https://api.github.com/repos/ppigazzini/fishtest/git/trees/masteres/master failed

[ppigazzini]

This failed randomly too:

      - name: Run worker tests
        run: |
          cd worker && python setup.py test
        env:
          #GITHUB_TOKEN: ${{ github.token }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

EDIT_000: failed setting a high level env too.

name: CI worker posix

on: [push, pull_request, workflow_dispatch]
env:
  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

[vdbergh]

Maybe we can download things using raw.githubusercontent.com ? I think this is not affected by api credits.

EDIT: I do not think there is a risk getting outdated content in this way.

[vdbergh]

I made a PR. Have a look!