GH-110793: add GitHub Actions job that runs tests after installing

[FFY00]

• Issue: Add GitHub Actions job that runs the tests in an installed Python #110793

[FFY00]

The job is now successful, this is ready for review. I'd still wait a little bit before merging, to see if there is any discussion on the issue.

[sobolevn]

Why do we assume that 4 is a good number (other build jobs also do this)?

Should we switch to -j$(nproc) / $(sysctl -n hw.ncpu)?

[FFY00]

I think that's the number of threads for Linux and Windows.

In this PR, I just copied the other jobs, so this comes from there. We can do the improvements you are suggesting (would also be nicer when running locally with act for eg), but it should probably be in a different PR.

[hugovk]

Agreed something for another PR, but worth investigating.

I've sometimes done a test on my fork with different numbers of jobs in a matrix.

For example, with macOS: make buildbottest TESTOPTS="-j${{ matrix.jobs }} -uall,-cpu"

• hugovk@05b0451
• https://github.com/hugovk/cpython/actions/runs/6361030227

Shows ~4 is about right:

[hugovk]

build_ubuntu and build_ubuntu_installed have a lot of duplicate lines: ~60/66 are identical.

build.yml is getting too long, it's over 700 lines!

If we go for this, let's do them as reusable workflows:

• https://docs.github.com/en/actions/using-workflows/reusing-workflows

Examples:

• https://github.com/python/cpython/blob/main/.github/workflows/reusable-docs.yml
• https://github.com/python-pillow/Pillow/blob/main/.github/workflows/wheels.yml
  • https://github.com/python-pillow/Pillow/blob/main/.github/workflows/wheels-linux.yml
  • https://github.com/python-pillow/Pillow/blob/main/.github/workflows/wheels-macos.yml

Possibly also build_ubuntu_ssltests.

[hugovk]

Let's also consider the CI time.

This job is 26m, compared to 14m for the standard Ubuntu one. Plus we recently added 3 x CIFuzz which are 21-29m:

https://github.com/python/cpython/actions/runs/6505589398

[FFY00]

This job is 26m, compared to 14m for the standard Ubuntu one. Plus we recently added 3 x CIFuzz which are 21-29m:

I noticed make test adds a couple options, I think that's why it takes less time.

./python -u -W default -bb -E -m test --fast-ci --timeout= --dont-add-python-opts

Let's try adding them to the new job.

build_ubuntu and build_ubuntu_installed have a lot of duplicate lines: ~60/66 are identical.

build.yml is getting too long, it's over 700 lines!

If we go for this, let's do them as reusable workflows:

Agreed. Do you want to implement that in this PR or in a followup one?

[hugovk]

It would be good in this PR.