Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CI test for cross-builds #127432

Open
FFY00 opened this issue Nov 29, 2024 · 0 comments
Open

Add CI test for cross-builds #127432

FFY00 opened this issue Nov 29, 2024 · 0 comments
Labels
build The build process and cross-build easy infra CI, GitHub Actions, buildbots, Dependabot, etc. type-feature A feature request or enhancement

Comments

@FFY00
Copy link
Member

FFY00 commented Nov 29, 2024

Feature or enhancement

Proposal:

Cross-builds depend on a fair bit of sysconfig internals, it would be good have a test in the CI to ensure they work.

This would have prevented #127178 (comment).

I don't think there's a need to run the full test suite, we should be fine running just the sysconfig, site, and test_embed tests. It would also make sense to run the tests in the build directory, and after installation.

The test should consist of:

  • Building a host Python, specifying a custom prefix (eg. workdir/host-python)
  • Installing the host Python
  • Configuring a new build using --with-build-python=workdir/host-python/bin/python, and also using a custom prefix (eg. workdir/cross-python)
  • Running the tests in the build directory (./python -m test test_sysconfig test_site test_embed)
  • Installing the cross-build
  • Running the tests again, but with the installed Python (workdir/cross-python/bin/python -m test test_sysconfig test_site test_embed)

I think we should be fine with Linux-only, but Windows would also be welcome. Additionally, possibly, we could also use a different architecture in the cross-build, like jit.yml does for "Emulated Linux".

- name: Emulated Linux
if: runner.os == 'Linux' && matrix.architecture != 'x86_64'
# The --ignorefile on ./python -m test is used to exclude tests known to fail when running on an emulated Linux.
run: |
sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ./llvm.sh ${{ matrix.llvm }}
export PATH="$(llvm-config-${{ matrix.llvm }} --bindir):$PATH"
./configure --prefix="$(pwd)/../build"
make install --jobs 4
make clean --jobs 4
export HOST=${{ matrix.architecture }}-linux-gnu
sudo apt install --yes "gcc-$HOST" qemu-user
export QEMU_LD_PREFIX="/usr/$HOST"
CC="$HOST-gcc" \
CPP="$HOST-gcc --preprocess" \
HOSTRUNNER=qemu-${{ matrix.architecture }} \
./configure --enable-experimental-jit ${{ matrix.debug && '--with-pydebug' || '' }} --build=x86_64-linux-gnu --host="$HOST" --with-build-python=../build/bin/python3 --with-pkg-config=no ac_cv_buggy_getaddrinfo=no ac_cv_file__dev_ptc=no ac_cv_file__dev_ptmx=yes
make all --jobs 4
./python -m test --ignorefile=Tools/jit/ignore-tests-emulated-linux.txt --multiprocess 0 --timeout 4500 --verbose2 --verbose3

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

@FFY00 FFY00 added type-feature A feature request or enhancement build The build process and cross-build easy infra CI, GitHub Actions, buildbots, Dependabot, etc. labels Nov 29, 2024
@FFY00 FFY00 changed the title Add test for cross-builds Add CI test for cross-builds Nov 29, 2024
gianfrancomauro added a commit to gianfrancomauro/cpython that referenced this issue Nov 30, 2024
Fixes python#127432

Add CI test for cross-builds.

* Add a new test function `test_cross_build` in `Lib/test/test_embed.py` to run `test_sysconfig`, `test_site`, and `test_embed` tests.
  * Use `subprocess` to run the tests in the build directory and after installation.
* Add a new workflow file `.github/workflows/cross-build.yml` to define the CI test for cross-builds.
  * Include steps to build a host Python, install it, configure a new build, and run the tests.
  * Run the tests in the build directory and after installation.
  * Use a different architecture in the cross-build.

---

For more details, open the [Copilot Workspace session](https://copilot-workspace.githubnext.com/python/cpython/issues/127432?shareId=XXXX-XXXX-XXXX-XXXX).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build The build process and cross-build easy infra CI, GitHub Actions, buildbots, Dependabot, etc. type-feature A feature request or enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant