Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a WASI pydebug builder #456

Merged
merged 10 commits into from
Jan 25, 2024
Merged

Add a WASI pydebug builder #456

merged 10 commits into from
Jan 25, 2024

Conversation

brettcannon
Copy link
Member

Requires 3.13+.

@brettcannon brettcannon marked this pull request as ready for review January 20, 2024 00:02
@zware
Copy link
Member

zware commented Jan 21, 2024

@brettcannon, are you ready for this to be deployed?

@brettcannon
Copy link
Member Author

@brettcannon, are you ready for this to be deployed?

I realized over the weekend I need to figure out how to migrate to wasmtime 14+, so not quite. 😅 I'll get an update to this PR and ping you when it's ready to go.

@brettcannon
Copy link
Member Author

brettcannon commented Jan 23, 2024

@zware OK, hopefully we're all good now. I took the simplest route and disabled the warning from using the old CLI.

Eventually, though, I will probably need to explicitly specify the HOSTRUNNER value, but it requires knowing what pybuilddir.txt is. Since I assume the steps in the config are not run such that a step can rely on a previous one, I will need to use branch names and what main represents to calculate everything ahead of time (which I think custom/__init_.py gives me).

Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall change LGTM, but the person who will merge the PR will have to check buildbot server logs to double check that the new config is deployed correctly ;-) Sadly, that's the only reliable test (and I'm fine with it, it's not a big deal.)

master/master.cfg Show resolved Hide resolved
master/custom/factories.py Show resolved Hide resolved
@brettcannon brettcannon marked this pull request as draft January 24, 2024 19:30
@brettcannon
Copy link
Member Author

I turned this to draft while I investigate https://buildbot.python.org/all/#/builders/1046/builds/4077/steps/11/logs/stdio to see if it's from my upgrade to wasmtime 16 or because this hasn't been merged yet.

@brettcannon brettcannon marked this pull request as ready for review January 24, 2024 22:09
@brettcannon
Copy link
Member Author

Turns out it was a bug in wasmtime (bytecodealliance/wasmtime#7813). I downgraded the version on the buildbot to 14.0.4, so this PR should be good to go!

@vstinner vstinner merged commit 72d562a into python:main Jan 25, 2024
1 check passed
@vstinner
Copy link
Member

PR merged, let's see how it goes.

@vstinner
Copy link
Member

It's kind of surprising that "wasm32-wasi" builder keeps its history, and switched from Release build to Debug build.

I scheduled manually a new build, so a Debug build: https://buildbot.python.org/all/#/builders/1046/builds/4086 So far, Python is building, it looks fine (the job was scheduled properly). Good good.

Nice work @brettcannon!

@vstinner
Copy link
Member

@brettcannon: Build failed at pythoninfo step. Now you know what's the next step, adjust the script for that step :-)

Simplified output:

make pythoninfo
 in dir /home/buildbot/buildarea/3.x.bcannon-wasi.wasi.debug/build/wasm32-wasi (timeout 1200 secs)
 argv: [b'make', b'pythoninfo']
 environment:
  HOME=/home/buildbot
  PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
  PWD=/home/buildbot/buildarea/3.x.bcannon-wasi.wasi.debug/build/wasm32-wasi
  USER=buildbot

make: *** No rule to make target 'pythoninfo'.  Stop.

https://buildbot.python.org/all/#/builders/1046/builds/4086

@brettcannon brettcannon deleted the wasi-pydebug branch January 25, 2024 19:44
@brettcannon
Copy link
Member Author

#461 should (hopefully) fix it. Turns out I had a typo in a path. I'm surprised Buildbot didn't show an error for changing the cwd to one that didn't exist, but maybe it creates it automatically?

@vstinner
Copy link
Member

I see that "wasm32-wasi 3.x" is back to green and it's a debug build.

Example of a recent build: https://buildbot.python.org/all/#/builders/1046/builds/4127

Extract of test.pythoninfo:

build.NDEBUG: build assertions (macro not defined)
build.Py_DEBUG: Yes (sys.gettotalrefcount() present)

Tests headers:

== CPython 3.13.0a3+ (heads/main-dirty:2c089b09ac, Jan 29 2024, 17:01:49) [GCC 12.2.0]
== Linux-6.1.0-13-cloud-amd64-x86_64-with-glibc2.36 little-endian
== Python build: debug
...
== cross compiled: Yes
== host python: /usr/local/bin/wasmtime run --wasm max-wasm-stack=8388608 --wasm threads=y --wasi threads=y --dir /home/buildbot/buildarea/3.x.bcannon-wasi.wasi.debug/build::/ --env PYTHONPATH=/cross-build/wasm32-wasi/build/lib.wasi-wasm32-3.13-pydebug python.wasm
== host platform: wasi-0.0.0-wasm32-32bit

Congrats @brettcannon, it works as expected!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants