Drop Python 3.8, add 3.13 #694

cretz · 2024-11-25T15:23:14Z

What was changed

Drop Python 3.8 support and confirm Python 3.13 support

Specifically this included:

Update CI GH workflow to do 3.9/3.13 instead of 3.8/3.12
Update binary build and bench GH workflows to do 3.13
Remove outdated workarounds for older Poetry and older Python versions in CI
Create a new branch in features repo named python-version-upgrade to pass CI
Update minimum Python versions in README
Update code to support protobuf 5.x
- Basically this was just doing a poetry update for protobuf and some minor tweaks, but it means we now test proto 3.x and proto 5.x in CI, but not 4.x
Update cibuildwheel and align grpcio and `grpcio-tools
Add protoc install step to wheel building since it was removed as part of pyproject.toml: Removed protoc-wheel dependency #684 and we didn't notice until now it broke wheel building
Update fix-wheel.py script to unzip wheel to fix it instead of wheel unpack which was failing on macos for strange hash mismatch
Updated wheel and bridge to have minimum 3.9 support instead of 3.8
Exclude os.stat from sandbox restrictions since as of gh-106922: Support multi-line error locations in traceback (attempt 2) python/cpython#112097 it appears that is invoked even when displaying errors
Update test teardown to, in 3.9, swallow certain loop close errors and disable pytest-asyncio's creation of a post-teardown loop creation since they were breaking in strange ways
Skip all unfinished-handler tests in 3.9 since they were reliably causing issues (instead of sometimes flaking as in all other Python versions). This needs to be investigated when the unfinished-handler test flakes are investigated in general.

After the PR is approved, I will change the required passing CI steps to be 3.9/3.13.

Checklist

Closes [Feature Request] Drop 3.8 support, confirm 3.13 support #672

cretz · 2024-12-18T20:26:27Z

This is now ready for review. But we may not merge until early next due to holiday period availability.

dandavison

Nice! LGTM.

dandavison · 2025-01-10T13:23:12Z

.github/workflows/build-binaries.yml

@@ -32,7 +32,7 @@ jobs:
          submodules: recursive
      - uses: actions/setup-python@v5
        with:
-          python-version: "3.12"
+          python-version: "3.13"


So just to confirm, the -cp38- part of the wheel name in PyPi means it is compatible with 3.8+, but in fact the wheel was built using 3.12 (and will now be built with 3.13 and will contain a -cp39- name fragment).

We tell cibuildwheel to build for these minimums: cp39-win_amd64, cp39-manylinux_x86_64, cp39-manylinux_aarch64, cp39-macosx_x86_64, and cp39-macosx_arm64. And we tell PyO3 to compile with ABI compat to abi3-py39. It is those two tools that make sure during the build-binary step to compile for 3.9 minimum. cibuildwheel sometimes runs inside a container, and it's a certain container they build within to get older libc compat and such (ref https://github.com/pypa/manylinux for Linux for instance). May have to reference those projects for more details or look at the logs of the builds at https://github.com/temporalio/sdk-python/actions/workflows/build-binaries.yml.

dandavison · 2025-01-10T13:29:29Z

.github/workflows/ci.yml

@@ -93,7 +82,7 @@ jobs:
        env:
          TEMPORAL_TEST_PROTO3: 1
        run: |
-          poetry add "protobuf<4"
+          poetry add --python 3.9 "protobuf<4"


Worth a comment explaining the --python 3.9 here?

I admit I do not remember exactly, but I think it was saying there wasn't compatibility with some of the other dependencies

dandavison · 2025-01-10T13:34:47Z

temporalio/worker/workflow_sandbox/_restrictions.py

@@ -207,6 +207,12 @@ def nested_child(path: Sequence[str], child: SandboxMatcher) -> SandboxMatcher:
    ``True`` and this matcher is on ``children`` of a parent.
    """

+    exclude: Set[str] = frozenset()  # type: ignore


We can now use the usual lower-case type names for collection types.

True, though inconsequential here I think. But would be opposed to a general porting.

dandavison · 2025-01-10T13:41:18Z

temporalio/worker/workflow_sandbox/_restrictions.py

@@ -939,7 +956,9 @@ def __init__(self, *args, **kwargs) -> None:
    def __getattribute__(self, __name: str) -> Any:


nit on existing code: double-underscore prefixes in Python should be (rarely) used, only when name-mangling is explicitly required. I don't think it's ever appropriate for a local variable in a function or a parameter name.

Sure. I think at the time I wrote this years ago, typeshed defined this signature as __name at https://github.com/python/typeshed/blob/2116158891a3c88b97038617855ce216499833f9/stdlib/builtins.pyi#L115 and I think I needed to match it. It was only a year ago that they updated it: python/typeshed#11250.

dandavison · 2025-01-10T13:52:15Z

tests/worker/test_workflow.py

+def skip_unfinished_handler_tests_in_older_python():
+    # These tests reliably fail or timeout in 3.9
+    if sys.version_info < (3, 10):
+        pytest.skip("Skipping unfinished handler tests in Python < 3.10")


Yeah, I still haven't pinned this down. I think it's mostly or always under time-skipping? This PR shows that in fact they do not seem to reliably fail any longer: #724 Basically I was trying to study the problem and of course they stopped failing.

cretz force-pushed the drop-3.8 branch 8 times, most recently from c761cff to 48c5bd5 Compare November 25, 2024 21:59

cretz force-pushed the drop-3.8 branch 5 times, most recently from a200bb5 to c7326f4 Compare December 11, 2024 16:22

cretz mentioned this pull request Dec 11, 2024

[Feature Request] Check Protobuf 5.x Support #497

Open

cretz force-pushed the drop-3.8 branch 16 times, most recently from 794c62b to bf725ec Compare December 13, 2024 16:47

cretz force-pushed the drop-3.8 branch 13 times, most recently from 0db2839 to fd57843 Compare December 18, 2024 16:37

Drop Python 3.8, confirm 3.13

abb91b7

cretz force-pushed the drop-3.8 branch from fd57843 to abb91b7 Compare December 18, 2024 20:08

cretz marked this pull request as ready for review December 18, 2024 20:26

cretz requested a review from a team as a code owner December 18, 2024 20:26

dandavison approved these changes Jan 10, 2025

View reviewed changes

cretz added 2 commits January 10, 2025 08:59

Merge branch 'main' into drop-3.8

50283db

Minor CI fix

0314c28

cretz force-pushed the drop-3.8 branch from f2626e5 to 0314c28 Compare January 10, 2025 15:23

cretz merged commit 7af48a7 into temporalio:main Jan 10, 2025
13 checks passed

cretz deleted the drop-3.8 branch January 10, 2025 16:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Drop Python 3.8, add 3.13 #694

Drop Python 3.8, add 3.13 #694

cretz commented Nov 25, 2024 •

edited

Loading

cretz commented Dec 18, 2024

dandavison left a comment

dandavison Jan 10, 2025

cretz Jan 10, 2025

dandavison Jan 10, 2025

cretz Jan 10, 2025

dandavison Jan 10, 2025

cretz Jan 10, 2025

dandavison Jan 10, 2025

cretz Jan 10, 2025 •

edited

Loading

dandavison Jan 10, 2025

		@@ -939,7 +956,9 @@ def __init__(self, args, *kwargs) -> None:
		def __getattribute__(self, __name: str) -> Any:

Drop Python 3.8, add 3.13 #694

Drop Python 3.8, add 3.13 #694

Conversation

cretz commented Nov 25, 2024 • edited Loading

What was changed

Checklist

cretz commented Dec 18, 2024

dandavison left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cretz Jan 10, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cretz commented Nov 25, 2024 •

edited

Loading

cretz Jan 10, 2025 •

edited

Loading