Drop support for Python 3.7 (not until January 2024)

[AlexWaygood]

Python 3.7 will reach end-of-life on 2023-06-27. For 3.6, we waited 6 months after EOL before dropping support. Do we want to do something similar this time, or are we happy with a shorter grace period?

3.7 is still a very popular Python version. On the other hand, large parts of the Python ecosystem have already dropped support for Python 3.7 in their latest versions (flake8, isort, numpy, pandas, Django).

[hauntsaninja]

6 months seems fine to me.

[JelleZijlstra]

Agree, I don't think we need to hurry to drop old versions. The maintenance load from old versions in typeshed isn't that great, and lots of people still use 3.7.

[AlexWaygood]

I don't have a strong opinion either way, I just wanted to have the discussion. 6 months is fine for me!

Edit: Well, one reason I am looking forward to dropping support for 3.7 is so that we can use PEP 570 syntax in typeshed. But that doesn't justify breaking other projects, and can definitely wait.

[Avasam]

6 months seems reasonable, it's also in line with typeshed's already established time frame of supporting obsolete stubs. And just like obsolete stubs, older versions will still be available on PyPI.
I'm really looking forward to using PEP 570 and waiting for it to properly type pywin32's C-modules.

[AlexWaygood]

I'm really looking forward to using PEP 570 and waiting for it to properly type pywin32's C-modules.

Hmm, can't you use the __ prefix for arguments? I expect we'll do some kind of codemod once we drop support for Python 3.7, that automatically converts the old-style syntax for positional-only arguments to PEP-570 syntax.

[Avasam]

Hmm, can't you use the __ prefix for arguments? I expect we'll do some kind of codemod once we drop support for Python 3.7, that automatically converts the old-style syntax for positional-only arguments to PEP-570 syntax.

Short answer is: I could, but I'd rather wait.

I didn't think of a codemod, so that removes the concern of having to do additional work to migrate.
But I still think it'll be easier to review if only a / is added in hundreds of methods (only a select few c-extension methods support named params), rather than all param names changing. It'll be easier and faster to do too.

[AlexWaygood]

We're going to face an issue when mypy 1.5 is released, due to the fact that mypy v1.5 will not support being run on Python 3.7. This will break our stubtest-stdlib job on Python 3.7, which has to be run using Python 3.7 (the --python-version 3.7 flag will not suffice, as it needs to know what exists on Python 3.7 at runtime). It will also break our mypy_test.py job on Python 3.7 -- this also needs to be run using Python 3.7 (rather than just with the --python-version 3.7 flag), or, when typechecking tensorflow, the script will install the latest version of numpy -- a version of numpy that's incompatible with Python 3.7, and will cause mypy to emit errors in CI when run with the --python-version 3.7 flag.

We have four options that I can see:

1. Don't bump our mypy pin until January 2024. This doesn't seem like a great idea.
2. Use a different mypy pin when running mypy on Python 3.7 than when running mypy on Python 3.8+. Probably feasible, but pretty icky.
3. Stop running stubtest on our py37 stubs; stop running mypy on our tensorflow stubs with Python 3.7
4. Drop support for Python 3.7 earlier than January 2024 (in practice I guess this isn't too different from (3), since (3) involves ceasing to test with Python 3.7 in many respects).

[Avasam]

About your second suggestion:
Since requirements support specifying a mypy version, you could use that.

Even the grep command that gets the mypy version in Github Actions may not need much updating since passing in both lines/restrictions should be fine (one will always be skipped due to python version requirement)

[srittau]

I think option 3 sounds reasonable to me. Our 3.7 support should be fairly stable by now, so we mainly lose regression tests for 3.7 running with Python. I'm okay with that.

[srittau]

The main difference to option 4 is that we don't actively clean up our code by removing 3.7 branches etc.

[AlexWaygood]

The main difference to option 4 is that we don't actively clean up our code by removing 3.7 branches etc.

Yes, we would "drop support" for Python 3.7 in that we wouldn't actively test with it in CI anymore, so we wouldn't be able to guarantee that our Python 3.7 stubs were bug-free. But we wouldn't remove the 3.7 branches or switch to using PEP-570 syntax etc. for another few months. That sounds okay to me as well.

[srittau]

I wonder whether we should update our documents to claim that we only support Python 3.8+, but leave the branches in and don't use PEP 570 syntax until January. We can't really guarantee Python 3.7 support anymore and this can lead to confusion due to tests not being run etc. But inofficially we'd still support it.

[AlexWaygood]

Note that we are still running py37 tests with pyright. But yeah, I agree that the current situation is somewhat confusing.

[srittau]

How to remove support. (Please amend as necessary.)

Initial PRs

• Update the documentation. (Drop support for Python 3.7 #11234)
• Remove 3.7 from remaining build scripts. (Drop support for Python 3.7 #11234)
• Remove 3.7 exclusive branches from stubs. (Remove Python 3.7 branches #11238)
• Remove obsolete stdlib modules (see VERSIONS). (Remove macpath (removed in Python 3.8) #11236)
• Update stubtest allow lists. (Deduplicate stdlib stubtest allowlist entries #10594)

Subsequent PRs

• Import from typing instead of typing_extensions: final, Final, Literal, SupportsIndex, TypedDict.
• Switch to standard pos-only argument syntax. (Switch to PEP 570 positional-only argument syntax #11237)

[robsdedude]

Sorry for pinging on such an old issue, but I'd like to bring to your attention for the next Python version drop: it's quite difficult for a given stub package to find out in which package version which python version was dropped support for. This is relevant for packages trying to keep support for older Python versions.

[AlexWaygood]

@robsdedude, our oldest supported Python version is declared here:

typeshed/pyproject.toml

Line 131 in 633a4d7

oldest_supported_python = "3.8"

IIRC, releases of our stubs packages pushed to PyPI should all have requires-python set to >= that version; so once we drop support for Python 3.8, all new versions of all our stubs packages pushed after that date will declare that they require >=3.9.

You can see in the "details" tab on PyPI that types-requests declares it requires Python 3.8+: https://pypi.org/project/types-requests/#data.

One caveat to this is that we do have a couple of stubs packages that require a more recent Python version than our global oldest supported version, however: an example of this is types-seaborn: https://pypi.org/project/types-seaborn/#data

[hauntsaninja]

curl https://pypi.org/simple/types-requests/ -H 'Accept: application/vnd.pypi.simple.v1+json' | jq '.files[] | .filename, ."requires-python" ' -r

[robsdedude]

Thanks, both. That works for me. But usability-wise this isn't great. Maybe consider having that history documented somewhere, e.g., a change log. Expecting users to crawl PyPI to get this information feels like not so great UX. Either way, as said, I've got what I need for now. So feel free to take my input or reject it as you see fit. Just some user feedback for consideration.