Dropping Python 2 support

[srittau]

As far as I'm aware, mypy is the last user of our Python 2 stubs. mypy plans to eventually drop Python 2 support, although no fixed date has been set. See python/mypy#12237 for details. Once mypy has dropped support, we can also consider dropping it. This would ease maintenance and we could simplify our release process quite a bit.

I'm opening and pinning this ticket for discussion and objections.

[srittau]

And as a side note: Python 2 support in third-party packages is being slowly phased out anyway. We dropped all Python 2-only packages that are not required by other packages (with only enum34 and ipaddress left) and we stopped supporting Python 2 in many packages where upstream stopped support for Python 2.

[srittau]

The first step was to drop most Python-2-only third party packages. As a second step I suggest to drop the remaining Python-2-only packages and remove Python 2 support of third-party packages that have separate Python 2 and Python 3 versions. This affects:

• six
• ipaddress
• enum34

This also means dropping Python 2 support for the following packages as they depend on the packages above:

• cryptography (which is replaced by inline stubs anyway)
• pyvmomi
• boto
• paramiko
• pysftp

Python 2 support always remains in the current version as published on PyPI, but I think not offering any updates is a reasonable compromise with lowering our maintenance effort.

[Akuli]

I think Python 2 support for six stubs should be dropped last, as six users typically care about Python 2.

[srittau]

We had one non-maintenance commit since the re-org more than a year ago, but two maintenance commits. types-six 1.16.10 is not going away, but the maintenance burden remains. I believe that six is nowadays often used, just because removing it from Python 3 code is effort.

Edit: One of my motiviations is to get rid of @python2 special handling here and in the stub uploader.

[hauntsaninja]

Would it make sense to remove six altogether? Seems unlikely that six's API would change much and it feels a little weird and potentially confusing for users to have types-six not support Python 2.

[srittau]

I have no strong feelings about six. Last release was less than a year ago, though.

[srittau]

The next step would be to drop Python 2 support from the remaining third-party stubs. This affects:

• atomicwrites
• backports.ssl_match_hostname
• certifi
• characteristic
• chardet
• click-spinner
• colorama
• croniter
• Deprecated
• docopt
• emoji
• entrypoints
• first
• mypy-extensions
• polib
• pyaudio
• python-dateutil
• python-gflags
• python-nmap
• pytz
• retry
• simplejson
• singledispatch
• tabulate
• termcolor
• toml
• xxhash

This would ease maintenance a bit more, since we don't have to take care not to break Python 2 when updating those stubs and we could remove Python 2 support from the stub uploader. As before, the current version of the stubs will remain on PyPI indefinitely and can be used for people still using Python 2, but updates to the stubs (for Python 2) won't be possible anymore.

[AlexWaygood]

Would it make sense to remove six altogether? Seems unlikely that six's API would change much and it feels a little weird and potentially confusing for users to have types-six not support Python 2.

I think we should keep six around for a little while longer. Dropbox is apparently not using Python 2 anymore, but (reading between the lines) is evidently still using six:

• Add back six.moves.http_cookies.Morsel #7617
• Add back six.moves.configparser.Error #7616
• Add missing definitions to six.moves.http_client #7615

So it seems likely that removing six stubs would be fairly disruptive.

[srittau]

Yes, I believe that even users that have dropped Python 2 might still use six as replacing it with native Python 3 calls is work and a risk to introduce new bugs. (Although it's also technical debt.)

[AlexWaygood]

The next step would be to drop Python 2 support from the remaining third-party stubs.

Doing this would also mean we could enable several flake8-pyi checks that currently don't run on third-party stubs (because some of them still support Python 2).

[JukkaL]

six is still one of the top 10 most downloaded packages on PyPI, according to https://hugovk.github.io/top-pypi-packages/. I wouldn't be surprised if six will continue to be used even after years from now, since tech debt cleanup tends to get postponed. Continuing to maintain the stubs seems useful to me.

[srittau]

This only leaves support for the stdlib, which we can drop after mypy drop Python 2 support (python/mypy#12237).

[ethanhs]

I was running the mypy test suite and I discovered that I am the only one foolish enough to keep Python 2 kicking around on their system to get picked up by mypy's test suite.

I ran

pytest mypy/test/testpythoneval.py::PythonEvaluationSuite::python2eval.test

And all of the Python 2 tests run against typeshed failed due to an incompatibility with __or__ and __ior__ between MutableSet and AbstractSet.

This seems to have broken in mypy's 0.950 release, I haven't tracked it to which commit. I know it is after this mypy typeshed sync

I personally don't really think it is worth the effort to fix this, I just uninstalled Python 2, but if someone else cares enough to fix it I figured I should report it.

[AlexWaygood]

It was probably this PR that broke things:

• Backport more Self-related changes to Python 2 #7166.

The PR was made to unblock some changes over at our linter, flake8-pyi. We no longer run flake8-pyi on the Python 2 stdlib, however, so we could probably just revert that PR fairly easily.

[srittau]

I'm fine with reverting, although I'd say we wait on mypy to decide whether they still want to support Python 2 in 0.960 and just remove the @python2 tree if they don't.

[AlexWaygood]

Mypy is now ready for us to remove support for Python 2: python/mypy#12237 (comment)

Who wants to do the honours?