Clarify Python 2 support

[codingjoe]

I was reading into Pip's Python support statement and it somehow leaves me with more questions than answers.

I am afraid I might cause some frustration for maintainers, contributors and users over time. It might also promote a false sense of security.

My main questions are:

• Can a user rely on Pip working on Python 2 going forward? Yes or no?
• Are patches that address Problems only apparent in Python 2 being reviewed and accepted?
• If so, by whom?
• Do I need to make sure that my code is Python 2 compliant when I want to contribute to Pip?
• If not, who will make the necessary adjustments?
• Are Python 3 only patches accepted?
• Is the test suite continuously run against Python 2.7.
• If so, did we consider the terrible impact that has on our carbon footprint and do we care?

[pfmoore]

Can a user rely on Pip working on Python 2 going forward? Yes or no?

Yes. At some point the latest version of pip will not support Python 2, but older versions will be available.

Are patches that address Problems only apparent in Python 2 being reviewed and accepted?

Yes, currently.

If so, by whom?

The pip maintainers. As with any volunteer effort, maintainers will prioritise based on what interests them. Over time, fixes for Python 2 only issues will inevitably be of less interest (not just to the maintainers, but also to the rest of our user base) and the time it takes for Python 2 only issues to get attention will increase. But there's no commitment or service level here, so that's just a fact of life that users have to take into account.

Do I need to make sure that my code is Python 2 compliant when I want to contribute to Pip?

Yes, until Python 2 support is formally dropped.

If not, who will make the necessary adjustments?

Not relevant, see above.

Are Python 3 only patches accepted?

Not until Python 2 support is formally dropped.

Is the test suite continuously run against Python 2.7.

Python 2.7 is part of our CI suite, yes. It will remain so until we drop Python 2 support formally.

If so, did we consider the terrible impact that has on our carbon footprint and do we care?

To run one additional Python version? I doubt that's a "terrible" additional burden, to be honest. There are likely to be far better ways to address environmental issues than modifying pip's test suite.

[codingjoe]

Can a user rely on Pip working on Python 2 going forward? Yes or no?

Yes. At some point the latest version of pip will not support Python 2, but older versions will be available.

I see and that will be reflected in the version number and release notes I presume.
We may want to explicitly write in the docs, that Python 2 is currently supported until stated otherwise.

Are patches that address Problems only apparent in Python 2 being reviewed and accepted?

Yes, currently.

If so, by whom?

The pip maintainers. As with any volunteer effort, maintainers will prioritise based on what interests them. Over time, fixes for Python 2 only issues will inevitably be of less interest (not just to the maintainers, but also to the rest of our user base) and the time it takes for Python 2 only issues to get attention will increase. But there's no commitment or service level here, so that's just a fact of life that users have to take into account.

OK, I am not 100% the current wording put a big enough emphasis on this potential issue.

Do I need to make sure that my code is Python 2 compliant when I want to contribute to Pip?

Yes, until Python 2 support is formally dropped.

I see, also somewhat inconvenient, since installing Python 2 will become increasingly inconvenient on some operating systems.

This might also make a great addition to the docs.

If not, who will make the necessary adjustments?

Not relevant, see above.

Are Python 3 only patches accepted?

Not until Python 2 support is formally dropped.

This too might make a great addition to the docs.

Is the test suite continuously run against Python 2.7.

Python 2.7 is part of our CI suite, yes. It will remain so until we drop Python 2 support formally.

If so, did we consider the terrible impact that has on our carbon footprint and do we care?

To run one additional Python version? I doubt that's a "terrible" additional burden, to be honest. There are likely to be far better ways to address environmental issues than modifying pip's test suite.

I wouldn't underestimate the carbon footprint of test VMs is terrible to begin with, but moreover this does not only affect pip. The handling of Python 2 Pip sets a president. Other projects might be inclined to hand on to Python 2 builds for longer too. In fact that how I got read the Pip Python 2 policy to being with.

I for myself am fairly certain that my monthly server power consumption vastly outgrows my domestic annual consumption, and I'm no Jeff Bezos. This is true for most of "us" developers I'd say.

[dstufft]

I would make one caveat to the response to the Python 3 only question-- Basically in the general case what Paul wrote is true, that we expect patches to work on all supported versions of Python. However, in certain cases we may be fine with certain features being gated behind certain Python versions when there is a good reason to do so. Those patches would still have to keep Python 2 functioning (sans that particular feature/change) but the feature itself wouldn't have to function in Python 2.

This isn't really related to Python 3 at all, since that question is really just a more specific phrasing of the general "does every change have to be equally supported across all supported Python versions".

An example of the kind of change I'm talking about, I believe that distutils (and thus setuptools) on later versions of Python supports a -j option to enable building multiple objects in parallel. If we exposed this in pip, we obviouslly couldn't expose it in a version of Python that didn't support the -j option, so we'd likely gate that feature to only versions of Python that did support -j (with the behavior on older Pythons varying depending on the specific circumstance).

That however should be considered a special case for when we need that particular escape hatch, and that in the common case we expect all patches to function equivalently across our entire set of supported Pythons (and certainly require that our CI passes for each patch across all the Pythons we test against).

[pfmoore]

The documentation already covers what we support here.

I see no reason to document the "potential issue" about developer attention. This is a volunteer based project and as such, there are no guarantees of any sort of response. Anyone using open source who isn't aware of that possibility has a bigger issue to address than just pip :-)

The point about PRs needing to work on Python 2 is already covered in the docs.

So I think the various points are already covered, although obviously we'd consider PRs to clarify things if you think it's necessary (no guarantees that we'd agree, but feel free to propose something).

I'm not going to get into the carbon footprint question here, as I don't think it's appropriate for the pip issue tracker.

[chrahunt]

For people interested in things impacting carbon footprint, please see #4768, #4497, #7294, and #7279 which probably have the most impact.

[codingjoe]

Thanks for all the detailed responses and considerations. I think I understand it now. I'll sleep over it and see if I can conjure up a sentence or two to improve the documentation. Again big thanks.

[pradyunsg]

IMO all these are good questions and we should clarify these more directly in the language we have in our documentation about support.

[pradyunsg]

Noting that we'd probably want to do this, especially since packages that we might want to depend on in the future (eg. tenacity et al) aren't easy to use while we're still depending on Python 2.

We ought to figure out and clarify pip's situation in case dependencies drop support (eg. what we'd do when requests or certifi drops Python 2 support). We'd probably also be helped if we identify some metrics to chart when we'd drop Python 2 support (remembering https://en.wikipedia.org/wiki/Goodhart%27s_law).

re metrics: https://discuss.python.org/t/metrics-for-quantifying-python-2-usage/1991

[hugovk]

pip 21.0 will drop support for Python 2 in January 2021: #6148 (comment)

Is there anything still needed for this issue?

[codingjoe]

Nope, I believe this clarifies things.