Ansible 5: raise python requirement from >=2.7 to >=3.8

[dmsimard]

Summary

As reported in issue ansible/ansible#76414, currently attempting to install ansible=5.0.0 with less than python3.8 will result in an error because ansible-core now requires >=3.8:

> podman run --rm -it  python:2.7 pip install ansible
Resolved "python" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/python:2.7...
Getting image source signatures
Copying blob 09b6f03ffac4 done  
Copying blob b32f6bf7d96d done  
Copying blob fd4b47407fc3 done  
Copying blob 7e2b2a5af8f6 done  
Copying blob dc3f0c679f0f done  
Copying blob 6f4489a7e4cf done  
Copying blob af4b99ad9ef0 done  
Copying blob 39db0bc48c26 done  
Copying blob acb4a89489fc done  
Copying config 68e7be49c2 done  
Writing manifest to image destination
Storing signatures
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting ansible
  Downloading ansible-5.0.0.tar.gz (38.4 MB)
     |████████████████████████████████| 38.4 MB 33.8 MB/s 
ERROR: Could not find a version that satisfies the requirement ansible-core<2.13,>=2.12.0 (from ansible) (from versions: 0.0.1a1, 2.11.0b1, 2.11.0b2, 2.11.0b3, 2.11.0b4, 2.11.0rc1, 2.11.0rc2, 2.11.0, 2.11.1rc1, 2.11.1, 2.11.2rc1, 2.11.2, 2.11.3rc1, 2.11.3, 2.11.4rc1, 2.11.4, 2.11.5rc1, 2.11.5, 2.11.6rc1, 2.11.6, 2.11.7rc1)
ERROR: No matching distribution found for ansible-core<2.13,>=2.12.0 (from ansible)
WARNING: You are using pip version 20.0.2; however, version 20.3.4 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.

We are currently requiring >=2.7 in setup.py:

    python_requires='>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*',
    packages=['ansible_collections'],
    include_package_data=True,
    install_requires=[
        'ansible-core>=2.12.0,<2.13',
    ],

We should raise the requirement to python3.8 to match ansible-core and thus avoid the error -- pip dependency solving should allow for the installation of Ansible 4.

We might need to yank 5.0.0 once an eventual 5.0.1 is released with the new requirement.

[gundalow]

We should ensure this is clear in the release announcements as well as major_change in release notes/porting_guide

[tadeboro]

An alternative would be to change the ansible-core requirement to ansible-core>=2.11.0,<2.13. This way, Python < 3.8 users can still get updated collections and run them using ansible-core-2.11.*

[dmsimard]

We should ensure this is clear in the release announcements as well as major_change in release notes/porting_guide

The release announcement for 5.0.0 mentions the following:

• ansible-core-2.12 now requires Python 3.8 or greater on the controller (where you run ansible-playbook).

It's also at the top of the ansible-core changelog as well as in the porting guide.

[dmsimard]

An alternative would be to change the ansible-core requirement to ansible-core>=2.11.0,<2.13. This way, Python < 3.8 users can still get updated collections and run them using ansible-core-2.11.*

This is something we can consider doing as well. It should be relatively safe to assume (though we should check) that collections are already testing against both 2.11 and 2.12 by now.

I would had liked to find this issue before the release of 5.0.0 because it's not an insignificant change to do post-release. Oh well...

We should think ahead to see what this would look like for Ansible 6 and beyond... some key dates:

• The expected EOL of ansible-core 2.11 is roughly ~6 months before 2.12 in this table so around November 2023.
• The Ansible 6 roadmap is not up yet but the release would trail ansible-core 2.13 which is expected 2022-05-23.
• Ansible 6 would be maintained for the usual six months with support ending around November 2022.
• Ansible 7 would trail ansible-core 2.14 which is expected around 2022-11-07.

With that in mind, we could (in theory) continue requiring ansible-core>=2.11 for both Ansible 5 and Ansible 6 which would delay lifting the requirement on >=python3.8 of the ansible package itself for a year or so.

I probably wouldn't continue longer than that at the risk of increasing maintenance burden too much.

[dmsimard]

In preparation for discussing this at the meeting, unless mistaken I see two potential outcomes that we can vote on:

1. Keep Ansible 5 requirement on ansible-core>=2.12 and raise the python requirement to >=3.8 (up from >=2.7)
2. Lower the Ansible 5 ansible-core requirement down to >=2.11 and keep the python requirement intact (until we decide to raise it, i.e, Ansible 6 or 7)

[felixfontein]

Im +1 for keeping the ansible-core>=2.12 requirement, and adding a python >= 3.8 requirement to the Ansible package.

We did introduce the ansiblecore>=2.12.X requirement to make sure that if you install an Ansible package, you also get at least the latest ansible-core release (that was available when that Ansible package was built). (We voted on this in one of the early community meetings, if someone wants to search for it: ansible/community#539)

[jillr]

I prefer 2 so folks aren't surprised, but think we should include a clear notice in 5 and plan to raise it for 6 (not 7)

ETA: The porting guide notice from 4.0 satisfies my user notification concerns https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_4.html#other

[cidrblock]

I prefer option 1, it is a major release so a change in core version + python version is not out of place.

If we haven't should we state in the future, major releases of the ansible pkg will require the most recent version of ansible-core? (if we haven't previously) This could help avoid the issue in the future

[dmsimard]

We discussed the issue during the community meeting and while we would appreciate feedback from additional steering committee members (ping @ssbarnea, @acozine , @thaumos, @Ompragash, @gundalow) we had the following results:

13:36:56 <@dmsimard> VOTE: Keep Ansible 5 requirement on ansible-core>=2.12 and raise the python requirement to >=3.8 (up from >=2.7)
13:37:01 <felixfontein> +1
13:37:13 <@jillr> +1
13:37:14 <cidrblock[m]> +1
13:37:17 <@andersson007_> +1
13:37:17 <@dericcrago> +1
13:37:20 <@dmsimard> +1
13:37:34 <@cybette> +1
13:37:42 <berkhan> +1
13:39:30 <cyberpear> -0
13:39:31 <samccann> +1

[tadeboro]

I am also +1 on raising the Python requirement.

[felixfontein]

The above is missing a -0 by @jamescassell and a +1 by @samccann. Also @Ompragash is so far not a steering committee member (but of course you can still voice your opinion :) ).

[felixfontein]

My arguments for +1:

• We did agree on always depending on the latest version of ansible-base/-core available for an Ansible release already (https://meetbot.fedoraproject.org/ansible-community/2020-07-01/community_working_group_meeting.2020-07-01-18.04.html);
• Also allowing ansible-core 2.11 as a dependency breaks the assumption by users that installing the latest Ansible release also gives them the latest ansible-core release (without explicitly having to do pip install --upgrade) available when that release was made (or a newer one);
• Changelog, porting guide and the docsite will be completely inaccurate for Ansible 5 since the parts on ansible-core only match the latest ansible-core release, and not ansible-core 2.11.

[dmsimard]

The above is missing a -0 by @jamescassell and a +1 by @samccann.

Indeed, their vote came later and I failed at copy/paste -- edited my comment.

Also @Ompragash is so far not a steering committee member (but of course you can still voice your opinion :) ).

Offtopic: we should update https://github.com/orgs/ansible-community/teams/steering-committee/members so it's accurate.

[hswong3i]

+1 for ansible 5.0.0 keep python >= 3.8, so sync with its ansible-core 2.12.0 python >= 3.8 requirement.

P.S. IMHO, that may be even better if able to keep as python >= 3.6 (see ansible/ansible#76257 (comment)), because now most LTS OS are not supported by Ansible 5.0.0 (e.g. our main downstream RHEL 7 and 8 are not officially supported; only RHEL 9 Beta with python == 3.9 could works)...

[noonedeadpunk]

+1, but agree that only ubuntu 20.04 and debian 11 are able to support ansible-core 2.12 as of today (the same will go for ansible 5.0.0).

Despite centos-stream shipping python38, it;s not really usable because of missing libselinux bindings that are kind of required for ansible.

[gundalow]

VOTE: Keep Ansible 5 requirement on ansible-core>=2.12 and raise the python requirement to >=3.8 (up from >=2.7)

+1

Offtopic: we should update https://github.com/orgs/ansible-community/teams/steering-committee/members so it's accurate.

@felixfontein @dmsimard steering-committee has been updated, thanks for pointing this out.

[ssbarnea]

+1 For raising python requirements to 3.8+

[cybette]

+1 for Keep Ansible 5 requirement on ansible-core>=2.12 and raise the python requirement to >=3.8 (up from >=2.7)

Plenty of good reasoning already mentioned, plus from a comms point of view, it's clearer to share the message "python >=3.8 for Ansible 5" than to sink into confused messaging of different -core versions as previously communicated, and we might have a repeated drama à la Ansible 3/core/base

[dmsimard]

ansible 5.0.1 (hotfix with python>=3.8 requirement) has been released: https://groups.google.com/g/ansible-announce/c/t0JoB6evpt8/m/krFCR5o9CgAJ

I've yanked 5.0.0 and confirmed that py<3.8 correctly falls back to ansible 4.9.0 instead of erroring out.

We also now provide a galaxy-requirements.yaml file and can suggest it as an alternative for users to continue using ansible-core 2.11 while getting the set of collections that we ship in ansible 5.

Another alternative that we can also consider in the future was proposed by bcoca: https://github.com/bcoca/acd.

[geerlingguy]

Heh, I'm starting to get a lot of support requests across a ton of repos for users of CentOS7/RHEL7, Debian 10, etc. who are now running into installation issues (regardless of 5.0.0 / 5.0.1). It might be nice to have some place to point people towards with instructions for either (a) installing Ansible 'community' 4.x, or (b) how to upgrade to a newer Python on the most popular Linux distros.

[hswong3i]

Heh, I'm starting to get a lot of support requests across a ton of repos for users of CentOS7/RHEL7, Debian 10, etc. who are now running into installation issues (regardless of 5.0.0 / 5.0.1). It might be nice to have some place to point people towards with instructions for either (a) installing Ansible 'community' 4.x, or (b) how to upgrade to a newer Python on the most popular Linux distros.

@geerlingguy I don't really think upgrade system default Python >= 3.8 could be simple for most legacy LTS OS (see ansible/ansible#76257 (comment)).

One the other hand, since ansible 5.0.1 released with enforce python>=3.8, legacy LTS OS with python>=3.6,<3.8 should now simply pointing to ansible 4.9.0 without issue?

IMHO we could loweransible>=5.0.1 minimal dependency to python>=3.6, so most LTS OS now a day could be supported. Case could be similar as https://github.com/pyca/cryptography:

• https://cryptography.io/en/latest/changelog/#v3-4-3 try to upgrade to rust >=1.45.0, but generate issue for most existing OS
• https://cryptography.io/en/latest/changelog/#v3-4-5 therefore lower the minimal rust to >=1.41.0, improves compatibility with system-provided Rust on several Linux distributions

[dmsimard]

Heh, I'm starting to get a lot of support requests across a ton of repos for users of CentOS7/RHEL7, Debian 10, etc. who are now running into installation issues (regardless of 5.0.0 / 5.0.1).

@geerlingguy thanks for the feedback. Could you link to some of those issues you are seeing ?

I've also seen ansible/ansible#76458 in addition to the one I mentioned at the beginning of this issue.

It might be nice to have some place to point people towards with instructions for either (a) installing Ansible 'community' 4.x, or (b) how to upgrade to a newer Python on the most popular Linux distros.

I'll ask around.

[geerlingguy]

@dmsimard - It seems like most of those support requests were coming in before 5.0.1 came out (and maybe 5.0.0 is still there on pywheels or something?). I just re-tested everything on a few fresh installs of Pi OS and Debian (before posting Install Python 3.9 on Raspberry Pi OS or Debian 10 (for Ansible or other uses), and it seems that 4.9.0 is installed correctly now without the scary warnings, if you just have Python 2.7 or 3.7.

[dmsimard]

I'm going to re-open this for the time being, it's being reported that 5.0.0 is still problematic with older releases of pip: https://twitter.com/zhenech/status/1467592374981378052

I've asked about it in #ansible-community, it looks like we may need to do something more than just yanking 5.0.0.

[dmsimard]

Yanking a package on PyPI was implemented back in 2019 (thanks @tadeboro). That explains why pip versions older than 19.2 don't "follow" the yank.

There is so far a consensus that deleting 5.0.0 is the right thing to do since PyPI wouldn't let us re-upload a "new" 5.0.0.
It means that pip install ansible==5.0.0 (explicitly) will stop working but 5.0.1 has been made available.

pip install ansible would result in:

• ansible==4.9.0 (w/ ansible-core 2.11) on python<3.8
• ansible==5.0.1 (w/ ansible-core 2.12) on python>=3.8

We can afford to wait until tomorrow so more people can weigh in.

[hswong3i]

+1 delete Ansible 5.0.0, which align with why Ansible 5.0.1 got released.

[felixfontein]

+1 for deleting 5.0.0.

[ssbarnea]

Yanking 5.0.0 is no brainer, the reason why 5.0.1 was released was to fix a serious packaging mistake in 5.0.0 in the end. Some projects would have decided to do it in a 5.0.0post1 version. Go for it, I doubt anyone would benefit from having 5.0.0 on pypi.

[tadeboro]

+1 for deleting a 5.0.0

[jillr]

+1 to delete 5.0.0

[gundalow]

+1 to delete 5.0.0

[felixfontein]

@ssbarnea we already yanked 5.0.0 end of last week; the problem is that pip < 19.2 does not know about yanking and will still try to install 5.0.0 - that's why we want to actually delete the 5.0.0 release instead of just keeping it marked as yanked. This will cause installations which pinned to 5.0.0 to fail in the future, but will resolve all the problems with older pip versions (assuming they can handle python_requires).

[acozine]

+1 to delete - let's make it a better user experience for everyone

[dmsimard]

5.0.0 has been deleted and I've sent an email out about it: https://groups.google.com/g/ansible-announce/c/t0JoB6evpt8/m/UHzmz8-2CgAJ

[felixfontein]

Closing since this has been implemented.