The latest version of requests (2.29.0) does not support urllib3 2.0.0

[mdmintz]

The latest version of requests (2.29.0) does not support urllib3 2.0.0

urllib3 2.0.0 was just released: https://github.com/urllib3/urllib3/releases/tag/2.0.0

But currently requests 2.29.0 has a range bound on it: <1.27 and >=1.21.1 for urllib3.

If you try to install a package that has urllib3==2.0.0 as a dependency (while using the latest version of requests), there will be errors:

<PACKAGE> depends on urllib3==2.0.0
requests 2.29.0 depends on urllib3<1.27 and >=1.21.1

Expecting requests to support the latest version of urllib3.

(For Python 3.7 or newer)

[nateprewitt]

Hi @mdmintz, this is intentional as discussed here. We'll move the pin once we get more data points on any issues in the major version bump. We have a responsibility to keep the majority of the Python ecosystem stable during the transition.

[hartwork]

@nateprewitt I would like to note that if requests blocks urllib3 2.0.0 and everyone has requests installed for something, most people will not be able to get any new data about urllib3 2.0.0 in practice because it just cannot be installed. Is that picture missing something?
Regarding the blast radius mentioned at #6430 (comment) I would like to note that the responsibility is with every project using urllib3 directly, not with requests. Why would it be requests' responsibility?

[nateprewitt]

Many, if not most, projects pull urllib3 into their closure through another dependency. Pinning transitive dependencies, while good practice through something like a lockfile, is not widely observed. We have a decade of issues on this project as empirical data on that. Often times, users aren't even aware they're using Requests, let alone urllib3. Those are the cases where this can go wrong very easily.

We'll likely move the pin within the next week, but pushing a major version update out isn't something we'll do lightly. Giving urllib3 the opportunity release before we pull everyone else in is the safest approach and least amount of pain for operation teams.

[hartwork]

Hi @nateprewitt, thanks for your reply, I appreciate it! 👍

Many, if not most, projects pull urllib3 into their closure through another dependency. Pinning transitive dependencies, while good practice through something like a lockfile, is not widely observed. We have a decade of issues on this project as empirical data on that. Often times, users aren't even aware they're using Requests, let alone urllib3. Those are the cases where this can go wrong very easily.

What I'm reading here is that you are concerned about users that (1) do not use urllib3 directly and (2) use urllib3 indirectly and (3) have urllib3 pulled in without a semver guard like "<2" and (4) the code using urllib3 would break with urllib3 2.0.0.
Can you confirm that's the correct reading of your scenario?

Could you name example tickets where I could have a closer look at reports like that? Thanks!

[lwoerdeman]

This is something that shouldn't be upgraded in a minor version of requests, because it causes breaking changes.

For example, we saw our elasticsearch client pull in the 2.0.0 version of urllib3 and it caused this failure on our machine. To resolve, we're adding a constraints.txt file, but not something that should be included in a minor release of requests.

ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2k-fips  26 Jan 2017. See: https://github.com/urllib3/urllib3/issues/2168

[nateprewitt]

Landon, that's ultimately a decision for users of Requests to decide. We're doing a very temporary overlap of this pin during launch to prevent legitimate bugs being pushed to millions of software environments.

We have no intention to release a new major version of Requests. Urllib3 2.0 was painstakingly made largely backwards compatible. We'll provide a prominent notice that we now support this version in the next minor release, and users are free to pin to older versions as needed.

[sigmavirus24]

This is something that shouldn't be upgraded in a minor version of requests, because it causes breaking changes.

For example, we saw our elasticsearch client pull in the 2.0.0 version of urllib3 and it caused this failure on our machine. To resolve, we're adding a constraints.txt file, but not something that should be included in a minor release of requests.

ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2k-fips  26 Jan 2017. See: https://github.com/urllib3/urllib3/issues/2168

To make what @nateprewitt already eloquently stated a bit more explicit:

We will not be requiring urllib3 2.0, we will be adding support for it by raising the version constraint and ensuring requests works correctly with that version.

If you cannot use 2.0 you can continue to prevent it from being installed and we won't break of you use 1.26.x

[nyetwurk]

The same thing happened to us for ansible.

[nateprewitt]

Given the edge cases remaining have been thoroughly documented in the urllib3 2.0 migration guide and we haven't heard any noise here in a couple months I'm going to close and lock this thread with final steps for anyone reading this.

• Requests 2.30.0 (and later) support both urllib3 1.26+ and urllib3 2.0+. pip will automatically install urllib3 2.x for you. If you have conflicts, you can downgrade by pinning to "urllib3<2" as shown here.
• Some platforms (notably RHEL 7 and certain AWS platforms) do not supply OpenSSL 1.1.1+ by default and are incompatible with urllib3 2.0. For those platforms, you can either follow the steps in the migration guide or pin to "urllib3<2".
• Some projects have not upgraded to support urllib3 2.0 due to breaking changes. You may need to pin "urllib3<2" until there is a clear path for their migration.