Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relax requests version requirement #138

Merged
merged 1 commit into from
Jun 16, 2023
Merged

Conversation

judahrand
Copy link
Contributor

Changes

Closes #136

Tests

  • make test run locally
  • make fmt applied
  • relevant integration tests applied

nfx
nfx previously requested changes Jun 1, 2023
@@ -10,7 +10,7 @@
version=version_data['__version__'],
packages=find_packages(exclude=["tests", "*tests.*", "*tests"]),
python_requires=">=3.7",
install_requires=["requests>=2.28.1,<2.29.0"],
install_requires=["requests>=2.28.1,<3"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we cannot increase the upper bound, as some of our downstream packages didn't migrate yet to requests 2.29.0+, lower bound can be decreased if it passes the tests.

see 1ea8f3b and psf/requests#6432

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we cannot increase the upper bound, as some of our downstream packages didn't migrate yet to requests 2.29.0+

Surely, the downstream packages should be responsible for pinning requests?! This is a publicly available package which can't be used with a version of requests without public security vulnerabilities (https://github.com/psf/requests/blob/main/HISTORY.md#2310-2023-05-22)...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we'll update the upper bound once our downstream packages update their upper bound

Copy link

@PeterJCLaw PeterJCLaw Jun 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nfx am I correct in understanding that the issue with supporting requests 2.29+ is to do with a mismatch in available ssl modules and urllib3 version 2.x (i.e: similar to urllib3/urllib3#2168)? If so, perhaps this package could pin to urllib3<2 (as the release notes for requests 2.30 suggest) as that would appear to avoid the issue? That would allow for requests to move forward to a version which has the security fix (i.e: 2.31) while keeping to the known-working version of urllib3.

I agree with @judahrand that such pinning really ought to be the responsibility of the downstream libraries/applications which actually depend on the particular versions of requests/urllib3, however if it's important to you that this package not admit urllib3 related issues then perhaps my suggestion above presents a way forwards.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PeterJCLaw @judahrand Sorry for the delay in addressing this ticket. As we saw during the original bump of urllib3, major version increases with backwards incompatibility has major risks for dependent software. We wanted to verify that at least our internal products wouldn't have yet another regression if we were to unpin this too soon. Especially with the release of 2.0.3, users who do not pin urllib3 in their setup.py or requirements.txt are likely to get a version that is compatible with their system's version of SSL. So, we're ready to merge this patch. This will be released in the next minor version (0.2.0).

Copy link
Contributor

@mgyucht mgyucht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for submitting this, and thanks for your patience while working through the details & implications of this change.

@@ -10,7 +10,7 @@
version=version_data['__version__'],
packages=find_packages(exclude=["tests", "*tests.*", "*tests"]),
python_requires=">=3.7",
install_requires=["requests>=2.28.1,<2.29.0"],
install_requires=["requests>=2.28.1,<3"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PeterJCLaw @judahrand Sorry for the delay in addressing this ticket. As we saw during the original bump of urllib3, major version increases with backwards incompatibility has major risks for dependent software. We wanted to verify that at least our internal products wouldn't have yet another regression if we were to unpin this too soon. Especially with the release of 2.0.3, users who do not pin urllib3 in their setup.py or requirements.txt are likely to get a version that is compatible with their system's version of SSL. So, we're ready to merge this patch. This will be released in the next minor version (0.2.0).

@mgyucht mgyucht enabled auto-merge (squash) June 16, 2023 13:07
@codecov-commenter
Copy link

codecov-commenter commented Jun 16, 2023

Codecov Report

Patch and project coverage have no change.

Comparison is base (3dee02a) 53.30% compared to head (a6bdaaf) 53.30%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #138   +/-   ##
=======================================
  Coverage   53.30%   53.30%           
=======================================
  Files          30       30           
  Lines       18346    18346           
=======================================
  Hits         9779     9779           
  Misses       8567     8567           

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@mgyucht mgyucht dismissed nfx’s stale review June 16, 2023 13:09

We have synced offline and agreed on this approach. nfx is on holiday.

@mgyucht
Copy link
Contributor

mgyucht commented Jun 16, 2023

@judahrand can you please setup a GPG key and rebase to sign your commits? See https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification#gpg-commit-signature-verification for instructions on setting this up.

auto-merge was automatically disabled June 16, 2023 14:02

Head branch was pushed to by a user without write access

@mgyucht mgyucht merged commit da0b6f5 into databricks:main Jun 16, 2023
@nfx nfx mentioned this pull request Jun 21, 2023
nfx added a commit that referenced this pull request Jun 21, 2023
* Added Sphinx documentation
([#184](#184),
[#191](#191),
[#183](#183),
[#193](#193)).
* Integrated with ReadTheDocs service
([#188](#188),
[#189](#189),
[#190](#190)).
* Create a deepcopy of config in api client
([#172](#172)).
* Fix client/secret auth
([#186](#186)).
* Increase DBFS copy buffer size
([#185](#185)).
* Move classes to other repository
([#192](#192)).
* Relax `requests` version upper bound to <3
([#138](#138)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Please loosen the dependency on requests
5 participants