Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix extra indexes #5220

Merged
merged 4 commits into from
Sep 7, 2022
Merged

Fix extra indexes #5220

merged 4 commits into from
Sep 7, 2022

Conversation

aronbierbaum
Copy link
Contributor

The issue

When installing a requirement that explicitly references an index all indexes are searched even though install_search_all_sources = False.

The fix

There were two different locations where the code appears to not have the expected behavior.

The checklist

  • Associated issue
  • A news fragment in the news/ directory to describe this fix with the extension .bugfix, .feature, .behavior, .doc. .vendor. or .trivial (this will appear in the release changelog). Use semantic line breaks and name the file after the issue number or the PR #.

@oz123
Copy link
Contributor

oz123 commented Aug 6, 2022

This seems like a good improvement, but I still would like to get a second opinion from @matteius .
Could you please also add a news file?

pipenv/utils/indexes.py Show resolved Hide resolved
@aronbierbaum aronbierbaum force-pushed the fix_extra_indexes branch 2 times, most recently from 85030e6 to f718642 Compare August 18, 2022 16:20
Comment on lines +1514 to +1507
source_names = {src.get("name") for src in sources}
if not search_all_sources and requirement.index in source_names:
Copy link
Contributor Author

@aronbierbaum aronbierbaum Aug 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change might have broken some tests. The proposed change only shows that the previous code would never find requirement.index in sources because the types didn't match. str and List[TSource].

I don't know enough about the desired behavior, but an alternative would be to remove the conditional completely since it would never be True.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the problem, thank you for your analysis. I'd like a little bit of time to consider what I was trying to accomplish with this code, given what I know now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aronbierbaum This patch fixes the other test that does fail:

diff --git a/tests/integration/test_lock.py b/tests/integration/test_lock.py
index 28e7f21f..d571a8d0 100644
--- a/tests/integration/test_lock.py
+++ b/tests/integration/test_lock.py
@@ -335,10 +335,11 @@ name = "testpypi"

 [packages]
 pipenv-test-private-package = {version = "*", index = "testpypi"}
+pipenv-test-public-package = {version = "*", index = "pypi"}
 requests = "*"
             """.strip()
             f.write(contents)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason that is important is because it doesn't search all sources and pipenv-test-private-package is in testpypi but requires pipenv-test-public-package is in pypi. When we initially switched to index restricted packages, there were some users that had to do similar things for their Pipfile index restrictions to work.

@matteius
Copy link
Member

@aronbierbaum I am not sure why earlier I could not get that second test failure, because I was able to reproduce it now. I think this would be a good way the fix the other test, since we showed both methods here, one where we rely on index restrictions, and this test where we allow all indexes to be searched:

diff --git a/tests/integration/test_install_uri.py b/tests/integration/test_install_uri.py
index f21da0f4..470274ef 100644
--- a/tests/integration/test_install_uri.py
+++ b/tests/integration/test_install_uri.py
@@ -209,6 +209,9 @@ name = "pypi"
 six = "*"

 [dev-packages]
+
+[pipenv]
+install_search_all_sources = true
             """.strip()
             f.write(contents)
         c = p.pipenv("install pipenv-test-private-package --index https://test.pypi.org/simple")

@matteius matteius requested a review from oz123 September 6, 2022 15:28
@AndreasBBS
Copy link

AndreasBBS commented Sep 7, 2022

@aronbierbaum Thanks for this fix! 👍 I've been pulling hairs for hours trying to understand why pipenv is installing the public "utils" package instead of my private one. You're awesome man! 🥇

@oz123 oz123 merged commit ffcbdfa into pypa:main Sep 7, 2022
@aronbierbaum aronbierbaum deleted the fix_extra_indexes branch September 7, 2022 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants