ERROR: No matching distribution found for torch==1.11.0+cu113 when using --extra-index-url #5022
Comments
|
Excellent, thanks for creating the ticket. I have managed to verify that 2022.1.8 version of Pipenv works: I have not checked all of the March 2022 releases but I know for a fact that Pipenv 2022.3.28 is broken. So the bug was introduced somewhere in the March releases. |
|
Please seem my comment here, I suspect this is an edge case of the fix for package index restrictions. If you name the index and specify you want it from the named index it should work on newer versions. Can you verify that point? What do you expect the Pipfile to look like when you are using the --extra-index-url argument (try on the older 2022.1.8 release)? |
|
I was looking in the pipenv docs for where it talks about I do see in the code base we have that option, and the help text is: Likely a regression that we never had a test case for, but more thought will need to go into it, especially considering the security concerns of package indexes. I have a hunch that for your use case you really should be restricting the package index by name, otherwise pip will still go look in pypi and potentially pick something from there if the version is newer. If you truly want to ensure you are using the specific index that hosts your torch distribution, I would pin to that. |
|
Under the older release my Pipfile looks like this: How do I name the index? |
|
@DanielPerezJensen It would be like this: |
|
@matteius Would I have to specify that in the Pipfile then by hand or is there a way to get it written like that through terminal commands? |
|
@DanielPerezJensen It seems very likely that the breaking change was the "stricter index searching" which @matteius mentioned. Indeed, the non-CUDA versions of Torch are listed as candidates in the error message, which suggests that Pipenv is searching on PyPi for Torch and is ignoring the Torch index. The Pipfile now needs a "index=XYZ" line to explicitly say "get torch from the downloadpytorch index/source". It's a bug/regression in Pipenv which makes it ignore the contents of the specified extra sources even though we want them. And @matteius: The |
|
@matteius I can confirm that manually editing the Pipfile to force Pipenv to get the package from PyTorch's repo works as a temporary solution. Any ideas if the general bug can be fixed so that Pipenv scans the additional repos again without having to be forcibly told to use them per-package? Related ticket: #4983 A fix may be something like:
|
Understood, but I know from triaging your other reports that it does in fact check pypi when you are using the extra index url, and that seems dangerous and takes additional time to do the locking and install than just being explicit about where you know that you want the package from.
I will be studying the code more to consider your use case, will have to consider what might be possible before getting back to you on the solution wrt to that.
This part will likely not be possible because this is a pip behavior and we try to use pip in a way that does not modify its internals wherever possible and that would be quite the modification to do -- in fact pip prefers PyPi and says its recommended that if you want to avoid pypi but need pypi packages that your mirror should contain all pypi packages that you need (or mirror pypi). We might be able to get back the old behavior if no index is specified though. |
Is there any way to modify May wanna remove
I see. So you're internally calling the pip program and telling it to install from the pip/index? I guess all of this is fixed/good enough if It's all a bit weird now though. It used to be possible to do something like That behavior was the best. If someone has a reason to manually pin a package to an exact index in the Pipfile then so be it. But for general easy usage, "pick the matching package and version from any repo that has it" was great. |
It's still running because its a huge package with multiple matching wheels, so debugging this will take some time.
Going to give it more thought, ideally we get the prior behavior working again for you folks that don't use index restrictions, but that is going to require some thinking time.
Yes, and more complicated than that is we vendor (copy in a specific version of pip) to the pipenv code base, and we call internal APIs of pip to do magical things that even pip itself doesn't do. For more insights into that, I have a recent thread in pip that goes into some detail around some of the complexities: pypa/pip#10964 (comment)
Yeah, I think that is why it will be necessary to contemplate getting that prior behavior working again without breaking index restrictions. I have to note, when calling out that many extra index urls, its still going to prefer pypi and its going to cause more churn on the resolver/locking to find your packages.
Debatable :-) It is also great for package confusion attackers because how do you know for sure where you are pulling your package from in that scenario?
So imagine someone can poison the DNS for one of the package servers and put up a private pypi with a version that is malicious -- now your resolver goes and gets that because it matches I'm glad to have having this discussion and thankful you opened the reports, so it will get investigated. For now you have two workaround, the prior version or being more explicit in your package needs which will I think benefit you more in the long run. For example, @Bananaman did you ever consider my other comment about calling out the wheel you specifically want to avoid downloading all 8 and having them in the lockfile? It really depends on if you need the lockfile to have all 8 hashes for the different system architectures or if you are just trying to support the one for your system. Ref: #4963 (comment) |
|
The installing of specific wheels also serves well for repo's that might not conform to PEP503. For example, PyTorch Geometric |
@DanielPerezJensen Yes it seems that this works: Generated this Pipfile: Initial reaction here is I am kind of surprised it has 3 sources, but it seemed to have worked. Generated this Pipfile.lock ... actually this doesn't look right either because it doesn't have the package hashes in the lockfile. Side-note: I was having some buggy experiences earlier using git shell in windows to the locking, and I can only figure the size of the wheel files were freezing up the shell. I just had a much much better experience using Windows Powershell, it was very fast actually. I wonder if the reason its so fast is because its not pulling the hash values anymore. |
|
I think you might have 3 sources because you put in a different command the second time around, namely your index is different.
|
|
When I run that first command I get a relatively clean Pipfile and Pipfile.lock. But the problem of 8 hashes persists. But, for now, it does seem to solve the issue of being unable to install PyTorch using pipenv. Although downloading 16GBs is of course unnecessary. |
|
@DanielPerezJensen Can you confirm what version of pipenv you are on? I think the newer version of pipenv have a bug where its only pulling the one hash instead of 8 -- that is what I got on the newer versions (or at least my branch where I try to fix the hash at all for these). But if you are getting 8, that is technically working by design that there would be 8 hashes for that package, the reason why is you are calling out The understanding is that when you are generating a Since you may just be using pipenv for yourself and only truly need the one wheel in your lockfile, you can do something like this to install the single wheel file. |
|
@matteius I am currently on version pipenv==2022.1.8 |
|
@DanielPerezJensen Would you be able to test with |
matteius
added
Type: Possible Bug
|
Running The verbose output is this: (test) daniel@daniel-desktop:~/Documents/test$ pipenv install "torch==1.11.0+cu113" --index https://download.pytorch.org/whl/cu113/ -v
Installing torch==1.11.0+cu113...
Installing package: torch==1.11.0+cu113
Writing supplied requirement line to temporary file: 'torch==1.11.0+cu113'
Installing 'torch'
⠸ Installing torch...$ /home/daniel/.local/share/virtualenvs/test-Kb_OZVBf/bin/python -m pip install --verbose --upgrade --exists-action=i -r /tmp/pipenv-zz18ayop-requirements/pipenv-edwy4lp9-requirement.txt -i https://download.pytorch.org/whl/cu113/ --extra-index-url https://pypi.org/simple --extra-index-url https://pypi.org/simple
Using source directory: '/home/daniel/.local/share/virtualenvs/test-Kb_OZVBf/src'
Adding torch to Pipfile's [packages]...
✔ Installation Succeeded
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
Reporter.starting()
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.starting()
Reporter.adding_requirement(SpecifierRequirement('torch==1.11.0+cu113'), None)
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('torch==1.11.0+cu113'), None)
Reporter.starting_round(0)
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.starting_round(0)
Reporter.adding_requirement(SpecifierRequirement('typing-extensions'), LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)'))
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('typing-extensions'), LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)'))
Reporter.pinning(LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)'))
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.pinning(LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)'))
Reporter.ending_round(0, state)
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.ending_round(0, state)
Reporter.starting_round(1)
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.starting_round(1)
Reporter.pinning(LinkCandidate('https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl#sha256=21c85e0fe4b9a155d0799430b0ad741cdce7e359660ccbd8b530613e8df88ce2 (from https://pypi.org/simple/typing-extensions/) (requires-python:>=3.6)'))
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.pinning(LinkCandidate('https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl#sha256=21c85e0fe4b9a155d0799430b0ad741cdce7e359660ccbd8b530613e8df88ce2 (from https://pypi.org/simple/typing-extensions/) (requires-python:>=3.6)'))
Reporter.ending_round(1, state)
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.ending_round(1, state)
Reporter.starting_round(2)
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.starting_round(2)
Reporter.ending(State(mapping=OrderedDict([('torch', LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)')), ('typing-extensions', LinkCandidate('https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl#sha256=21c85e0fe4b9a155d0799430b0ad741cdce7e359660ccbd8b530613e8df88ce2 (from https://pypi.org/simple/typing-extensions/) (requires-python:>=3.6)'))]), criteria={'torch': Criterion((SpecifierRequirement('torch==1.11.0+cu113'), via=None)), 'typing-extensions': Criterion((SpecifierRequirement('typing-extensions'), via=LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)')))}))
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.ending(State(mapping=OrderedDict([('torch', LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)')), ('typing-extensions', LinkCandidate('https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl#sha256=21c85e0fe4b9a155d0799430b0ad741cdce7e359660ccbd8b530613e8df88ce2 (from https://pypi.org/simple/typing-extensions/) (requires-python:>=3.6)'))]), criteria={'torch': Criterion((SpecifierRequirement('torch==1.11.0+cu113'), via=None)), 'typing-extensions': Criterion((SpecifierRequirement('typing-extensions'), via=LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)')))}))
Warning: Error generating hash for torch
⠋ Locking...
✔ Success!
Updated Pipfile.lock (20805a)!
Installing dependencies from Pipfile.lock (20805a)...
🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00With the following Pipfile: Importing torch The numpy dependency not being installed is a "problem" that I have seen many times when installing PyTorch through their index. Installing numpy solves the issue and torch can be imported. Furthermore running some test code to verify that the gpu is being used shows that indeed the GPU is getting used. |
|
@DanielPerezJensen Thanks for checking that, and I assume that if you look in the |
|
My Pipfile lock looks like this (with numpy installed as well): |
|
Thanks for confirming this @DanielPerezJensen -- I have a commit that ensure that the 1-hash that gets installed for "torch" or other index restricted packages gets included, but I've had more difficulties getting it to include all 8 hashes again. I think the ideal, would be it includes the two hashes that match your required python_version or all of them (all 8) if a version is not specified. I will be looking to find more time to look into this, at a minimum will ship new version in April that includes at least the matching hash, but ideally will figure out how to include all the relevant hashes as well for private index restricted packages. |
|
Also, I just checked the wheel archive |
|
@DanielPerezJensen I have a branch that appears to correct include all 8 hashes again for this example, and thus should fix the hash collection for all index restricted packages in pipenv. If you care to try it out, the branch is here and you can install it using |
|
I could not get python to work properly without having the latest installed, so I worked around this by manually changing install scripts, wherever there was python replace with python3.7, and also ran python3.7 -m ensurepip --upgrade, then creating venv with python3.7 -m venv . |
It seems that this error is also a problem when installing from other sources, such as in #5021.
Issue description
After lots of debugging and trial and error, we have run into a bug with pipenv relating to installing libraries while using another index url (--extra-index-url).
When trying to install PyTorch using the following command
pipenv install --extra-index-url https://download.pytorch.org/whl/cu113/ "torch==1.11.0+cu113" Installing torch==1.11.0+cu113, as specified under this issue: #4961, we get the following error:ERROR: No matching distribution found for torch==1.11.0+cu113.According to @Bananaman this was possible to do in a previous version of pipenv.
Expected result
Expected an installation using a 3rd party index url, in this case PyTorch.
Actual result
Steps to replicate
Be sure to use the latest version of pipenv, if I use a previous version (latest confirmed is 2022.1.8) the error goes away.
pipenv install --extra-index-url https://download.pytorch.org/whl/cu113/ "torch==1.11.0+cu113"$ pipenv --support
Pipenv version:
'2022.3.28'Pipenv location:
'/home/daniel/.local/lib/python3.8/site-packages/pipenv'Python location:
'/usr/bin/python3'Python installations found:
3.8.12:/home/daniel/.pyenv/versions/3.8.12/bin/python3.8.10:/usr/bin/python33.8.10:/usr/bin/python3.83.8.10:/bin/python33.8.10:/bin/python3.8PEP 508 Information:
System environment variables:
SHELLSESSION_MANAGERQT_ACCESSIBILITYCOLORTERMPYENV_SHELLXDG_CONFIG_DIRSXDG_MENU_PREFIXGNOME_DESKTOP_SESSION_IDMANDATORY_PATHLC_ADDRESSGNOME_SHELL_SESSION_MODELC_NAMESSH_AUTH_SOCKXMODIFIERSDESKTOP_SESSIONLC_MONETARYSSH_AGENT_PIDGTK_MODULESPWDLOGNAMEXDG_SESSION_DESKTOPXDG_SESSION_TYPEGPG_AGENT_INFOXAUTHORITYGJS_DEBUG_TOPICSWINDOWPATHHOMEUSERNAMEIM_CONFIG_PHASELC_PAPERLANGLS_COLORSXDG_CURRENT_DESKTOPVTE_VERSIONGNOME_TERMINAL_SCREENINVOCATION_IDMANAGERPIDGJS_DEBUG_OUTPUTLESSCLOSEXDG_SESSION_CLASSSETUPTOOLS_USE_DISTUTILSTERMLC_IDENTIFICATIONDEFAULTS_PATHLESSOPENUSERGNOME_TERMINAL_SERVICEDISPLAYSHLVLLC_TELEPHONEQT_IM_MODULELC_MEASUREMENTLD_LIBRARY_PATHXDG_RUNTIME_DIRPYENV_ROOTLC_TIMEJOURNAL_STREAMXDG_DATA_DIRSPATHGDMSESSIONDBUS_SESSION_BUS_ADDRESSLC_NUMERICOLDPWD_PIP_SHIMS_BASE_MODULEPIP_DISABLE_PIP_VERSION_CHECKPYTHONDONTWRITEBYTECODEPIP_PYTHON_PATHPYTHONFINDER_IGNORE_UNSUPPORTEDPipenv–specific environment variables:
Debug–specific environment variables:
PATH:/home/daniel/.pyenv/shims:/home/daniel/.pyenv/bin:/usr/local/cuda-11.3/bin:/home/daniel/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/binSHELL:/bin/bashLANG:en_US.UTF-8PWD:/home/daniel/DocumentsThe text was updated successfully, but these errors were encountered: