Can't install detectron2 #2113
Comments
ghost
added
the
|
Came to file a bug about not being able to install detectron2 needs Even though I explicitly have Curiously, if I remove the But, even if I do the above w/o manually doing |
|
I imagine this might be a difficult problem (building wheels from general |
|
@malcolmgreaves the issue is that the library does not add The fix is for the project to add a |
|
😭 Oh that's very unfortunate! I appreciate the very prompt reply @abn 💯 I'll try and bug the FAIR folk to see if this can be done. |
|
No worries. Wish it was a more helpful response. You can just send them a PR with a minimal |
|
@abn Would the minimum viable (I originally first thought that I should make it a poetry project https://github.com/malcolmgreaves/detectron2/pull/1/files#diff-50c86b7ed8ac2cf95bd48334961bf0530cdc77b5a56f852c5c61b89d735fd711 But then realized that it should be possible to keep their build current build system, but just indicate that building the wheel requires |
|
🎉 I have answered my own question -- just adding |
|
@malcolmgreaves for completeness this is what you need to keep legacy build. [build-system]
requires = ["setuptools", "wheel", "torch"]
build-backend = "setuptools.build_meta:__legacy__" |
|
Wonderful! Thank you a ton 💯 💯 🤗 |
|
I am going to close this issue as the original issue has been resolved and the current issue has to do with lack of PEP 517 metadata in the package source. |
|
@malcolmgreaves I have this exact same issue, did you open an issue or pull request at the original Detectron2 repo? |
|
@joostmeulenbeld I have a workaround here -- you can |
|
@malcolmgreaves I just tried your workaround to integrate detectron2 with an existing project but still get errors... 😕 My pyproject.toml looks like this: [tool.poetry.dependencies]
...
detectron2 = { git = "https://github.com/facebookresearch/detectron2.git"}
[build-system]
requires = ["setuptols", "wheel", "torch"]
build-backend = "setuptools.build_meta:__legacy__"However, when I run PackageInfoError
Unable to determine package info for path: /tmp/pypoetry-git-detectron20h25u89n
Fallback egg_info generation failed.
Command ['/tmp/tmpi8lubyub/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1, and output:
Traceback (most recent call last):
File "/tmp/pypoetry-git-detectron20h25u89n/setup.py", line 10, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
at ~/.poetry/lib/poetry/inspection/info.py:502 in _pep517_metadata
498│ try:
499│ venv.run("python", "setup.py", "egg_info")
500│ return cls.from_metadata(path)
501│ except EnvCommandError as fbe:
→ 502│ raise PackageInfoError(
503│ path, "Fallback egg_info generation failed.", fbe
504│ )
505│ finally:
506│ os.chdir(cwd.as_posix())Any idea why that might be the case? |
|
I'm also still seeing this same issue with installing detectron2 using poetry 1.1.6 despite adding the poetry configuration mentioned in #2113 (comment) Relevant poetry configuration: [tool.poetry.dependencies]
python = "^3.8"
detectron2 = { git = "https://github.com/facebookresearch/detectron2.git", branch = "master" }
[build-system]
requires = ["poetry-core>=1.0.0", "setuptools", "wheel", "torch"]
build-backend = "setuptools.build_meta:__legacy__"Exception stacktrace for poetry lock:
@abn I see that you closed this due to "lack of PEP 517 metadata in the package source.". However, is there the possibility of changing the way the detectron2 package is installed by adding the torch dependency beforehand somehow? e.g. I realize this may come across as a new feature request, but I'm not sure how else to proceed without forking the detectron2 project. If you see here, the maintainers of the detectron2 project do not appear open to fixing this issue on their side:
For now, this is our workaround, in a project |
We are open as long as there is a reasonable fix. From what I can see that's not the case due to limitations in poetry (and all other python package management tools). |
|
Thanks for chiming in, @ppwwyyxx, good to know that you are open to helping solve this issue. Maybe @abn or another maintainer (@sdispater?) from the poetry side has an idea how to remove such limitations. The comments that @abn made above [1, 2] led me to believe this was something that needed to be addressed on the detectron2 project-side, and not the poetry side. Perhaps the poetry maintainers require more information as to how they could remove the limitations you are mentioning. It's not clear to me, but for now, the result is that we cannot use poetry to install detectron2. |
|
I'm getting the same error as @lucinvitae . All I want to do is something similar to
|
|
@lucinvitae as far as I can tell, the issue here is the package build specification does to convey that it needs Fro poetry's perspective, we first need to build the bdist of the package if a compatible one is not already available on the configured indices. This will lead to a poetry attempting to build the package's wheel, this happens inside an ephemeral isolated build environment according to PEP-517. When creating this environment, the root@c9958409c9fd:/# python -m pip wheel --disable-pip-version-check --use-pep517 git+https://github.com/facebookresearch/detectron2.git
Collecting git+https://github.com/facebookresearch/detectron2.git
Cloning https://github.com/facebookresearch/detectron2.git to /tmp/pip-req-build-ax2ddphy
Running command git clone -q https://github.com/facebookresearch/detectron2.git /tmp/pip-req-build-ax2ddphy
Resolved https://github.com/facebookresearch/detectron2.git to commit c081505af16f54c5a013916b81636d03f7c0a8fd
Installing build dependencies ... done
Getting requirements to build wheel ... error
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmphk82qqi5
cwd: /tmp/pip-req-build-ax2ddphy
Complete output (18 lines):
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 349, in <module>
main()
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 331, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 117, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-slhwwdsz/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 154, in get_requires_for_build_wheel
return self._get_build_requires(
File "/tmp/pip-build-env-slhwwdsz/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 135, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-slhwwdsz/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 258, in run_setup
super(_BuildMetaLegacyBackend,
File "/tmp/pip-build-env-slhwwdsz/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 150, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 10, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
----------------------------------------
WARNING: Discarding git+https://github.com/facebookresearch/detectron2.git. Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmphk82qqi5 Check the logs for full command output.
ERROR: Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmphk82qqi5 Check the logs for full command output.However, as @ppwwyyxx identified, there are other limitations that limit the options you have at the moment - ie. you cannot discover automatically the pre-built wheels provided by For instance, poetry resolver today does not handle local tags well neither does using link repositories work. This means that you cannot use the indices described in https://github.com/facebookresearch/detectron2/blob/main/INSTALL.md#install-pre-built-detectron2. However, you can use the direct links to the wheels here, but this assumes everwhere you install your package the same wheel can be used (this should be fine for cpu, alternatively you can have various extras. This is being fixed (partially done). That said, I do not think the python packaging ecosystem has a decent solution for gpu packages anyway - eg. there is no standard way to determine if a cuda enabled wheel can be used in active environment.
Having the dependency metadata specify something like Another option is to allow for something like What poetry should do in the near future are the following:
I still think that the As for the gpu specific versions, I am not sure where this needs to go. Ideally, there will be a packaging PEP that will help tools like 'poetry' to implement a standardised approach. |
|
Also, I'd recommend this moves to a discussion instead of this closed issue. |
|
What's the solution, then? How to add detectron2 using poetry? @abn |
The new pyproject.toml specifies the minimum requirements necessary to build the project's wheel. This makes this project up to the PEP-518 standard. Specifically, this change adds torch as a build-time dependency. The version of torch can be set by the installer when they specify a version of torch in their own project settings (e.g. a pyproject.toml, requirements.txt, setup.py, environment.yaml, etc.). Alteratively, if torch is in the current installing environment, that version will be used. If not specified, the latest torch version will be used. This change preserves using `setup.py` for project building. python-poetry/poetry#2113
Thanks a lot this solution works |
|
This seems to no longer work - or at least not in the general case. The issue that everyone seems to be missing - please correct me if I'm wrong about it - is that simply specifying Since The problem runs even deeper. Even if ^1. please confirm this, I'm new to setuptools and custom package building in general. |
|
I managed to get it run by specifying: in a detectron2 fork using the following config using poetry==1.2.0b2 following this issue, it is important to mention that this file will not work on the latest non-beta release and additionally adding The last step is necessary, as PEP-518 specifies to use isolated builds for building wheels, so the index specified in poetry won't be used. However, globally configured indexes will also be used by the newly isolated venv. Some side notes:
|
|
Hi, does anyone have any solution to this issue? I still can't find a way to install detectron2. I tried #2113 (comment) but got an error: Resolving dependencies... (7.9s)
CalledProcessError
Command '['/tmp/tmpbzglt_2o/.venv/bin/python', '-']' returned non-zero exit status 1.
at ~/.pyenv/versions/3.8.11/lib/python3.8/subprocess.py:516 in run
512│ # We don't call process.wait() as .__exit__ does that for us.
513│ raise
514│ retcode = process.poll()
515│ if check and retcode:
→ 516│ raise CalledProcessError(retcode, process.args,
517│ output=stdout, stderr=stderr)
518│ return CompletedProcess(process.args, retcode, stdout, stderr)
519│
520│
The following error occurred when trying to handle this error:
EnvCommandError
Command ['/tmp/tmpbzglt_2o/.venv/bin/python', '-'] errored with the following return code 1, and output:
ERROR: Could not find a version that satisfies the requirement torchvision==0.11.3+cu113 (from versions: 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.2.2, 0.2.2.post2, 0.2.2.post3, 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.11.0, 0.11.1, 0.11.2, 0.11.3, 0.12.0, 0.13.0, 0.13.1)
ERROR: No matching distribution found for torchvision==0.11.3+cu113
WARNING: You are using pip version 21.1.1; however, version 22.2.2 is available.
You should consider upgrading via the '/tmp/build-env-wccfwciw/bin/python -m pip install --upgrade pip' command.
Traceback (most recent call last):
File "<stdin>", line 15, in <module>
File "/tmp/tmpbzglt_2o/.venv/lib/python3.8/site-packages/build/env.py", line 211, in install
_subprocess(cmd)
File "/tmp/tmpbzglt_2o/.venv/lib/python3.8/site-packages/build/env.py", line 81, in _subprocess
raise e
File "/tmp/tmpbzglt_2o/.venv/lib/python3.8/site-packages/build/env.py", line 78, in _subprocess
subprocess.check_output(cmd, stderr=subprocess.STDOUT)
File "/home/sonywicaksono/.pyenv/versions/3.8.11/lib/python3.8/subprocess.py", line 415, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/home/sonywicaksono/.pyenv/versions/3.8.11/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/tmp/build-env-wccfwciw/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '-r', '/tmp/build-reqs-u1urbb0b.txt']' returned non-zero exit status 1.
input was : import build
import build.env
import pep517
source = '/tmp/pypoetry-git-detectron2qmhb669o'
dest = '/tmp/tmpbzglt_2o/dist'
with build.env.IsolatedEnvBuilder() as env:
builder = build.ProjectBuilder(
srcdir=source,
scripts_dir=env.scripts_dir,
python_executable=env.executable,
runner=pep517.quiet_subprocess_runner,
)
env.install(builder.build_system_requires)
env.install(builder.get_requires_for_build('wheel'))
builder.metadata_path(dest)
at ~/.pyenv/versions/3.8.11/lib/python3.8/site-packages/poetry/utils/env.py:1421 in _run
1417│ output = subprocess.check_output(
1418│ cmd, stderr=subprocess.STDOUT, env=env, **kwargs
1419│ )
1420│ except CalledProcessError as e:
→ 1421│ raise EnvCommandError(e, input=input_)
1422│
1423│ return decode(output)
1424│
1425│ def execute(self, bin: str, *args: str, **kwargs: Any) -> int | None:
The following error occurred when trying to handle this error:
CalledProcessError
Command '['/tmp/tmpbzglt_2o/.venv/bin/python', 'setup.py', 'egg_info']' returned non-zero exit status 1.
at ~/.pyenv/versions/3.8.11/lib/python3.8/subprocess.py:516 in run
512│ # We don't call process.wait() as .__exit__ does that for us.
513│ raise
514│ retcode = process.poll()
515│ if check and retcode:
→ 516│ raise CalledProcessError(retcode, process.args,
517│ output=stdout, stderr=stderr)
518│ return CompletedProcess(process.args, retcode, stdout, stderr)
519│
520│
The following error occurred when trying to handle this error:
EnvCommandError
Command ['/tmp/tmpbzglt_2o/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1, and output:
Traceback (most recent call last):
File "setup.py", line 10, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
at ~/.pyenv/versions/3.8.11/lib/python3.8/site-packages/poetry/utils/env.py:1421 in _run
1417│ output = subprocess.check_output(
1418│ cmd, stderr=subprocess.STDOUT, env=env, **kwargs
1419│ )
1420│ except CalledProcessError as e:
→ 1421│ raise EnvCommandError(e, input=input_)
1422│
1423│ return decode(output)
1424│
1425│ def execute(self, bin: str, *args: str, **kwargs: Any) -> int | None:
The following error occurred when trying to handle this error:
PackageInfoError
Unable to determine package info for path: /tmp/pypoetry-git-detectron2qmhb669o
Fallback egg_info generation failed.
Command ['/tmp/tmpbzglt_2o/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1, and output:
Traceback (most recent call last):
File "setup.py", line 10, in <module>
import torch
ModuleNotFoundError: No module named 'torch' |
Please refer to the thread -- See this user's commit for an example of the patch: If you have more questions, please start a Discussion or join Discord -- this issue is not a great forum for support as problems in Poetry are already solved, and we are notifying many users unnecessarily. |
The missing pyproject.toml results in an inability to install detectron2 via poetry. fixes python-poetry/poetry#2113
|
Has this issue been solved? |
|
All the claimed "solutions" or patches so far are incorrect, as explained in #2113 (comment). This is why detectron2 did not accept any of these solutions. |
Has this been addressed / solved in the meantime? The standard installation still does not work. But what about the many suggested modifications to poetry that you outlined. Have they been implemented? How can you install detectron2 now with poetry? |
|
just using the executable python to run the setup script, like: |
|
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
-vvvoption).Issue
Poetry can't figure out the version of detectron2. I've tried to debug this myself but the error messages don't tell me anything useful:
Is poetry unable to get dynamically generated versions?
The text was updated successfully, but these errors were encountered: