Duplicated entries in poetry.lock that cause export to requirements.txt to fail

[pspachtholz]

• Poetry version: 1.2.2
• Python version: 3.8.13
• OS version and name: macOS 13.0.1
• pyproject.toml:

`[tool.poetry]
name = "debug"
version = "0.1.0"
description = ""
authors = []
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.8"
statsmodels = "^0.13.5"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"`

• [ x ] I am on the latest stable Poetry version, installed using a recommended method.
• [ x ] I have searched the issues of this repo and believe that this is not a duplicate.
• [ x ] I have consulted the FAQ and blog for any relevant entries or release notes.
• [ x ] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

When trying to export poetry.lock to requirements.txt format i receive an error:
Dependency walk failed at scipy(>=1.3, <1.9)

After digging deeper it seems like the error is coming from statsmodels which leads to scipy being in poetry.lock with 2 different versions.

poetry.lock:
scipy = [ {file = "scipy-1.8.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:65b77f20202599c51eb2771d11a6b899b97989159b7975e9b5259594f1d35ef4"}, {file = "scipy-1.8.1-cp310-cp310-macosx_12_0_arm64.whl", hash = "sha256:e013aed00ed776d790be4cb32826adb72799c61e318676172495383ba4570aa4"}, {file = "scipy-1.8.1-cp310-cp310-macosx_12_0_universal2.macosx_10_9_x86_64.whl", hash = "sha256:02b567e722d62bddd4ac253dafb01ce7ed8742cf8031aea030a41414b86c1125"}, {file = "scipy-1.8.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1da52b45ce1a24a4a22db6c157c38b39885a990a566748fc904ec9f03ed8c6ba"}, {file = "scipy-1.8.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a0aa8220b89b2e3748a2836fbfa116194378910f1a6e78e4675a095bcd2c762d"}, {file = "scipy-1.8.1-cp310-cp310-win_amd64.whl", hash = "sha256:4e53a55f6a4f22de01ffe1d2f016e30adedb67a699a310cdcac312806807ca81"}, {file = "scipy-1.8.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:28d2cab0c6ac5aa131cc5071a3a1d8e1366dad82288d9ec2ca44df78fb50e649"}, {file = "scipy-1.8.1-cp38-cp38-macosx_12_0_arm64.whl", hash = "sha256:6311e3ae9cc75f77c33076cb2794fb0606f14c8f1b1c9ff8ce6005ba2c283621"}, {file = "scipy-1.8.1-cp38-cp38-macosx_12_0_universal2.macosx_10_9_x86_64.whl", hash = "sha256:3b69b90c9419884efeffaac2c38376d6ef566e6e730a231e15722b0ab58f0328"}, {file = "scipy-1.8.1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:6cc6b33139eb63f30725d5f7fa175763dc2df6a8f38ddf8df971f7c345b652dc"}, {file = "scipy-1.8.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9c4e3ae8a716c8b3151e16c05edb1daf4cb4d866caa385e861556aff41300c14"}, {file = "scipy-1.8.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:23b22fbeef3807966ea42d8163322366dd89da9bebdc075da7034cee3a1441ca"}, {file = "scipy-1.8.1-cp38-cp38-win32.whl", hash = "sha256:4b93ec6f4c3c4d041b26b5f179a6aab8f5045423117ae7a45ba9710301d7e462"}, {file = "scipy-1.8.1-cp38-cp38-win_amd64.whl", hash = "sha256:70ebc84134cf0c504ce6a5f12d6db92cb2a8a53a49437a6bb4edca0bc101f11c"}, {file = "scipy-1.8.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:f3e7a8867f307e3359cc0ed2c63b61a1e33a19080f92fe377bc7d49f646f2ec1"}, {file = "scipy-1.8.1-cp39-cp39-macosx_12_0_arm64.whl", hash = "sha256:2ef0fbc8bcf102c1998c1f16f15befe7cffba90895d6e84861cd6c6a33fb54f6"}, {file = "scipy-1.8.1-cp39-cp39-macosx_12_0_universal2.macosx_10_9_x86_64.whl", hash = "sha256:83606129247e7610b58d0e1e93d2c5133959e9cf93555d3c27e536892f1ba1f2"}, {file = "scipy-1.8.1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:93d07494a8900d55492401917a119948ed330b8c3f1d700e0b904a578f10ead4"}, {file = "scipy-1.8.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d3b3c8924252caaffc54d4a99f1360aeec001e61267595561089f8b5900821bb"}, {file = "scipy-1.8.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:70de2f11bf64ca9921fda018864c78af7147025e467ce9f4a11bc877266900a6"}, {file = "scipy-1.8.1-cp39-cp39-win32.whl", hash = "sha256:1166514aa3bbf04cb5941027c6e294a000bba0cf00f5cdac6c77f2dad479b434"}, {file = "scipy-1.8.1-cp39-cp39-win_amd64.whl", hash = "sha256:9dd4012ac599a1e7eb63c114d1eee1bcfc6dc75a29b589ff0ad0bb3d9412034f"}, {file = "scipy-1.8.1.tar.gz", hash = "sha256:9e3fb1b0e896f14a85aa9a28d5f755daaeeb54c897b746df7a55ccb02b340f33"}, {file = "scipy-1.9.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:1884b66a54887e21addf9c16fb588720a8309a57b2e258ae1c7986d4444d3bc0"}, {file = "scipy-1.9.3-cp310-cp310-macosx_12_0_arm64.whl", hash = "sha256:83b89e9586c62e787f5012e8475fbb12185bafb996a03257e9675cd73d3736dd"}, {file = "scipy-1.9.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1a72d885fa44247f92743fc20732ae55564ff2a519e8302fb7e18717c5355a8b"}, {file = "scipy-1.9.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d01e1dd7b15bd2449c8bfc6b7cc67d630700ed655654f0dfcf121600bad205c9"}, {file = "scipy-1.9.3-cp310-cp310-win_amd64.whl", hash = "sha256:68239b6aa6f9c593da8be1509a05cb7f9efe98b80f43a5861cd24c7557e98523"}, {file = "scipy-1.9.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:b41bc822679ad1c9a5f023bc93f6d0543129ca0f37c1ce294dd9d386f0a21096"}, {file = "scipy-1.9.3-cp311-cp311-macosx_12_0_arm64.whl", hash = "sha256:90453d2b93ea82a9f434e4e1cba043e779ff67b92f7a0e85d05d286a3625df3c"}, {file = "scipy-1.9.3-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:83c06e62a390a9167da60bedd4575a14c1f58ca9dfde59830fc42e5197283dab"}, {file = "scipy-1.9.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:abaf921531b5aeaafced90157db505e10345e45038c39e5d9b6c7922d68085cb"}, {file = "scipy-1.9.3-cp311-cp311-win_amd64.whl", hash = "sha256:06d2e1b4c491dc7d8eacea139a1b0b295f74e1a1a0f704c375028f8320d16e31"}, {file = "scipy-1.9.3-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:5a04cd7d0d3eff6ea4719371cbc44df31411862b9646db617c99718ff68d4840"}, {file = "scipy-1.9.3-cp38-cp38-macosx_12_0_arm64.whl", hash = "sha256:545c83ffb518094d8c9d83cce216c0c32f8c04aaf28b92cc8283eda0685162d5"}, {file = "scipy-1.9.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0d54222d7a3ba6022fdf5773931b5d7c56efe41ede7f7128c7b1637700409108"}, {file = "scipy-1.9.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:cff3a5295234037e39500d35316a4c5794739433528310e117b8a9a0c76d20fc"}, {file = "scipy-1.9.3-cp38-cp38-win_amd64.whl", hash = "sha256:2318bef588acc7a574f5bfdff9c172d0b1bf2c8143d9582e05f878e580a3781e"}, {file = "scipy-1.9.3-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:d644a64e174c16cb4b2e41dfea6af722053e83d066da7343f333a54dae9bc31c"}, {file = "scipy-1.9.3-cp39-cp39-macosx_12_0_arm64.whl", hash = "sha256:da8245491d73ed0a994ed9c2e380fd058ce2fa8a18da204681f2fe1f57f98f95"}, {file = "scipy-1.9.3-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4db5b30849606a95dcf519763dd3ab6fe9bd91df49eba517359e450a7d80ce2e"}, {file = "scipy-1.9.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c68db6b290cbd4049012990d7fe71a2abd9ffbe82c0056ebe0f01df8be5436b0"}, {file = "scipy-1.9.3-cp39-cp39-win_amd64.whl", hash = "sha256:5b88e6d91ad9d59478fafe92a7c757d00c59e3bdc3331be8ada76a4f8d683f58"}, {file = "scipy-1.9.3.tar.gz", hash = "sha256:fbc5c05c85c1a02be77b1ff591087c83bc44579c6d2bd9fb798bb64ea5e1a027"},

Original error:
`Source (poetry-locked): 1 packages found for statsmodels >=0.13.5,<0.14.0

Stack trace:

12 ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/cleo/application.py:329 in run
327│
328│ try:
→ 329│ exit_code = self._run(io)
330│ except Exception as e:
331│ if not self._catch_exceptions:

11 ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/console/application.py:185 in _run
183│ self._load_plugins(io)
184│
→ 185│ exit_code: int = super()._run(io)
186│ return exit_code
187│

10 ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/cleo/application.py:423 in _run
421│ io.input.set_stream(stream)
422│
→ 423│ exit_code = self._run_command(command, io)
424│ self._running_command = None
425│

9 ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/cleo/application.py:465 in _run_command
463│
464│ if error is not None:
→ 465│ raise error
466│
467│ return event.exit_code

8 ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/cleo/application.py:449 in _run_command
447│
448│ if event.command_should_run():
→ 449│ exit_code = command.run(io)
450│ else:
451│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

7 ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/cleo/commands/base_command.py:119 in run
117│ io.input.validate()
118│
→ 119│ status_code = self.execute(io)
120│
121│ if status_code is None:

6 ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/cleo/commands/command.py:83 in execute
81│
82│ try:
→ 83│ return self.handle()
84│ except KeyboardInterrupt:
85│ return 1

5 ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry_plugin_export/command.py:107 in handle
105│ exporter.with_credentials(self.option("with-credentials"))
106│ exporter.with_urls(not self.option("without-urls"))
→ 107│ exporter.export(fmt, Path.cwd(), output or self.io)
108│

4 ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry_plugin_export/exporter.py:86 in export
84│ raise ValueError(f"Invalid export format: {fmt}")
85│
→ 86│ getattr(self, self.EXPORT_METHODS[fmt])(cwd, output)
87│
88│ def _export_generic_txt(

3 ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry_plugin_export/exporter.py:101 in _export_generic_txt
99│ )
100│
→ 101│ for dependency_package in get_project_dependency_packages(
102│ self._poetry.locker,
103│ project_requires=root.all_requires,

2 ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry_plugin_export/walker.py:103 in get_project_dependency_packages
101│ selected.append(dependency)
102│
→ 103│ for package, dependency in get_project_dependencies(
104│ project_requires=selected,
105│ locked_packages=repository.packages,

1 ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry_plugin_export/walker.py:129 in get_project_dependencies
127│ )
128│
→ 129│ nested_dependencies = walk_dependencies(
130│ dependencies=project_requires,
131│ packages_by_name=packages_by_name,

RuntimeError

Dependency walk failed at scipy (>=1.3,<1.9)

at ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry_plugin_export/walker.py:155 in walk_dependencies
151│ requirement, packages_by_name, nested_dependencies
152│ )
153│
154│ if not locked_package:
→ 155│ raise RuntimeError(f"Dependency walk failed at {requirement}")
156│
157│ if requirement.extras:
158│ locked_package = locked_package.with_features(requirement.extras)`

[dimbleby]

this issue belongs in https://github.com/python-poetry/poetry-plugin-export

[Secrus]

Looking at the information you provided, seems like some issues with your lockfile. At the same time, you have Dependency walk failed at scipy (>=1.3,<1.9) and locked files for scipy 1.9.3. Had some merging gone wrong maybe?

[pspachtholz]

I don't know whether this is the expected behavior of the lockfile or whether its an issue with the export. It's unlikely to be related to some merge, because you can reproduce the behavior in a clean project:
`
poetry init

poetry add statsmodels

poetry export
`

It looks like this has to due with the requirements of statsmodels which specifies scipy multiple times

[JohnGiorgi]

Facing what I believe is the exact same issue, poetry export returns

Dependency walk failed at scipy (>=1.3,<1.9)

and I can also see that statsmodels is in my poetry.lock. I tried upgrading poetry and poetry-core from git (as I can see #528 was marked as closing this)

pip install --upgrade "git+https://github.com/python-poetry/poetry.git"
pip install --upgrade git+https://github.com/python-poetry/poetry-core.git

But I still get the error Dependency walk failed at scipy (>=1.3,<1.9). Am I doing something wrong?

[AFTownsend]

We are seeing the same thing:
Dependency walk failed at scipy (>=1.3,<1.9)
poetry 1.2.0, statsmodel 0.35

[neersighted]

This fix is not in Poetry 1.2.0.

[Lucasje19]

I too experience this issue. Though it is not a poetry issue, but one caused by statsmodels as already mentioned by @pspachtholz. There also exists a feature requests to solve this dependency mismatch at statsmodels, but so far with no progress: statsmodels/statsmodels#8543.

Others that stumble upon this would do well to reply to the linked issue.

[jimmerzzz52]

+1

[simonprovost]

Need issue to be solved too..

[sbor23]

This issue still exists in poetry 1.4.2. Can we please reopen this?

Example:

• poetry init
• poetry add dagster-graphql
• poetry export

Output

Source (poetry-lockfile): 1 packages found for dagster-graphql >=1.3.6,<2.0.0

  Stack trace:

  12  ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/cleo/application.py:327 in run
       325│ 
       326│             try:
     → 327│                 exit_code = self._run(io)
       328│             except BrokenPipeError:
       329│                 # If we are piped to another process, it may close early and send a

  11  ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/poetry/console/application.py:190 in _run
       188│         self._load_plugins(io)
       189│ 
     → 190│         exit_code: int = super()._run(io)
       191│         return exit_code
       192│ 

  10  ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/cleo/application.py:431 in _run
       429│             io.input.interactive(interactive)
       430│ 
     → 431│         exit_code = self._run_command(command, io)
       432│         self._running_command = None
       433│ 

   9  ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/cleo/application.py:473 in _run_command
       471│ 
       472│         if error is not None:
     → 473│             raise error
       474│ 
       475│         return terminate_event.exit_code

   8  ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/cleo/application.py:457 in _run_command
       455│ 
       456│             if command_event.command_should_run():
     → 457│                 exit_code = command.run(io)
       458│             else:
       459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

   7  ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/cleo/commands/base_command.py:119 in run
       117│         io.input.validate()
       118│ 
     → 119│         status_code = self.execute(io)
       120│ 
       121│         if status_code is None:

   6  ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/cleo/commands/command.py:62 in execute
        60│ 
        61│         try:
     →  62│             return self.handle()
        63│         except KeyboardInterrupt:
        64│             return 1

   5  ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/poetry_plugin_export/command.py:109 in handle
       107│         exporter.with_credentials(self.option("with-credentials"))
       108│         exporter.with_urls(not self.option("without-urls"))
     → 109│         exporter.export(fmt, Path.cwd(), output or self.io)
       110│ 
       111│         return 0

   4  ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/poetry_plugin_export/exporter.py:80 in export
        78│             raise ValueError(f"Invalid export format: {fmt}")
        79│ 
     →  80│         getattr(self, self.EXPORT_METHODS[fmt])(cwd, output)
        81│ 
        82│     def _export_generic_txt(

   3  ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/poetry_plugin_export/exporter.py:95 in _export_generic_txt
        93│         )
        94│ 
     →  95│         for dependency_package in get_project_dependency_packages(
        96│             self._poetry.locker,
        97│             project_requires=root.all_requires,

   2  ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/poetry_plugin_export/walker.py:97 in get_project_dependency_packages
        95│         selected.append(dependency)
        96│ 
     →  97│     for package, dependency in get_project_dependencies(
        98│         project_requires=selected,
        99│         locked_packages=repository.packages,

   1  ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/poetry_plugin_export/walker.py:125 in get_project_dependencies
       123│         )
       124│ 
     → 125│     nested_dependencies = walk_dependencies(
       126│         dependencies=project_requires,
       127│         packages_by_name=packages_by_name,

  RuntimeError

  Dependency walk failed at grpcio (>=1.54.2)

  at ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/poetry_plugin_export/walker.py:155 in walk_dependencies
      151│             requirement, packages_by_name, nested_dependencies
      152│         )
      153│ 
      154│         if not locked_package:
    → 155│             raise RuntimeError(f"Dependency walk failed at {requirement}")
      156│ 
      157│         if requirement.extras:
      158│             locked_package = locked_package.with_features(requirement.extras)
      159│ 

[dimbleby]

#176, #183, #186, #187.

Having a fifth open duplicate will not help

[sbor23]

And #168 just for completeness' sake. Adding here just because this seems to be the original issue.

[sbor23]

@radoering would you please re-open this ticket? As @neersighted confirmed this is not fixed.

[dimbleby]

just as when you asked the same thing in May: no.

There are already too many open issues tracking this, there is no point in adding yet further duplicates.