Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Poetry solve unnecessary dependency path #5731

Closed
3 tasks done
hungbie opened this issue May 31, 2022 · 4 comments
Closed
3 tasks done

Poetry solve unnecessary dependency path #5731

hungbie opened this issue May 31, 2022 · 4 comments
Labels
kind/bug Something isn't working as expected

Comments

@hungbie
Copy link

hungbie commented May 31, 2022

  • I am on the latest Poetry version.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
  • OS version and name: ubuntu 20.04 and MacOS 12.3.1
  • Poetry version: 1.1.13
  • Link of a Gist with the contents of your pyproject.toml file:
    example pyproject.toml that can reproduce the behavior:
[build-system]
requires = ["poetry-core<=1.0.4"]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "package"
version = "0.0.0"
description = "Application"
authors = ['author']

[tool.poetry.dependencies]
python = "3.8.*"
azure-identity = "^1.5"
portalocker = "^1.6"

[tool.poetry.dev-dependencies]
pytest = "*"
pytest-cov = "*"

Issue

azure-identity depends on msal-extension that in turn depends on portalocker.
msal-extension has this requirement:

"portalocker<3,>=1.0;python_version>='3.5' and platform_system!='Windows'",
"portalocker<3,>=1.6;python_version>='3.5' and platform_system=='Windows'",

Running in debug mode show that poetry tried to resolve 3 times:

 1: Version solving took 0.547 seconds.
   1: Tried 1 solutions.
   0: Retrying dependency resolution with the following overrides ({Package('msal-extensions', '0.3.0'): {'portalocker': <Dependency portalocker (>=1.6,<2.0)>}}).
  ....
 0: Different requirements found for portalocker (>=1.0,<2.0) with markers platform_system != "Windows" and portalocker (>=1.6,<2.0) with markers platform_system == "Windows".
   1: Version solving took 0.085 seconds.
   1: Tried 1 solutions.
   0: Retrying dependency resolution with the following overrides ({Package('msal-extensions', '0.3.0'): {'portalocker': <Dependency portalocker (>=1.0,<2.0)>}}).
   ...
 1: Version solving took 0.232 seconds.
   1: Tried 1 solutions.
   0: Complete version solving took 1.151 seconds with 2 overrides
   0: Resolved with overrides: ({Package('msal-extensions', '0.3.0'): {'portalocker': <Dependency portalocker (>=1.0,<2.0)>}}), ({Package('msal-extensions', '0.3.0'): {'portalocker': <Dependency portalocker (>=1.6,<2.0)>}})

In normal situation I understand why it would have this behaviour, but I wonder if there is anyway to optimize it? For example, the OS is definitely not Window, why do we have to try this?
Next, I tried to pin the version of portalocker to >=1.6 so that it would ignore the other condition, but it still tried to resolve for portalocker version >=1.0.
Would this be a waste of resource? In this toy example it seems ok, but in our current project, each run of the solver cost us 10 minutes, so rerun this 3 times and the poetry lock time is 30 minutes.
And we have this similar problem with other dependencies (for example pandas and numpy), making the numbers of retry bigger, and hence the lock time becomes hours.

@hungbie hungbie added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels May 31, 2022
@dimbleby
Copy link
Contributor

poetry tries to produce a solution that is valid everywhere, not only in the place where you are running poetry lock.

#4956 contains work in the general direction that you suggest, I think this can be treated as a duplicate of that.

@hungbie
Copy link
Author

hungbie commented Jun 2, 2022

poetry tries to produce a solution that is valid everywhere, not only in the place where you are running poetry lock.

#4956 contains work in the general direction that you suggest, I think this can be treated as a duplicate of that.

Yes this seems like what we need, especially for big project where we have our own targeted environment and don’t need to produce a solution that valid everywhere! Hope it will be released soon!

@dimbleby
Copy link
Contributor

so please close this as a duplicate

@Secrus Secrus closed this as completed Jun 16, 2022
@mkniewallner mkniewallner removed the status/triage This issue needs to be triaged label Jun 16, 2022
Copy link

github-actions bot commented Mar 1, 2024

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Something isn't working as expected
Projects
None yet
Development

No branches or pull requests

4 participants