Option to bypass version resolving on specific dependency

[orctom]

Say I have an app that have to be runnable on these boxes:

• Ubuntu, with CUDA 11.4 (which requires the cu111 version)
• Ubuntu, with CUDA 10.2 (cu111 doesn't work)
• Ubuntu, CPU only
• MacOS

If only for Linux and MacOS, we can use environment markers such as:

torch = [
  {version = "1.8.1", markers = "sys_platform == 'linux'"},
  {version = "1.8.1", markers = "sys_platform == 'darwin'"}
]

But there're no options for different CUDA versions.

PROPOSAL:

[tool.poetry.dependencies]
#...
torch = {version = "1.8.1", resolve = False}
#...

For production Docker image:
Since pytorch with cuda is big (e.g. torch-1.9.1+cu111-cp38-cp38-linux_x86_64.whl is 2GB).
Downloading 2GB for frequent builds (CI/CD) is a wast of time, so I built base Docker images with different pytorch/CUDA version.
With pip install -r requirements.txt, it detects the pytorch dep is already fit, and moves on.
Expecting an option to have poetry install to treat pytorch as fit as well.

For local dev env:
Just run the plain pip install xxxx only once for the venv manually, or in poe plugin, if the default one does not fit.

[finswimmer]

Hello @orctom,

without resolving the dependencies, Poetry cannot create a lock file. But this is a core functionality. This is why it is very unlikely that we will implement something like you suggest.

fin swimmer

[getorca]

I think this is an excellent idea to allow forcing ignoring dependency conflicts. Either that or burn poetry to the ground. Ooops, did I type that.

eg:

Because no versions of spacy match >3.4.2,<4.0.0
 and spacy (3.4.2) depends on typer (>=0.3.0,<0.5.0), spacy (>=3.4.2,<4.0.0) requires typer (>=0.3.0,<0.5.0).
And because typer (0.6.1) depends on typer (0.6.1)
 and no versions of typer match >0.6.1,<0.7.0, spacy (>=3.4.2,<4.0.0) is incompatible with typer (>=0.6.1,<0.7.0).
So, because wee-benchmarking-tool depends on both typer (^0.6.1) and spacy (^3.4.2), version solving failed.

It's fine. Like how many packages do I need to fork, test and submit pull request for. It's ridiculous, if we enjoyed restrictions most of us wouldn't have become developers.

[neersighted]

You are expressing a different problem, @getorca. For those interested in the feature sort-of proposed in this issue, but without any concrete direction/definition of scope, see #6847.

You are in fact looking for #697, and its many duplicates. Keep in mind that Poetry provides useful hints to get yourself out of dependency hell, and ultimately it is really the upstream author's responsibility to provide as wide a set of constraints as can be reasonably achieved.

The ability to override constraints is a bandaid and a footgun (really, a foot-shotgun) in one, and should only ever be a temporary solution. There has never been a coherent proposal for the UI/metadata format as well as the technical implementation -- if you are interested in solving this problem in Poetry, feel free to start working on it, but please be aware that it's large, complex, invasive, and we'll have to live with/support it for a long time to come; thus, the review criteria will be burdensome and the process to come to consensus on a design will be long.

[getorca]

-- if you are interested in solving this problem in Poetry, feel free to start working on it, but please be aware that it's large, complex, invasive, and we'll have to live with/support it for a long time to come; thus, the review criteria will be burdensome and the process to come to consensus on a design will be long.

Not a good way to sell an already busy developer on contributing. Move faster. This is tech. Poetry has some decent use cases, but is more of a shot in the face than any marginally stubbed toe from dependency hell.

[neersighted]

Poetry is a voulenteer-maintained project. If it is not suitable for your use case, no one is forcing you to use it, or selling it to you. We are as interoperable with the rest of the ecosystem as we can be; there are no major barriers to other solutions, if they are more appropriate for you.

The simple matter is that no one who regularly works on Poetry or that is deeply familiar with the code is interested in solving this; until someone interested and motivated to get a design that meets your needs steps up, no amount of berating the Poetry maintainers and community will grant your ask.

We're off-topic for this issue, please direct any more discussion of dependency constraint overrides to #697.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.