Lock file contains an absolute path when using extras

[FelixDQ]

Description

When a path dependency uses extras with another path dependency, the resulting lock files contains an absolute path even though all the paths in pyproject.toml are relative.

I created a simple reproduction example here https://github.com/FelixDQ/poetry-lock-file-repro which contains 3 projects.

proj2 pyproject.toml:

[tool.poetry.dependencies]
python = "^3.8"
proj3 = {path = "../proj3", optional = true}

[tool.poetry.extras]
example = ["proj3"]

proj1 pyprojct.toml:

[tool.poetry.dependencies]
python = "^3.8"
proj2 = {path = "../proj2", extras = ["example"]}

This results in the following getting added to the lock file for proj1:

[package.extras]
example = ["proj3 @ file:///Users/felix/Projects/tmrow/poetry-lock-file-repro/proj3"]

This causes problems for us because the lock file is updated every time a developer runs poetry lock—- no-update even though nothing has changed. I would expect the file path to be relative as well

Workarounds

Not as far as i know.

Poetry Installation Method

pipx

Operating System

MacOs

Poetry Version

1.8.0

Poetry Configuration

cache-dir = "/Users/felix/Library/Caches/pypoetry"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /Users/felix/Library/Caches/pypoetry/virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true

Python Sysconfig

No response

Example pyproject.toml

No response

Poetry Runtime Logs

.

[FelixDQ]

I found this PR, so I'm a bit unsure if this is actually a bug as it seems like it was changed to this intentionally: python-poetry/poetry-core#512

I would appreciate any workarounds for not having the lockfile change depending on the users absolute path :-)

[eamanu]

Hi!

Please also see #5273 and pypa/pip#9127.

I think that we can add a new option to use relatives paths

[FelixDQ]

From reading the issues, it seems like it needs to use the absolute file:/// URL to be compatible with PEP 508. The poetry commands however seem to still work just fine after manually changes the path to a relative one.

Would it be possible for us to create a poetry plugin or similar that converts the path to a relative one?

[neersighted]

As noted, this is intentional, and this is not specific at all to extras. If you are not intending to build distributable content (wheels) from this package, I suggest you instead use dependency groups, which do not create metadata.

If you intend to create wheels and install them on another system, then you need to refer to the other package by name, even if you have a locally available copy on disk.

If it's not quite either of those (that is, you want a path dependency during development and a dependency by name at build time), this is a duplicate of #1168.

[FelixDQ]

Thank you @neersighted! I guess that we've only seen this in our lockfiles after we started using extras. We don't intend to distribute wheels. We have a monorepo setup and would just like better control over which dependencies we are including in different projects.

Maybe I'm misunderstanding dependency groups, but is there a way to install optional ones when importing the project in another project. For example:

proj2:

[tool.poetry.dependencies]
python = "^3.8"

[tool.poetry.group.example]
optional = true

[tool.poetry.group.example.dependencies]
proj3 = {path = "../proj3"}

proj1:

[tool.poetry.dependencies]
python = "^3.8"
proj2 = {path = "../proj2"}

How can I specify in proj1 that it should include the optional group example and thereby install proj3?

[neersighted]

Ah, you are correct in that there is no mechanism for this; indeed, not creating metadata means that (without Poetry-specific mechanics) there is no way to communicate that dependency information across the monorepo using Poetry alone.

This is instead a duplicate of #2270; please see some solutions there, as well as linked issues with several design proposals for Poetry itself.

[FelixDQ]

Thank you! I will take a look at those

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.