You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have searched the issues of this repo and believe that this is not a duplicate.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
Issue
TLDR; installed package detection breaks when multiple site-specific directories exist.
In certain scenarios, there exists multiple site-specific package directories. The most
common of this is due to the distinction between purelib and platlib paths
(see sysconfig#Installation Paths).
Different distros have their own view of site-specific file directories. And I suspect, different python installers do too.
The issue can be reproduced using the snippet provided at the end. Here, in an environment with virtualenv >= 20.0.0, tox creates the environment such that lib64 directory is not linked to lib.
Note that;
we use a Red Hat distro (Fedora) as this is one where the platlib directory differs from the purelib directory.
if the virtual environment was created using the venv module or virtualenv < 20.0.0, lib64 will be a symlink to lib.
The root cause for this is because the package is being detected as a non-standard package since
only the first seen site-package directory is being checked.
The checking of multiple directories were briefly talked about in a comment in #1786. However, I do not think that was resolved there.
Possible Solutions
In both these cases, Env.site_packages will need to be an iterable.
Iterate over all paths in sys.path filtering for site-packages. This will be the simplest solution, however will not account for cases like above where the directory is /usr/local/lib/site-python.
Use site.getsitepackages(). This will not work for environments created using virtualenv < 20.0.0 as a bundled version for site.py was being used. Since in these environments lib64 is already being symlinked, we could simply fallback to searching for site-packages assuming that the virtual environments will not be creating dist-packages. We should also bump minimum version for poetry's virtualenv dependency as well.
Not sure if there is a solution without any trade-off.
Reproducer
docker run --rm -i fedora:31 <<SCRIPT_EOFset -xdnf -qy install -y pippip install --quiet --upgrade poetry tox virtualenv>=20.0.0install -d poetry-issue-reproducer/poetry_issue_reproducerpushd poetry-issue-reproducertouch poetry_issue_reproducer/__init__.pycat > pyproject.toml <<EOF[build-system]requires = ["poetry>=1.0"]build-backend = "poetry.masonry.api"[tool.poetry]name = "poetry-issue-reproducer"packages = [ { include = "poetry_issue_reproducer" }]version = "1.0.0-alpha.0"description = "A reproducer for a poetry issue"authors = [ "Issue Reproducer <issues@noreply.com>"]license = "MIT"[tool.poetry.dependencies]python = "^3.7"[tool.poetry.dev-dependencies]tox = "^3.13"[tool.tox]legacy_tox_ini = """[tox]isolated_build = Trueenvlist = py[testenv]whitelist_externals = poetrycommands = poetry install -vvv"""EOF# first run works as expectedtox# second run updates packages, this is not expectedtoxSCRIPT_EOF
The text was updated successfully, but these errors were encountered:
-vvv
option).Issue
TLDR; installed package detection breaks when multiple site-specific directories exist.
In certain scenarios, there exists multiple site-specific package directories. The most
common of this is due to the distinction between
purelib
andplatlib
paths(see sysconfig#Installation Paths).
Another example is the
python:2.7
image.Different distros have their own view of site-specific file directories. And I suspect, different python installers do too.
The issue can be reproduced using the snippet provided at the end. Here, in an environment with
virtualenv >= 20.0.0
, tox creates the environment such thatlib64
directory is not linked tolib
.Note that;
platlib
directory differs from thepurelib
directory.venv
module orvirtualenv < 20.0.0
,lib64
will be a symlink tolib
.virtualenv
is by design (lib64 directory is no longer symlinked to lib? pypa/virtualenv#1751)Once executed you will notice that the first tox run executes as expected.
However, the second run will update all packages. This is not expected.
The cause for this is the differing source types for the installed package and required package.
poetry/poetry/puzzle/solver.py
Lines 95 to 96 in 4715f1d
The root cause for this is because the package is being detected as a non-standard package since
only the first seen site-package directory is being checked.
poetry/poetry/repositories/installed_repository.py
Lines 45 to 52 in 4715f1d
The checking of multiple directories were briefly talked about in a comment in #1786. However, I do not think that was resolved there.
Possible Solutions
In both these cases,
Env.site_packages
will need to be an iterable.sys.path
filtering forsite-packages
. This will be the simplest solution, however will not account for cases like above where the directory is/usr/local/lib/site-python
.site.getsitepackages()
. This will not work for environments created usingvirtualenv < 20.0.0
as a bundled version forsite.py
was being used. Since in these environmentslib64
is already being symlinked, we could simply fallback to searching forsite-packages
assuming that the virtual environments will not be creatingdist-packages
. We should also bump minimum version for poetry'svirtualenv
dependency as well.Not sure if there is a solution without any trade-off.
Reproducer
The text was updated successfully, but these errors were encountered: