poetry export removes sys_platform for transitive packages

[jbasila-orca]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: MacOS
• Poetry version: 1.1.11

[tool.poetry]
name = "foo"
version = "0.0.0"
description = "foo"
authors = ["foo"]

[tool.poetry.dependencies]
python = "^3.7"
docker = ">=2.5.1"
msal-extensions = ">=0.3.0,<0.4.0"

After locking the dependencies and exporting with the command:
poetry export --dev --without-hashes
The list will contain pywin32 without platform-specific. If we check the dependency tree with poetry show --tree we see that pywin32 comes from a transitive dependency of portalocker:

├── pywin32 227
├── requests >=2.14.2,<2.18.0 || >2.18.0
│   ├── certifi >=2017.4.17
│   ├── charset-normalizer >=2.0.0,<2.1.0
│   ├── idna >=2.5,<4
│   └── urllib3 >=1.21.1,<1.27
└── websocket-client >=0.32.0
msal-extensions 0.3.0
├── msal >=0.4.1,<2.0.0
│   ├── cryptography >=0.6,<38
│   │   └── cffi >=1.12
│   │       └── pycparser *
│   ├── pyjwt >=1.0.0,<3
│   │   └── cryptography >=3.3.1 (circular dependency aborted here)
│   └── requests >=2.0.0,<3
│       ├── certifi >=2017.4.17
│       ├── charset-normalizer >=2.0.0,<2.1.0
│       ├── idna >=2.5,<4
│       └── urllib3 >=1.21.1,<1.27
├── portalocker >=1.0,<2.0
│   └── pywin32 !=226
└── portalocker >=1.6,<2.0
    └── pywin32 !=226

If we export the dependencies using poetry export --dev --without-hashes:

certifi==2021.10.8; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.6"
cffi==1.15.0; python_version >= "3.6"
charset-normalizer==2.0.7; python_full_version >= "3.6.0" and python_version >= "3.6"
cryptography==35.0.0; python_version >= "3.6"
docker==5.0.3; python_version >= "3.6"
idna==3.3; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.6"
msal-extensions==0.3.0
msal==1.15.0
portalocker==1.7.1
pycparser==2.20; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.4.0" and python_version >= "3.6"
pyjwt==2.3.0; python_version >= "3.6"
pywin32==227
requests==2.26.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.6"
urllib3==1.26.7; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version < "4" and python_version >= "3.6"

Notice pywin32

If we change the pyproject.toml to contain only portalocker directly:

[tool.poetry]
name = "foo"
version = "0.0.0"
description = "foo"
authors = ["foo"]

[tool.poetry.dependencies]
python = "^3.7"
docker = ">=2.5.1"
portalocker = "==1.7.1"

and after poetry lock and then poetry export --dev --without-hashes:

certifi==2021.10.8; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.6"
charset-normalizer==2.0.7; python_full_version >= "3.6.0" and python_version >= "3.6"
docker==5.0.3; python_version >= "3.6"
idna==3.3; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.6"
portalocker==1.7.1
pywin32==227; sys_platform == "win32" and python_version >= "3.6" and platform_system == "Windows"
requests==2.26.0; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version >= "3.6"
urllib3==1.26.7; python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.6.0" and python_version < "4" and python_version >= "3.6"
websocket-client==1.2.1; python_version >= "3.6"

pywin32 now has sys_platform

[yoav-orca]

I tried to debug this it seems that portalocker has two markers:

1. platform_system == "Windows"
2. platform_system != "Windows"

And pywin32 has also a marker: platform_system == "Windows".

The issues seem to be in the __walk_dependency_level function.
If portalocker with the platform_system != "Windows" is examined first, then pywin32 marker is overridden in this line https://github.com/python-poetry/poetry/blob/master/poetry/packages/locker.py#L256-L258 with EmtpyMarker. It will not be examined again, since the code will only examine dependency one time regardless of marker.
If portalocker with the platform_system == "Windows" is examined first then poetry exports the information correctly (you can just swap the marker lines in the poetry.lock and see)

Update:
It seems that preventing the code from overriding the marker with EmptyMarker will cause the code to output the right result, something like this:

diff --git a/poetry/packages/locker.py b/poetry/packages/locker.py
index 6855c14c..37f7d435 100644
--- a/poetry/packages/locker.py
+++ b/poetry/packages/locker.py
@@ -253,9 +253,11 @@ class Locker:
                         if require.marker.is_empty():
                             require.marker = requirement.marker
                         else:
-                            require.marker = require.marker.intersect(
+                            require_marker = require.marker.intersect(
                                 requirement.marker
                             )
+                            if not require_marker.is_empty():
+                                require.marker = require_marker
 
                         require.marker = require.marker.intersect(locked_package.marker)
                         next_level_dependencies.append(require)

[dimbleby]

I think this is a duplicate of #3511

[yoav-orca]

@dimbleby You're right, it's a duplicate, and #4686 fixes the issue for me. @jbasila-orca I think you can close the issue

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.