Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duplicate lines in the wheel RECORD file when a native extension is built #7505

Closed
4 tasks done
SpecLad opened this issue Feb 12, 2023 · 6 comments · Fixed by python-poetry/poetry-core#555
Closed
4 tasks done
Labels
area/build-system Related to PEP 517 packaging (see poetry-core) area/core Related to the poetry-core library kind/bug Something isn't working as expected

Comments

@SpecLad
Copy link

SpecLad commented Feb 12, 2023

  • Poetry version: 1.3.2
  • Python version: 3.11.2
  • OS version and name: Windows 11
  • pyproject.toml: see below
  • I am on the latest stable Poetry version, installed using a recommended method.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have consulted the FAQ and blog for any relevant entries or release notes.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Create a project as follows:

pyproject.toml:

[tool.poetry]
name = "poetry-test"
version = "0.1.0"
description = ""
authors = ["John Smith <john@example.com>"]
packages = [{include = "poetry_test"}]

[tool.poetry.dependencies]
python = "^3.11"

[tool.poetry.build]
generate-setup-file = true
script = "build.py"

[build-system]
requires = ["poetry-core", "setuptools"]
build-backend = "poetry.core.masonry.api"

build.py:

from setuptools import Extension

def build(setup_kwargs):
    setup_kwargs['ext_modules'] = [
        Extension(
            name="poetry_test.test",
            sources=["poetry_test/test.c"],
        ),
    ]

poetry_test\__init__.py: empty

poetry_test\test.c:

#define PY_SSIZE_T_CLEAN
#include <Python.h>

PyModuleDef def = {
    PyModuleDef_HEAD_INIT,
    "poetry_test.test",
    "",
};

PyObject *PyInit_test(void) {
    return PyModule_Create(&def);
}

Now build:

> poetry build

Preparing build environment with build-system requirements poetry-core, setuptoolsBuilding poetry-test (0.1.0)
running build
running build_py
creating C:\Users\dpb\Documents\Source\test\poetry-test\build
creating C:\Users\dpb\Documents\Source\test\poetry-test\build\lib.win-amd64-cpython-311
creating C:\Users\dpb\Documents\Source\test\poetry-test\build\lib.win-amd64-cpython-311\poetry_test
copying poetry_test\__init__.py -> C:\Users\dpb\Documents\Source\test\poetry-test\build\lib.win-amd64-cpython-311\poetry_test
copying poetry_test\test.c -> C:\Users\dpb\Documents\Source\test\poetry-test\build\lib.win-amd64-cpython-311\poetry_test
running build_ext
building 'poetry_test.test' extension
[...snipped...]
C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.752.0_x64__qbz5n2kfra8p0\Lib\zipfile.py:1547: UserWarning: Duplicate name: 'poetry_test/test.c'
  return self._open_to_write(zinfo, force_zip64=force_zip64)
C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.752.0_x64__qbz5n2kfra8p0\Lib\zipfile.py:1547: UserWarning: Duplicate name: 'poetry_test/__init__.py'
  return self._open_to_write(zinfo, force_zip64=force_zip64)

Note the warnings.

Now, examine the RECORD file inside the wheel:

> tar -xOf .\dist\poetry_test-0.1.0-cp311-cp311-win_amd64.whl poetry_test-0.1.0.dist-info/RECORD
poetry_test/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
poetry_test/test.c,sha256=LV2Cw22u_-WHeTIx6r4_b8QOPSxRF_46zhXfV6HkUic,207
poetry_test/test.c,sha256=LV2Cw22u_-WHeTIx6r4_b8QOPSxRF_46zhXfV6HkUic,207
poetry_test/test.cp311-win_amd64.pyd,sha256=93EsxTSbGK32w4PIDSsKumB8e0HFoW7AMfvcibYfh80,10752
poetry_test/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
poetry_test-0.1.0.dist-info/METADATA,sha256=FRvgZ3bDNIep0KQkK6KxJetccHVs-0NGTd7B8egelQE,243
poetry_test-0.1.0.dist-info/WHEEL,sha256=q-KLDJQebzIZyKLb3oMnX-cCkloRZtwcIVhfy70qOs8,98
poetry_test-0.1.0.dist-info/RECORD,,

As you can see, test.c and __init__.py are listed twice. This contradicts the spec:

The RECORD file holds the list of installed files. It is a CSV file containing one record (line) per installed file.

@SpecLad SpecLad added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Feb 12, 2023
@SpecLad
Copy link
Author

SpecLad commented Feb 12, 2023

I believe this code is supposed to prevent this issue:

https://github.com/python-poetry/poetry-core/blob/b5d1934fed190bbfd1a303b44d5e2ef9a600efed/src/poetry/core/masonry/builders/wheel.py#L199

However, it doesn't work, because rel_path is a Windows path, while wheel.namelist() contains POSIX paths.

@radoering
Copy link
Member

There are even warnings about this in our test environment (which I have not attached much importance to so far):

tests/masonry/test_api.py::test_build_wheel_extended
tests/masonry/builders/test_complete.py::test_wheel_c_extension
tests/masonry/builders/test_complete.py::test_wheel_c_extension_src_layout
  C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\zipfile.py:1547: UserWarning: Duplicate name: 'extended/__init__.py'
    return self._open_to_write(zinfo, force_zip64=force_zip64)

tests/masonry/test_api.py::test_build_wheel_extended
tests/masonry/builders/test_complete.py::test_wheel_c_extension
tests/masonry/builders/test_complete.py::test_wheel_c_extension_src_layout
  C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\zipfile.py:1547: UserWarning: Duplicate name: 'extended/extended.c'
    return self._open_to_write(zinfo, force_zip64=force_zip64)

I assume that when fixing this bug, these tests can be extended to check that there are no duplicate entries in the RECORD file.

@radoering radoering added area/build-system Related to PEP 517 packaging (see poetry-core) area/core Related to the poetry-core library and removed status/triage This issue needs to be triaged labels Feb 12, 2023
@chapmanjacobd
Copy link

chapmanjacobd commented Mar 4, 2023

I guess the short-term solution here is to skip building on Windows ? Not sure what else I could do

https://github.com/chapmanjacobd/library/actions/runs/4328813746/jobs/7558895451#step:6:67

@gpongelli
Copy link

this happens also with poetry 1.4 .

other OS does not suffer this issue.

@gpongelli
Copy link

as per my comment on poetry-core, that PR fixes this issue.

Thanks,

Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/build-system Related to PEP 517 packaging (see poetry-core) area/core Related to the poetry-core library kind/bug Something isn't working as expected
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants