Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

maturin upload: Invalid distribution file. PKG-INFO not found #2362

Closed
1 of 2 tasks
ZJaume opened this issue Dec 3, 2024 · 8 comments · Fixed by #2367
Closed
1 of 2 tasks

maturin upload: Invalid distribution file. PKG-INFO not found #2362

ZJaume opened this issue Dec 3, 2024 · 8 comments · Fixed by #2367
Labels
bug Something isn't working

Comments

@ZJaume
Copy link

ZJaume commented Dec 3, 2024 •
edited
Loading

Bug Description

Maturin upload command is failing to upload the source distribution because PyPi complains about PKG-INFO not found. The action job logging messages can be seen here: https://github.com/ZJaume/heliport/actions/runs/12141036958/job/33852428765#step:4:39

It seems that is checking for heliport/PKG-INFO but the tarball has heliport-0.8.0/PKG-INFO. It is strange, because for the previous version it worked and the version suffix was there also. Did PyPi changed the packaging criteria? Maybe I am missing something?

Your maturin version (maturin --version)

1.7.7

Your Python version (python -V)

Python 3.10.12

Your pip version (pip -V)

pip 24.0

What bindings you're using

pyo3

Does cargo build work?

  • Yes, it works

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

  • Yes

Steps to Reproduce

I think nobody can reproduce this with my package, as I am the only person with permissions, but I did the upload manually from my machine and still fails.

  1. Download the sdist artifact produced by GA from: https://github.com/ZJaume/heliport/actions/runs/12141036958/job/33852176378#step:4:29
  2. Unzip the downloaded file.
  3. maturin upload heliport-0.8.0.tar.gz and obtain
🚀 Uploading 1 packages
💥 maturin failed
  Caused by: 💥 Failed to upload "heliport-0.8.0.tar.gz" (53.3 MB)
  Caused by: Failed to upload the wheel with status 400: <html>
 <head>
  <title>400 Invalid distribution file. PKG-INFO not found at heliport/PKG-INFO</title>
 </head>
 <body>
  <h1>400 Invalid distribution file. PKG-INFO not found at heliport/PKG-INFO</h1>
  The server could not comply with the request since it is either malformed or otherwise incorrect.<br/><br/>
Invalid distribution file. PKG-INFO not found at heliport/PKG-INFO
@ZJaume ZJaume added the bug Something isn't working label Dec 3, 2024
@messense
Copy link
Member

messense commented Dec 3, 2024

Not sure what's going on, can you try twine upload it?

@ZJaume
Copy link
Author

ZJaume commented Dec 3, 2024

Same error

ERROR    HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
         Invalid distribution file. PKG-INFO not found at heliport/PKG-INFO

@messense
Copy link
Member

messense commented Dec 4, 2024

Related pypi code

https://github.com/pypi/warehouse/blob/c9231a69413f218adcc8c2236b78059f524eadb6/warehouse/forklift/legacy.py#L332-L354

will investigate later when I'm free.

@rdong8
Copy link

rdong8 commented Dec 4, 2024

Unsure if related, but I am unable to get maturin publish to work:

Please enter your password: [hidden]
🚀 Uploading 2 packages
⛔ 403 Username/Password authentication is no longer supported. Migrate to API Tokens or Trusted Publishers instead. See https://pypi.org/help/#apitoken and https://pypi.org/help/#trusted-publishers
💥 maturin failed
  Caused by: Username and/or password are possibly wrong

I'm using my PyPI username and API token.

@messense
Copy link
Member

messense commented Dec 4, 2024

That is expected, the error message is pretty clear, see https://pypi.org/help/#apitoken you must use __token__ as username when using apitoken.

@messense messense mentioned this issue Dec 4, 2024
Merged
@ZJaume
Copy link
Author

ZJaume commented Dec 4, 2024

Related pypi code

https://github.com/pypi/warehouse/blob/c9231a69413f218adcc8c2236b78059f524eadb6/warehouse/forklift/legacy.py#L332-L354

Thanks for finding the relevant code. Now I see that the issue is caused by the package data (heliport.data)

tar tvf heliport-0.8.0.tar.gz | tail

-rw-r--r-- 1001/127      61135 2024-12-03 13:49 heliport-0.8.0/Cargo.lock
-rw-r--r-- 1001/127       1184 2024-12-03 13:49 heliport-0.8.0/pyproject.toml
-rw-r--r-- 0/0            5832 1970-01-01 00:00 heliport-0.8.0/PKG-INFO
-rw-r--r-- 0/0              14 1970-01-01 00:00 heliport.data/platlib/heliport/.gitignore

causing the common prefix to be heliport and not heliport-0.8.0. Now you may wonder why it is there that gitignore. The thing is that I'm generating binarized models during compilation with build.rs and I want those to be included in the wheel. Then I found that, if the data folder does not exist at the time maturin build -r is executed and reads the pyproject (models are generated aferwards), it is not included in the wheel. That is not a big deal for development, but the GitHub Actions start from a clean repo so they fail. The only way I found (without manually doing an mkdir before running maturin build in GA) to have the directory already there, is including it in git and explicitly putting it in the pyproject.toml

[tool.maturin]
data = "heliport.data"

Another solution could be to set the data directory to a different name, but it seems that, even the files are included in the wheel, they are not being unpacked on the package directory:

Archive:  target/wheels/heliport-0.8.1-cp310-cp310-manylinux_2_34_x86_64.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
     5832  2024-12-04 11:21   heliport-0.8.1.dist-info/METADATA
      108  2024-12-04 11:21   heliport-0.8.1.dist-info/WHEEL
       44  2024-12-04 11:21   heliport-0.8.1.dist-info/entry_points.txt
    35149  2024-12-04 11:21   heliport-0.8.1.dist-info/licenses/LICENSE
      115  2024-12-04 11:21   heliport/__init__.py
  4604800  2024-12-04 11:21   heliport/heliport.cpython-310-x86_64-linux-gnu.so
 13100940  2024-12-04 11:21   asdf/platlib/heliport/trigram.bin
 36101657  2024-12-04 11:21   asdf/platlib/heliport/word.bin
 31640661  2024-12-04 11:21   asdf/platlib/heliport/hexagram.bin
 20020969  2024-12-04 11:21   asdf/platlib/heliport/quadgram.bin
  2552827  2024-12-04 11:21   asdf/platlib/heliport/bigram.bin
     6018  2024-12-04 11:21   asdf/platlib/heliport/confidenceThresholds
       14  2024-12-04 11:21   asdf/platlib/heliport/.gitignore
 26053887  2024-12-04 11:21   asdf/platlib/heliport/quingram.bin
   252916  2024-12-04 11:21   asdf/platlib/heliport/unigram.bin
     1430  2024-12-04 11:21   heliport-0.8.1.dist-info/RECORD
---------                     -------
134377367                     16 files

ll venv/lib/python3.10/site-packages/heliport

total 4.5M
-rw-rw-r-- 1 jzaragoza jzaragoza  115 Dec  4 11:21 __init__.py
drwxrwxr-x 2 jzaragoza jzaragoza 4.0K Dec  4 11:21 __pycache__
-rwxrwxr-x 1 jzaragoza jzaragoza 4.4M Dec  4 11:21 heliport.cpython-310-x86_64-linux-gnu.so

@ZJaume
Copy link
Author

ZJaume commented Dec 4, 2024

Could setting the data dir to a diferent name and that not being unpacked from the wheel be another bug?

@messense
Copy link
Member

messense commented Dec 4, 2024

not being unpacked from the wheel be another bug

No, see https://packaging.python.org/en/latest/specifications/binary-distribution-format/#installing-a-wheel-distribution-1-0-py32-none-any-whl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Don't add wheel data to sdist PyO3/maturin
3 participants
@messense @ZJaume @rdong8