Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Normalize sdist filenames #484

Merged
merged 2 commits into from
Oct 5, 2022
Merged

Conversation

dimbleby
Copy link
Contributor

Addresses python-poetry/poetry#5782

Needs a companion fix in poetry proper: poetry "knows" the names of the sdists that poetry-core builds, so in case of mismatch source distributions won't be uploaded at all.

This MR leaves poetry insisting on normalizing the package name at upload (ie it did this normalization prior to this MR, and it still does). That is: if you have a package foo.bar then that gets uploaded as foo-bar. The advance that this MR makes is that both sdist and wheels are successfully uploaded now (and also poetry is more spec-compliant).

Currently it's not possible to (i) leave the package name unnormalized and also (ii) follow the packaging specs re wheel and sdist filenames - see pypi/warehouse#10030. So while in principle it's straightforward for poetry to do that - it should use the pretty_name on the metadata, not the canonicalized name - it would currently make things worse not better.

@neersighted
Copy link
Member

Haven't had time to take a close look yet -- where does this put us relative to PEP 625?

@dimbleby
Copy link
Contributor Author

this should be in agreement with that pep

@radoering radoering force-pushed the normalize-sdists branch 2 times, most recently from bb1a230 to 9449fb0 Compare October 5, 2022 15:26
@radoering radoering enabled auto-merge (rebase) October 5, 2022 15:26
radoering
radoering previously approved these changes Oct 5, 2022
@sonarcloud
Copy link

sonarcloud bot commented Oct 5, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@radoering radoering enabled auto-merge (rebase) October 5, 2022 18:13
@radoering radoering merged commit 533ffb5 into python-poetry:main Oct 5, 2022
@dimbleby dimbleby deleted the normalize-sdists branch October 5, 2022 18:36
dlaehnemann added a commit to bioconda/bioconda-recipes that referenced this pull request Mar 20, 2023
… underscores)

The official specs for sdist file names require that the package name is in snake case, see these places:
* https://peps.python.org/pep-0625/#specification
* https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode
* https://peps.python.org/pep-0491/#escaping-and-unicode

[`poetry` switched to conforming to these specs with version `1.2.2`](https://github.com/python-poetry/poetry/releases/tag/1.2.2), see:
* poetry-core pull request with the actual code changes: python-poetry/poetry-core#484
* poetry pull request pulling them in: python-poetry/poetry#6621

Thus, all (bio-)conda recipes whose sources are built with `poetry` and uploaded to pypi are likely to face the same issue.

As `snakemake-wrapper-utils` only contains hyphens (`-`) as non-word characters, the suggested `replace("-","_")` should be safe and enough.

But a thorough solution should be implemented in `grayskull` (that does conda recipe templating for pypi packages), and in the tooling of `conda-forge` and `bioconda`. One solution to make such recipes more future-proof would be to make tooling respect the [official guidance to query the pypi JSON API to get download urls](https://warehouse.pypa.io/api-reference/integration-guide.html#official-guidance). Examples of how to do this are here:
* [stackoverflow answer with example code of querying the pypi JSON API for a download URL](https://stackoverflow.com/a/48327216)
* [link to the pypi code for generating the currently used standardized redirect links at `pypi.io`, that also uses the JSON API](pypi/warehouse#13240 (comment))
dlaehnemann added a commit to bioconda/bioconda-recipes that referenced this pull request Mar 20, 2023
* update snakemake-wrapper-utils to 0.5.2

* make package name in sdist `.tar.gz` filename conform to specs (using underscores)

The official specs for sdist file names require that the package name is in snake case, see these places:
* https://peps.python.org/pep-0625/#specification
* https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode
* https://peps.python.org/pep-0491/#escaping-and-unicode

[`poetry` switched to conforming to these specs with version `1.2.2`](https://github.com/python-poetry/poetry/releases/tag/1.2.2), see:
* poetry-core pull request with the actual code changes: python-poetry/poetry-core#484
* poetry pull request pulling them in: python-poetry/poetry#6621

Thus, all (bio-)conda recipes whose sources are built with `poetry` and uploaded to pypi are likely to face the same issue.

As `snakemake-wrapper-utils` only contains hyphens (`-`) as non-word characters, the suggested `replace("-","_")` should be safe and enough.

But a thorough solution should be implemented in `grayskull` (that does conda recipe templating for pypi packages), and in the tooling of `conda-forge` and `bioconda`. One solution to make such recipes more future-proof would be to make tooling respect the [official guidance to query the pypi JSON API to get download urls](https://warehouse.pypa.io/api-reference/integration-guide.html#official-guidance). Examples of how to do this are here:
* [stackoverflow answer with example code of querying the pypi JSON API for a download URL](https://stackoverflow.com/a/48327216)
* [link to the pypi code for generating the currently used standardized redirect links at `pypi.io`, that also uses the JSON API](pypi/warehouse#13240 (comment))

* closing bracket; use jinja2 `replace()` filter instead of python `str.replace()`

* update sha256 according to https://pypi.org/pypi/snakemake-wrapper-utils/0.5.2/json
cokelaer pushed a commit to cokelaer/bioconda-recipes that referenced this pull request Apr 28, 2023
* update snakemake-wrapper-utils to 0.5.2

* make package name in sdist `.tar.gz` filename conform to specs (using underscores)

The official specs for sdist file names require that the package name is in snake case, see these places:
* https://peps.python.org/pep-0625/#specification
* https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode
* https://peps.python.org/pep-0491/#escaping-and-unicode

[`poetry` switched to conforming to these specs with version `1.2.2`](https://github.com/python-poetry/poetry/releases/tag/1.2.2), see:
* poetry-core pull request with the actual code changes: python-poetry/poetry-core#484
* poetry pull request pulling them in: python-poetry/poetry#6621

Thus, all (bio-)conda recipes whose sources are built with `poetry` and uploaded to pypi are likely to face the same issue.

As `snakemake-wrapper-utils` only contains hyphens (`-`) as non-word characters, the suggested `replace("-","_")` should be safe and enough.

But a thorough solution should be implemented in `grayskull` (that does conda recipe templating for pypi packages), and in the tooling of `conda-forge` and `bioconda`. One solution to make such recipes more future-proof would be to make tooling respect the [official guidance to query the pypi JSON API to get download urls](https://warehouse.pypa.io/api-reference/integration-guide.html#official-guidance). Examples of how to do this are here:
* [stackoverflow answer with example code of querying the pypi JSON API for a download URL](https://stackoverflow.com/a/48327216)
* [link to the pypi code for generating the currently used standardized redirect links at `pypi.io`, that also uses the JSON API](pypi/warehouse#13240 (comment))

* closing bracket; use jinja2 `replace()` filter instead of python `str.replace()`

* update sha256 according to https://pypi.org/pypi/snakemake-wrapper-utils/0.5.2/json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants