Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

validate extras against dependencies and in schema #542

Merged
merged 7 commits into from
Jan 9, 2023

Conversation

Panaetius
Copy link
Contributor

Resolves: python-poetry/poetry/issues/7226

  • Added tests for changed code.
  • Updated documentation for changed code.

With this change, extras are validated to only contain valid characters and extras that reference dependencies that can't be found in the main dependency group raise a warning in poetry check (not in poetry lock/poetry install, though)

@Panaetius
Copy link
Contributor Author

I've addressed the PR comments.

@radoering do you have any thoughts on whether this being a warning in poetry check as per this PR is fine or should it be added to check_result["errors"] instead?

@radoering
Copy link
Member

Good point. Comparing it to the other warnings and errors, I think it should be an error.

It's just an edge case but I just noticed extras should be checked if there are no dependencies. Can you unindent the new block by one level and replace config["dependencies"] by config.get("dependencies", {})?

@Panaetius Panaetius force-pushed the fix-validating-extras branch from 086c6c0 to 017df47 Compare January 8, 2023 11:02
@Panaetius Panaetius requested a review from radoering January 8, 2023 11:17
@sonarqubecloud
Copy link

sonarqubecloud bot commented Jan 9, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@radoering radoering merged commit 978fb55 into python-poetry:main Jan 9, 2023
@radoering radoering mentioned this pull request Jan 24, 2023
@lovetheguitar
Copy link

@radoering FYI, this was a breaking change for some.

E.g. isort 5.10.1 in our .pre-commit-config.yaml failed to install everywhere because of the 1.5 release.

https://github.com/PyCQA/isort/blob/9e4b5ee05264e6e566b18bea88b3073abd4b1014/pyproject.toml#L51

I guess version pinning of extras is not allowed/correct there (anymore)?

asottile-sentry added a commit to getsentry/pypi that referenced this pull request Jan 30, 2023
works around some poetry breakage python-poetry/poetry-core#542
@radoering
Copy link
Member

radoering commented Jan 30, 2023

I guess version pinning of extras is not allowed/correct there (anymore)?

It's still possible to define constraints for extras the same way as before: By defining an optional dependency as explained in the documentation and now done by isort in PyCQA/isort#2078. You can verify that the latest isort release has the same constraint on pip-shims as before1:

$ curl -s https://pypi.org/pypi/isort/5.12.0/json | jq .info.requires_dist | grep pip-shims
  "pip-shims (>=0.5.2) ; extra == \"pipfile-deprecated-finder\"",

To the best of my knowledge, the way isort specified the extra before is not documented and was never officially supported by poetry. We were not aware that this was possible and even resulted in valid metadata1.

1 Before the invalid extra was just ignored as can be seen in distributions of isort 5.11.4. (Distributions of isort 5.11.5 were fixed some hours ago.)

br3ndonland added a commit to br3ndonland/fastenv that referenced this pull request Feb 9, 2023
poetry-core 1.5.0 introduced a breaking change (in a minor version) by
altering parsing of `extras` metadata (python-poetry/poetry-core#542).
This broke package builds from source for isort (PyCQA/isort#2077),
which is how pre-commit installs isort.

This commit will update pre-commit to the fixed version of isort.
br3ndonland added a commit to br3ndonland/inboard that referenced this pull request Feb 9, 2023
poetry-core 1.5.0 introduced a breaking change (in a minor version) by
altering parsing of `extras` metadata (python-poetry/poetry-core#542).
This broke package builds from source for isort (PyCQA/isort#2077),
which is how pre-commit installs isort.

This commit will update pre-commit to the fixed version of isort.
br3ndonland added a commit to br3ndonland/template-python that referenced this pull request Feb 9, 2023
poetry-core 1.5.0 introduced a breaking change (in a minor version) by
altering parsing of `extras` metadata (python-poetry/poetry-core#542).
This broke package builds from source for isort (PyCQA/isort#2077),
which is how pre-commit installs isort.

This commit will update pre-commit to the fixed version of isort.
mwalbeck pushed a commit to mwalbeck/docker-python-poetry that referenced this pull request Feb 28, 2023
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [poetry](https://python-poetry.org/) ([source](https://github.com/python-poetry/poetry), [changelog](https://python-poetry.org/history/)) | minor | `1.3.2` -> `1.4.0` |

---

### Release Notes

<details>
<summary>python-poetry/poetry</summary>

### [`v1.4.0`](https://github.com/python-poetry/poetry/blob/HEAD/CHANGELOG.md#&#8203;140---2023-02-27)

[Compare Source](python-poetry/poetry@1.3.2...1.4.0)

##### Added

-   **Add a modern installer (`installer.modern-installation`) for faster installation of packages and independence from pip** ([#&#8203;6205](python-poetry/poetry#6205)).
-   Add support for `Private ::` trove classifiers ([#&#8203;7271](python-poetry/poetry#7271)).
-   Add the version of poetry in the `@generated` comment at the beginning of the lock file ([#&#8203;7339](python-poetry/poetry#7339)).
-   Add support for `virtualenvs.prefer-active-python` when running `poetry new` and `poetry init` ([#&#8203;7100](python-poetry/poetry#7100)).

##### Changed

-   **Deprecate the old installer, i.e. setting `experimental.new-installer` to `false`** ([#&#8203;7358](python-poetry/poetry#7358)).
-   Remove unused `platform` field from cached package info and bump the cache version ([#&#8203;7304](python-poetry/poetry#7304)).
-   Extra dependencies of the root project are now sorted in the lock file ([#&#8203;7375](python-poetry/poetry#7375)).
-   Remove upper boundary for `importlib-metadata` dependency ([#&#8203;7434](python-poetry/poetry#7434)).
-   Validate path dependencies during use instead of during construction ([#&#8203;6844](python-poetry/poetry#6844)).
-   Remove the deprecated `repository` modules ([#&#8203;7468](python-poetry/poetry#7468)).

##### Fixed

-   Fix an issue where an unconditional dependency of an extra was not installed in specific environments ([#&#8203;7175](python-poetry/poetry#7175)).
-   Fix an issue where a pre-release of a dependency was chosen even if a stable release fulfilled the constraint ([#&#8203;7225](python-poetry/poetry#7225), [#&#8203;7236](python-poetry/poetry#7236)).
-   Fix an issue where HTTP redirects were not handled correctly during publishing ([#&#8203;7160](python-poetry/poetry#7160)).
-   Fix an issue where `poetry check` did not handle the `-C, --directory` option correctly ([#&#8203;7241](python-poetry/poetry#7241)).
-   Fix an issue where the subdirectory information of a git dependency was not written to the lock file ([#&#8203;7367](python-poetry/poetry#7367)).
-   Fix an issue where the wrong Python version was selected when creating an virtual environment ([#&#8203;7221](python-poetry/poetry#7221)).
-   Fix an issue where packages that should be kept were uninstalled when calling `poetry install --sync` ([#&#8203;7389](python-poetry/poetry#7389)).
-   Fix an issue where an incorrect value was set for `sys.argv[0]` when running installed scripts ([#&#8203;6737](python-poetry/poetry#6737)).
-   Fix an issue where hashes in `direct_url.json` files were not written according to the specification ([#&#8203;7475](python-poetry/poetry#7475)).
-   Fix an issue where poetry commands failed due to special characters in the path of the project or virtual environment ([#&#8203;7471](python-poetry/poetry#7471)).
-   Fix an issue where poetry crashed with a `JSONDecodeError` when running a Python script that produced certain warnings ([#&#8203;6665](python-poetry/poetry#6665)).

##### Docs

-   Add advice on how to maintain a poetry plugin ([#&#8203;6977](python-poetry/poetry#6977)).
-   Update tox examples to comply with the latest tox release ([#&#8203;7341](python-poetry/poetry#7341)).
-   Mention that the `poetry export` can export `constraints.txt` files ([#&#8203;7383](python-poetry/poetry#7383)).
-   Add clarifications for moving configuration files ([#&#8203;6864](python-poetry/poetry#6864)).
-   Mention the different types of exact version specifications ([#&#8203;7503](python-poetry/poetry#7503)).

##### poetry-core ([`1.5.1`](https://github.com/python-poetry/poetry-core/releases/tag/1.5.1))

-   Improve marker handling ([#&#8203;528](python-poetry/poetry-core#528),
    [#&#8203;534](python-poetry/poetry-core#534),
    [#&#8203;530](python-poetry/poetry-core#530),
    [#&#8203;546](python-poetry/poetry-core#546),
    [#&#8203;547](python-poetry/poetry-core#547)).
-   Validate whether dependencies referenced in `extras` are defined in the main dependency group ([#&#8203;542](python-poetry/poetry-core#542)).
-   Poetry no longer generates a `setup.py` file in sdists by default ([#&#8203;318](python-poetry/poetry-core#318)).
-   Fix an issue where trailing newlines were allowed in `tool.poetry.description` ([#&#8203;505](python-poetry/poetry-core#505)).
-   Fix an issue where the name of the data folder in wheels was not normalized ([#&#8203;532](python-poetry/poetry-core#532)).
-   Fix an issue where the order of entries in the RECORD file was not deterministic ([#&#8203;545](python-poetry/poetry-core#545)).
-   Fix an issue where zero padding was not correctly handled in version comparisons ([#&#8203;540](python-poetry/poetry-core#540)).
-   Fix an issue where sdist builds did not support multiple READMEs ([#&#8203;486](python-poetry/poetry-core#486)).

##### poetry-plugin-export ([`^1.3.0`](https://github.com/python-poetry/poetry-plugin-export/releases/tag/1.3.0))

-   Fix an issue where the export failed if there was a circular dependency on the root package ([#&#8203;118](python-poetry/poetry-plugin-export#118)).

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4xNTIuNSIsInVwZGF0ZWRJblZlciI6IjM0LjE1Mi41In0=-->

Reviewed-on: https://git.walbeck.it/walbeck-it/docker-python-poetry/pulls/655
Co-authored-by: renovate-bot <bot@walbeck.it>
Co-committed-by: renovate-bot <bot@walbeck.it>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

extras are not validated against dependencies
3 participants