Vendor a TOML v1.0.0 compatible TOML parser #10034
Labels
type: feature request
Request for a new feature
Comments
|
On a completely separate but related note: shouldn't PEP 518 specify supported TOML spec version for pyproject.toml to avoid these sorts of issues? Otherwise the pyproject.toml shared by countless tools, each reading it with a parser of their own, is just asking for issues like this... |
Good point, could you raise this on discuss.python.org? PEP amendment should happen there to reach the most related people. |
|
FWIW, I think we should bump up to TOML 1.0.0 here, and update our standards to that. |
|
@pradyunsg Would you be happy if I made a PR where I replace The alternatives I can think of are:
|
|
I haven't done any analysis of the various TOML libraries, but I'm personally OK with switching to |
Yup yup, and @pfmoore said what I wanted to say.
This, I've been working toward. |
|
@pradyunsg your last sentence confused me a bit, haha. So to be 100% certain before I put in the work for a PR:
Are you still working toward this and want to have it in |
|
Pradyun meant he was working toward having And yes, please do make a PR for the switch to |
inmantaci
added a commit
to inmanta/inmanta-core
that referenced
this issue
Jul 26, 2021
Bumps [pip](https://github.com/pypa/pip) from 21.1.3 to 21.2.1. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p> <blockquote> <h1>21.2.1 (2021-07-25)</h1> <h2>Process</h2> <ul> <li>The source distribution re-installation feature removal has been delayed to 21.3.</li> </ul> <h1>21.2 (2021-07-24)</h1> <h2>Process</h2> <ul> <li><code>pip freeze</code>, <code>pip list</code>, and <code>pip show</code> no longer normalize underscore (<code>_</code>) in distribution names to dash (<code>-</code>). This is a side effect of the migration to <code>importlib.metadata</code>, since the underscore-dash normalization behavior is non-standard and specific to setuptools. This should not affect other parts of pip (for example, when feeding the <code>pip freeze</code> result back into <code>pip install</code>) since pip internally performs standard PEP 503 normalization independently to setuptools.</li> </ul> <h2>Deprecations and Removals</h2> <ul> <li>Git version parsing is now done with regular expression to prepare for the pending upstream removal of non-PEP-440 version parsing logic. (<code>[#10117](pypa/pip#10117) <https://github.com/pypa/pip/issues/10117></code>_)</li> <li>Re-enable the "Value for ... does not match" location warnings to field a new round of feedback for the <code>distutils</code>-<code>sysconfig</code> transition. (<code>[#10151](pypa/pip#10151) <https://github.com/pypa/pip/issues/10151></code>_)</li> <li>Remove deprecated <code>--find-links</code> option in <code>pip freeze</code> (<code>[#9069](pypa/pip#9069) <https://github.com/pypa/pip/issues/9069></code>_)</li> </ul> <h2>Features</h2> <ul> <li> <p>New resolver: Loosen URL comparison logic when checking for direct URL reference equivalency. The logic includes the following notable characteristics:</p> <ul> <li>The authentication part of the URL is explicitly ignored.</li> <li>Most of the fragment part, including <code>egg=</code>, is explicitly ignored. Only <code>subdirectory=</code> and hash values (e.g. <code>sha256=</code>) are kept.</li> <li>The query part of the URL is parsed to allow ordering differences. (<code>[#10002](pypa/pip#10002) <https://github.com/pypa/pip/issues/10002></code>_)</li> </ul> </li> <li> <p>Support TOML v1.0.0 syntax in <code>pyproject.toml</code>. (<code>[#10034](pypa/pip#10034) <https://github.com/pypa/pip/issues/10034></code>_)</p> </li> <li> <p>Added a warning message for errors caused due to Long Paths being disabled on Windows. (<code>[#10045](pypa/pip#10045) <https://github.com/pypa/pip/issues/10045></code>_)</p> </li> <li> <p>Change the encoding of log file from default text encoding to UTF-8. (<code>[#10071](pypa/pip#10071) <https://github.com/pypa/pip/issues/10071></code>_)</p> </li> <li> <p>Log the resolved commit SHA when installing a package from a Git repository. (<code>[#10149](pypa/pip#10149) <https://github.com/pypa/pip/issues/10149></code>_)</p> </li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pypa/pip/commit/bd41229cdced10d2b7c304a1ef2d61baad3c7da0"><code>bd41229</code></a> Bump for release</li> <li><a href="https://github.com/pypa/pip/commit/765a4b40227653d22d30c85448ec343e542f41d7"><code>765a4b4</code></a> Bump for development</li> <li><a href="https://github.com/pypa/pip/commit/3d25c5327d0887271958999ac44709728b5c4f5a"><code>3d25c53</code></a> Bump for release</li> <li><a href="https://github.com/pypa/pip/commit/27b9a9c35c96e4032af7603ca03651d8f5917681"><code>27b9a9c</code></a> Update AUTHORS.txt</li> <li><a href="https://github.com/pypa/pip/commit/b9dbab277b7206fd9efc16cef793ab01ccfb32fb"><code>b9dbab2</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/10199">#10199</a> from uranusjr/remove-local-sdist-reinstall</li> <li><a href="https://github.com/pypa/pip/commit/33b7f0cd9e0fff6bd532941a530d7f8b3472eb55"><code>33b7f0c</code></a> Remove deprecated sdist reinstall feature for 21.2</li> <li><a href="https://github.com/pypa/pip/commit/a8b8d4d7fe214a0d4e78b0d98d0b97e471858fe5"><code>a8b8d4d</code></a> Document how to install provides_extras from local wheel file (<a href="https://github-redirect.dependabot.com/pypa/pip/issues/9698">#9698</a>)</li> <li><a href="https://github.com/pypa/pip/commit/239a30737277887fed9f609ae786a0fb80591be4"><code>239a307</code></a> Error handling upon <code>uninstall</code> invalid parameter (<a href="https://github-redirect.dependabot.com/pypa/pip/issues/10171">#10171</a>)</li> <li><a href="https://github.com/pypa/pip/commit/5e86264b5ac1c62d08f91a16a068647d8872e4e8"><code>5e86264</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/10189">#10189</a> from harupy/type-annotations-index-models</li> <li><a href="https://github.com/pypa/pip/commit/02b1855b45eb143b524f773eafb46d77406504d3"><code>02b1855</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/10188">#10188</a> from harupy/type-annotations-req</li> <li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/21.1.3...21.2.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
What's the problem this feature will solve?
Pip currently only supports TOML v0.5.0. Reading a pyproject.toml containing TOML v1.0.0-rc.1+ syntax will cause a failure, even if the unsupported syntax is in a section that pip doesn't care about (e.g.
[tool.*]).Some projects in the Python ecosystem are hesitant to support TOML v1.0.0 because pip does not. Meanwhile other projects are using Poetry and its v1.0.0-rc.1 compatible tomlkit for building packages, thus potentially creating pip incompatible builds without noticing an issue.
Describe the solution you'd like
Vendor a TOML v1.0.0 compatible TOML parser. I suggest my very own Tomli which, in addition to being v1.0.0 compatible, fixes many bugs present in the toml library, parses 2x as fast, and is much smaller with only 50% the amount of lines of code.
Alternative Solutions
Stick to a TOML v0.5.0 compatible parser. Perhaps mention in the docs that projects where pyproject.toml includes TOML v1.0.0-rc.1+ syntax are not supported.
The text was updated successfully, but these errors were encountered: