-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Faster build and installation of packages #6205
Conversation
455e638
to
96c7640
Compare
63ce58f
to
ca99889
Compare
62fb172
to
fb2e022
Compare
6b23b02
to
cdeb660
Compare
Deploy preview for website ready! ✅ Preview Built with commit 838f0ee. |
cdeb660
to
c8f4468
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some first (mostly superficial) feedback.
It seems the commits are not cut optimally: The commit "Add a wheel builder" already requires WheelInstaller
which is added only later in "Add a wheel installer". (At least if we keep the commits separated when merging it may be worth to fix that.)
c8f4468
to
2e97898
Compare
Good point. It gave similar verbose output as pip but of course no note that this is probably not a problem with poetry. I added such a disclaimer. PTAL if the message can be improved. |
3041a2b
to
bd35898
Compare
f59fbc0
to
7f159dc
Compare
7f159dc
to
82d0bde
Compare
Co-authored-by: Randy Döring <30527984+radoering@users.noreply.github.com>
Co-authored-by: Randy Döring <30527984+radoering@users.noreply.github.com>
…t behavior of old installer)
82d0bde
to
838f0ee
Compare
This PR contains the following updates: | Package | Update | Change | |---|---|---| | [poetry](https://python-poetry.org/) ([source](https://github.com/python-poetry/poetry), [changelog](https://python-poetry.org/history/)) | minor | `1.3.2` -> `1.4.0` | --- ### Release Notes <details> <summary>python-poetry/poetry</summary> ### [`v1.4.0`](https://github.com/python-poetry/poetry/blob/HEAD/CHANGELOG.md#​140---2023-02-27) [Compare Source](python-poetry/poetry@1.3.2...1.4.0) ##### Added - **Add a modern installer (`installer.modern-installation`) for faster installation of packages and independence from pip** ([#​6205](python-poetry/poetry#6205)). - Add support for `Private ::` trove classifiers ([#​7271](python-poetry/poetry#7271)). - Add the version of poetry in the `@generated` comment at the beginning of the lock file ([#​7339](python-poetry/poetry#7339)). - Add support for `virtualenvs.prefer-active-python` when running `poetry new` and `poetry init` ([#​7100](python-poetry/poetry#7100)). ##### Changed - **Deprecate the old installer, i.e. setting `experimental.new-installer` to `false`** ([#​7358](python-poetry/poetry#7358)). - Remove unused `platform` field from cached package info and bump the cache version ([#​7304](python-poetry/poetry#7304)). - Extra dependencies of the root project are now sorted in the lock file ([#​7375](python-poetry/poetry#7375)). - Remove upper boundary for `importlib-metadata` dependency ([#​7434](python-poetry/poetry#7434)). - Validate path dependencies during use instead of during construction ([#​6844](python-poetry/poetry#6844)). - Remove the deprecated `repository` modules ([#​7468](python-poetry/poetry#7468)). ##### Fixed - Fix an issue where an unconditional dependency of an extra was not installed in specific environments ([#​7175](python-poetry/poetry#7175)). - Fix an issue where a pre-release of a dependency was chosen even if a stable release fulfilled the constraint ([#​7225](python-poetry/poetry#7225), [#​7236](python-poetry/poetry#7236)). - Fix an issue where HTTP redirects were not handled correctly during publishing ([#​7160](python-poetry/poetry#7160)). - Fix an issue where `poetry check` did not handle the `-C, --directory` option correctly ([#​7241](python-poetry/poetry#7241)). - Fix an issue where the subdirectory information of a git dependency was not written to the lock file ([#​7367](python-poetry/poetry#7367)). - Fix an issue where the wrong Python version was selected when creating an virtual environment ([#​7221](python-poetry/poetry#7221)). - Fix an issue where packages that should be kept were uninstalled when calling `poetry install --sync` ([#​7389](python-poetry/poetry#7389)). - Fix an issue where an incorrect value was set for `sys.argv[0]` when running installed scripts ([#​6737](python-poetry/poetry#6737)). - Fix an issue where hashes in `direct_url.json` files were not written according to the specification ([#​7475](python-poetry/poetry#7475)). - Fix an issue where poetry commands failed due to special characters in the path of the project or virtual environment ([#​7471](python-poetry/poetry#7471)). - Fix an issue where poetry crashed with a `JSONDecodeError` when running a Python script that produced certain warnings ([#​6665](python-poetry/poetry#6665)). ##### Docs - Add advice on how to maintain a poetry plugin ([#​6977](python-poetry/poetry#6977)). - Update tox examples to comply with the latest tox release ([#​7341](python-poetry/poetry#7341)). - Mention that the `poetry export` can export `constraints.txt` files ([#​7383](python-poetry/poetry#7383)). - Add clarifications for moving configuration files ([#​6864](python-poetry/poetry#6864)). - Mention the different types of exact version specifications ([#​7503](python-poetry/poetry#7503)). ##### poetry-core ([`1.5.1`](https://github.com/python-poetry/poetry-core/releases/tag/1.5.1)) - Improve marker handling ([#​528](python-poetry/poetry-core#528), [#​534](python-poetry/poetry-core#534), [#​530](python-poetry/poetry-core#530), [#​546](python-poetry/poetry-core#546), [#​547](python-poetry/poetry-core#547)). - Validate whether dependencies referenced in `extras` are defined in the main dependency group ([#​542](python-poetry/poetry-core#542)). - Poetry no longer generates a `setup.py` file in sdists by default ([#​318](python-poetry/poetry-core#318)). - Fix an issue where trailing newlines were allowed in `tool.poetry.description` ([#​505](python-poetry/poetry-core#505)). - Fix an issue where the name of the data folder in wheels was not normalized ([#​532](python-poetry/poetry-core#532)). - Fix an issue where the order of entries in the RECORD file was not deterministic ([#​545](python-poetry/poetry-core#545)). - Fix an issue where zero padding was not correctly handled in version comparisons ([#​540](python-poetry/poetry-core#540)). - Fix an issue where sdist builds did not support multiple READMEs ([#​486](python-poetry/poetry-core#486)). ##### poetry-plugin-export ([`^1.3.0`](https://github.com/python-poetry/poetry-plugin-export/releases/tag/1.3.0)) - Fix an issue where the export failed if there was a circular dependency on the root package ([#​118](python-poetry/poetry-plugin-export#118)). </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4xNTIuNSIsInVwZGF0ZWRJblZlciI6IjM0LjE1Mi41In0=--> Reviewed-on: https://git.walbeck.it/walbeck-it/docker-python-poetry/pulls/655 Co-authored-by: renovate-bot <bot@walbeck.it> Co-committed-by: renovate-bot <bot@walbeck.it>
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
This PR (which is an updated version of #4213) is the first step towards getting rid of subprocess calls to
pip
for package installations and removals.The wheel building is handled by https://github.com/pypa/build and the wheel installation is handled by https://github.com/pypa/installer (with modifications to be more lenient).
Note that the wheel building is not parallelized due to race conditions that I am yet to track down (it seems like the issue comes from
build
when calling the backend hooks). This is not a blocker though.Overall this speeds up quite a bit the installation times, especially once wheels are cached.
An example with the installation of 115 packages (warm cache and artifacts already downloaded):
Without the new builder/installer:
226.24s user 15.26s system 488% cpu 49.413 total
With the new builder/installer:
11.52s user 3.08s system 117% cpu 12.452 total
Note that this can be deactivated by setting
experimental.installer.modern-installation
.Pull Request Check List
Resolves: python-poetry/roadmap#1