-
-
Notifications
You must be signed in to change notification settings - Fork 39.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Nix shell updates for develop
#18131
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The used Nixpkgs snapshot contains `tomlkit` version 0.7.0, which is affected by https://www.github.com/sdispater/tomlkit/issues/148; that bug is triggered by `pyproject.toml` from `jsonschema` >= 4.11.0, preventing the build of that module. Just adding `tomlkit = "*"` to the `[tool.poetry.dev-dependencies]` section of `nix/pyproject.toml` does not fix the `jsonschema` build, because `makeRemoveSpecialDependenciesHook` inside `poetry2nix` is not affected by `nix/pyproject.toml`. Add a Nixpkgs overlay which updates the `tomlkit` Python module globally, so that `poetry2nix` would also use the updated version internally.
The new `poetry2nix` version includes overrides which are required for recent versions of some Python packages (in particular, `jsonschema` and `dotty-dict`).
Update `pyproject.toml` to match `requirements*.txt`: - add `pyserial = "*"` - replace `qmk-dotty-dict = "*"` with `dotty-dict = "*"` (qmk#18117, also required for compatibility with `qmk` 1.1.1, where this replacement had already been performed) Add build dependencies of various Python modules to `pyproject.toml`: - `hatchling`, `hatch-vcs`, `hatch-fancy-pypi-readme` (required by `jsonschema` >= 4.11.0) - `pytest` (a newer version is required to solve the dependency conflict with the `hatchling` module due to the upper bound on `pluggy`) - `flit-core` (a more recent version is required to build `tomli`) - `poetry-core` (required for `dotty-dict` >= 1.3.1, and the version from Nixpkgs does not build on Darwin due to NixOS/nix#4758) Update `poetry.lock` to use the most recent versions of Python modules. The complete list of Python module updates as listed in `poetry.lock` (note that other modules might be present in the Python environment, e.g., if they come from Nixpkgs): - `atomicwrites`: none -> 1.4.1 (but this module is not actually used, because the corresponding dependency of `pytest` is win32-only) - `attrs`: 21.4.0 -> 22.1.0 - `colorama`: 0.4.4 -> 0.4.5 - `coverage`: 6.4 -> none - `dotty-dict`: none -> 1.3.1 (used instead of `qmk-dotty-dict`) - `editables`: none -> 0.3 - `flake8`: 4.0.1 -> 5.0.4 - `flake8-polyfill`: 1.0.2 -> none - `flit-core`: none -> 3.7.1 - `hatch-fancy-pypi-readme`: none -> 22.3.0 - `hatch-vcs`: none -> 0.2.0 - `hatchling`: none -> 1.8.0 - `hjson`: 3.0.2 -> 3.1.0 - `importlib-resources`: 5.7.1 -> 5.9.0 - `iniconfig`: none -> 1.1.1 - `jsonschema`: 4.5.1 -> 4.14.0 - `mccabe`: 0.6.1 -> 0.7.0 - `nose2`: 0.11.0 -> 0.12.0 - `packaging`: none -> 21.3 - `pathspec`: none -> 0.9.0 - `pep8-naming`: 0.12.1 -> 0.13.2 - `pillow`: 9.1.1 -> 9.2.0 - `pkgutil-resolve-name`: none -> 1.3.10 - `pluggy`: none -> 1.0.0 - `poetry-core`: none -> 1.0.8 - `py`: none -> 1.11.0 - `pycodestyle`: 2.8.0 -> 2.9.1 - `pyflakes`: 2.4.0 -> 2.5.0 - `pygments`: 2.12.0 -> 2.13.0 - `pyparsing`: none -> 3.0.9 - `pyserial`: none -> 3.5 - `pytest`: none -> 7.1.2 - `qmk`: 1.1.0 -> 1.1.1 - `qmk-dotty-dict`: 1.3.0.post1 -> none (replaced by `dotty-dict`) - `setuptools-scm`: none -> 7.0.5 - `tomli`: none -> 2.0.1 - `typing-extensions`: none -> 4.3.0 - `zipp`: 3.8.0 -> 3.8.1
14 tasks
evils
approved these changes
Aug 22, 2022
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
works for me, thanks :)
purcell
approved these changes
Aug 22, 2022
zvecr
approved these changes
Aug 23, 2022
nolanseaton
pushed a commit
to nolanseaton/qmk_firmware
that referenced
this pull request
Jan 23, 2023
* shell.nix: Update `tomlkit` to 0.11.4 using a Nixpkgs overlay The used Nixpkgs snapshot contains `tomlkit` version 0.7.0, which is affected by https://www.github.com/sdispater/tomlkit/issues/148; that bug is triggered by `pyproject.toml` from `jsonschema` >= 4.11.0, preventing the build of that module. Just adding `tomlkit = "*"` to the `[tool.poetry.dev-dependencies]` section of `nix/pyproject.toml` does not fix the `jsonschema` build, because `makeRemoveSpecialDependenciesHook` inside `poetry2nix` is not affected by `nix/pyproject.toml`. Add a Nixpkgs overlay which updates the `tomlkit` Python module globally, so that `poetry2nix` would also use the updated version internally. * shell.nix: Bump `poetry2nix` to the most recent version The new `poetry2nix` version includes overrides which are required for recent versions of some Python packages (in particular, `jsonschema` and `dotty-dict`). * shell.nix: Bump QMK CLI to 1.1.1; update other Python deps Update `pyproject.toml` to match `requirements*.txt`: - add `pyserial = "*"` - replace `qmk-dotty-dict = "*"` with `dotty-dict = "*"` (qmk#18117, also required for compatibility with `qmk` 1.1.1, where this replacement had already been performed) Add build dependencies of various Python modules to `pyproject.toml`: - `hatchling`, `hatch-vcs`, `hatch-fancy-pypi-readme` (required by `jsonschema` >= 4.11.0) - `pytest` (a newer version is required to solve the dependency conflict with the `hatchling` module due to the upper bound on `pluggy`) - `flit-core` (a more recent version is required to build `tomli`) - `poetry-core` (required for `dotty-dict` >= 1.3.1, and the version from Nixpkgs does not build on Darwin due to NixOS/nix#4758) Update `poetry.lock` to use the most recent versions of Python modules. The complete list of Python module updates as listed in `poetry.lock` (note that other modules might be present in the Python environment, e.g., if they come from Nixpkgs): - `atomicwrites`: none -> 1.4.1 (but this module is not actually used, because the corresponding dependency of `pytest` is win32-only) - `attrs`: 21.4.0 -> 22.1.0 - `colorama`: 0.4.4 -> 0.4.5 - `coverage`: 6.4 -> none - `dotty-dict`: none -> 1.3.1 (used instead of `qmk-dotty-dict`) - `editables`: none -> 0.3 - `flake8`: 4.0.1 -> 5.0.4 - `flake8-polyfill`: 1.0.2 -> none - `flit-core`: none -> 3.7.1 - `hatch-fancy-pypi-readme`: none -> 22.3.0 - `hatch-vcs`: none -> 0.2.0 - `hatchling`: none -> 1.8.0 - `hjson`: 3.0.2 -> 3.1.0 - `importlib-resources`: 5.7.1 -> 5.9.0 - `iniconfig`: none -> 1.1.1 - `jsonschema`: 4.5.1 -> 4.14.0 - `mccabe`: 0.6.1 -> 0.7.0 - `nose2`: 0.11.0 -> 0.12.0 - `packaging`: none -> 21.3 - `pathspec`: none -> 0.9.0 - `pep8-naming`: 0.12.1 -> 0.13.2 - `pillow`: 9.1.1 -> 9.2.0 - `pkgutil-resolve-name`: none -> 1.3.10 - `pluggy`: none -> 1.0.0 - `poetry-core`: none -> 1.0.8 - `py`: none -> 1.11.0 - `pycodestyle`: 2.8.0 -> 2.9.1 - `pyflakes`: 2.4.0 -> 2.5.0 - `pygments`: 2.12.0 -> 2.13.0 - `pyparsing`: none -> 3.0.9 - `pyserial`: none -> 3.5 - `pytest`: none -> 7.1.2 - `qmk`: 1.1.0 -> 1.1.1 - `qmk-dotty-dict`: 1.3.0.post1 -> none (replaced by `dotty-dict`) - `setuptools-scm`: none -> 7.0.5 - `tomli`: none -> 2.0.1 - `typing-extensions`: none -> 4.3.0 - `zipp`: 3.8.0 -> 3.8.1
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Update the nix-shell environment in the
develop
branch to use QMK CLI 1.1.1 and newest versions of other required Python modules (includingpyserial
, which is a recent addition to the Python dependencies).The update is required to make the current state of the
develop
branch buildable usingnix-shell
(with the existing versionqmk
fails to do anything due to the missingpyserial
module).@AstridYu: This somewhat interferes with your change from #16602, because any externally supplied
pkgs
must also contain at least version 0.8.0 ofpython3.pkgs.tomlkit
(but I did not specifically test that version, just noticed that python-poetry/tomlkit#148 should be fixed there). It could be possible to add anixpkgsSource
argument to override justsources.nixpkgs
(then theshell.nix
code could import that source with the overlay which upgradestomlkit
), but I'm not sure how this would align with your use case; and if you actually intend to supply a recent enough Nixpkgs inpkgs
, things will probably just work without that overlay.Some seemingly useless changes in
util/nix/poetry.lock
are caused by python-poetry/poetry#6153 (probably the order of some entries depended on the order of results from the PyPI API, which changed at some point); looks like the fix won't be backported to the 1.1 branch of Poetry, so we are stuck with that issue for some time (at least the order does not seem to change between further runs for now).Tested on ubuntu-latest, macos-11, macos-12 (failures in that run are for the
develop
branch ofqmk/qmk_firmware
without this change).Python module version updates:
atomicwrites
: none → 1.4.1 (but this module is not actually used, because the corresponding dependency ofpytest
is win32-only)attrs
: 21.4.0 → 22.1.0colorama
: 0.4.4 → 0.4.5coverage
: 6.4 → nonedotty-dict
: none → 1.3.1 (used instead ofqmk-dotty-dict
)editables
: none → 0.3flake8
: 4.0.1 → 5.0.4flake8-polyfill
: 1.0.2 → noneflit-core
: none → 3.7.1hatch-fancy-pypi-readme
: none → 22.3.0hatch-vcs
: none → 0.2.0hatchling
: none → 1.8.0hjson
: 3.0.2 → 3.1.0importlib-resources
: 5.7.1 → 5.9.0iniconfig
: none → 1.1.1jsonschema
: 4.5.1 → 4.14.0mccabe
: 0.6.1 → 0.7.0nose2
: 0.11.0 → 0.12.0packaging
: none → 21.3pathspec
: none → 0.9.0pep8-naming
: 0.12.1 → 0.13.2pillow
: 9.1.1 → 9.2.0pkgutil-resolve-name
: none → 1.3.10pluggy
: none → 1.0.0poetry-core
: none → 1.0.8py
: none → 1.11.0pycodestyle
: 2.8.0 → 2.9.1pyflakes
: 2.4.0 → 2.5.0pygments
: 2.12.0 → 2.13.0pyparsing
: none → 3.0.9pyserial
: none → 3.5pytest
: none → 7.1.2qmk
: 1.1.0 → 1.1.1qmk-dotty-dict
: 1.3.0.post1 → none (replaced bydotty-dict
)setuptools-scm
: none → 7.0.5tomli
: none → 2.0.1tomlkit
: 0.7.0 → 0.11.4 (used only during build, not included into the resulting environment)typing-extensions
: none → 4.3.0zipp
: 3.8.0 → 3.8.1Cc: @purcell @AstridYu @andresilva @evils
Types of Changes
Checklist