Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shell.nix: Fix compatibility with Nix >= 2.10.0 on x86_64-darwin #17673

Merged
merged 1 commit into from
Jul 18, 2022

Conversation

sigprof
Copy link
Contributor

@sigprof sigprof commented Jul 13, 2022

Description

The poetry package from the used Nixpkgs snapshot is no longer buildable when using the official binary releases of Nix >= 2.10.0 on x86_64-darwin due to a regex compatibility issue (NixOS/nix#4758). The failure looks like:

error: invalid regular expression '^.*?(egg|tar|tar.bz2|tar.gz|tar.lz|tar.lzma|tar.xz|tbz|tgz|tlz|txz|whl|zip)'
       at /nix/store/gm4rp1786ligd81v2qz420nkrwy0r50k-nixpkgs-src/pkgs/development/tools/poetry2nix/poetry2nix/mk-poetry-dep.nix:42:42:
           41|           matchesVersion = fname: builtins.match ("^.*" + builtins.replaceStrings [ "." ] [ "\\." ] version + ".*$") fname != null;
           42|           hasSupportedExtension = fname: builtins.match supportedRegex fname != null;
             |                                          ^
           43|           isCompatibleEgg = fname: ! lib.strings.hasSuffix ".egg" fname || lib.strings.hasSuffix "py${python.pythonVersion}.egg" fname;

In addition, all poetry version earlier than 1.1.14 became effectively non-functional after a breaking change of the PyPI JSON API (python-poetry/poetry#5973) — they generate broken poetry.lock files without any hash values, therefore the old version of poetry that was added to the Nix shell environment became useless anyway.

Updating the poetry package to the most recent stable release (1.1.14 at the moment) without bumping nixpkgs is not trivial:

  • Just adding poetry = ">=1.1.14" to pyproject.toml does not work due to dependency version conflicts with other modules.
  • Although poetry2nix exports a poetry package, it makes that package available only as a flake output (it could be possible to import overlay.nix from there directly, but that is not documented; this would also require changing the arguments added in Allow overriding Niv's sources in shell.nix #16602poetry2nix ? pkgs.callPackage (import sources.poetry2nix) { } would no longer be sufficient).

Given those facts, looks like the only way to fix the problem without major changes is to remove the poetry package from the Nix shell environment. This should not cause any problems for QMK users, because the poetry tool is not needed to compile the QMK firmware — it may be needed only if someone wants to update the Nix shell environment itself, and in that case something like nix run github:NixOS/nixpkgs/nixos-unstable#poetry could be used for now.

I don't actually have any Apple hardware to test this on, but the failure could be observed on the macos-* machines provided by GitHub (e.g., https://github.com/sigprof/qmk-nix-support/actions/runs/2666127722 — failed tests are for the master code without this change). The failure started to happen after the Nix 2.10.0 release a couple of days ago (cachix/install-nix-action installs the latest stable Nix release by default); Nix 2.10.1 did not change anything.

Cc: @purcell @AstridYu @andresilva

Types of Changes

  • Core
  • Bugfix
  • New feature
  • Enhancement/optimization
  • Keyboard (addition or update)
  • Keymap/layout/userspace (addition or update)
  • Documentation

Checklist

  • My code follows the code style of this project: C, Python
  • I have read the PR Checklist document and have made the appropriate changes.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • I have tested the changes and verified that they work and don't break anything (as well as I can manage).

The `poetry` package from the used Nixpkgs snapshot triggers the regex
compatibility issue in Nix >= 2.10.0 binaries for `x86_64-darwin`:

    https://www.github.com/NixOS/nix/issues/4758

Remove the `poetry` package from the Nix shell environment for now
(it is not really required to compile QMK, only to develop the Nix shell
environment itself).

In addition, all `poetry` version earlier than 1.1.14 became effectively
non-functional after a breaking change of the PyPI JSON API:

    https://www.github.com/python-poetry/poetry/pull/5973

Updating the `poetry` package is not trivial (just adding it it to
`pyproject.toml` does not work due to dependency version conflicts with
other modules), therefore removing it seems to be the easiest solution
to restore compatibility with new Nix versions while not creating any
major inconvenience for QMK users.
Copy link
Contributor

@purcell purcell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me, thanks

@sigprof sigprof merged commit 627e35b into qmk:master Jul 18, 2022
smocky pushed a commit to smocky/qmk_firmware that referenced this pull request Jul 22, 2022
The `poetry` package from the used Nixpkgs snapshot triggers the regex
compatibility issue in Nix >= 2.10.0 binaries for `x86_64-darwin`:

    https://www.github.com/NixOS/nix/issues/4758

Remove the `poetry` package from the Nix shell environment for now
(it is not really required to compile QMK, only to develop the Nix shell
environment itself).

In addition, all `poetry` version earlier than 1.1.14 became effectively
non-functional after a breaking change of the PyPI JSON API:

    https://www.github.com/python-poetry/poetry/pull/5973

Updating the `poetry` package is not trivial (just adding it it to
`pyproject.toml` does not work due to dependency version conflicts with
other modules), therefore removing it seems to be the easiest solution
to restore compatibility with new Nix versions while not creating any
major inconvenience for QMK users.
Haruki pushed a commit to Haruki/qmk_firmware that referenced this pull request Jul 23, 2022
The `poetry` package from the used Nixpkgs snapshot triggers the regex
compatibility issue in Nix >= 2.10.0 binaries for `x86_64-darwin`:

    https://www.github.com/NixOS/nix/issues/4758

Remove the `poetry` package from the Nix shell environment for now
(it is not really required to compile QMK, only to develop the Nix shell
environment itself).

In addition, all `poetry` version earlier than 1.1.14 became effectively
non-functional after a breaking change of the PyPI JSON API:

    https://www.github.com/python-poetry/poetry/pull/5973

Updating the `poetry` package is not trivial (just adding it it to
`pyproject.toml` does not work due to dependency version conflicts with
other modules), therefore removing it seems to be the easiest solution
to restore compatibility with new Nix versions while not creating any
major inconvenience for QMK users.
schattenbrot pushed a commit to schattenbrot/qmk_firmware that referenced this pull request Aug 2, 2022
The `poetry` package from the used Nixpkgs snapshot triggers the regex
compatibility issue in Nix >= 2.10.0 binaries for `x86_64-darwin`:

    https://www.github.com/NixOS/nix/issues/4758

Remove the `poetry` package from the Nix shell environment for now
(it is not really required to compile QMK, only to develop the Nix shell
environment itself).

In addition, all `poetry` version earlier than 1.1.14 became effectively
non-functional after a breaking change of the PyPI JSON API:

    https://www.github.com/python-poetry/poetry/pull/5973

Updating the `poetry` package is not trivial (just adding it it to
`pyproject.toml` does not work due to dependency version conflicts with
other modules), therefore removing it seems to be the easiest solution
to restore compatibility with new Nix versions while not creating any
major inconvenience for QMK users.
imhoffman pushed a commit to imhoffman/qmk_firmware that referenced this pull request Aug 20, 2022
The `poetry` package from the used Nixpkgs snapshot triggers the regex
compatibility issue in Nix >= 2.10.0 binaries for `x86_64-darwin`:

    https://www.github.com/NixOS/nix/issues/4758

Remove the `poetry` package from the Nix shell environment for now
(it is not really required to compile QMK, only to develop the Nix shell
environment itself).

In addition, all `poetry` version earlier than 1.1.14 became effectively
non-functional after a breaking change of the PyPI JSON API:

    https://www.github.com/python-poetry/poetry/pull/5973

Updating the `poetry` package is not trivial (just adding it it to
`pyproject.toml` does not work due to dependency version conflicts with
other modules), therefore removing it seems to be the easiest solution
to restore compatibility with new Nix versions while not creating any
major inconvenience for QMK users.
nolanseaton pushed a commit to nolanseaton/qmk_firmware that referenced this pull request Jan 23, 2023
The `poetry` package from the used Nixpkgs snapshot triggers the regex
compatibility issue in Nix >= 2.10.0 binaries for `x86_64-darwin`:

    https://www.github.com/NixOS/nix/issues/4758

Remove the `poetry` package from the Nix shell environment for now
(it is not really required to compile QMK, only to develop the Nix shell
environment itself).

In addition, all `poetry` version earlier than 1.1.14 became effectively
non-functional after a breaking change of the PyPI JSON API:

    https://www.github.com/python-poetry/poetry/pull/5973

Updating the `poetry` package is not trivial (just adding it it to
`pyproject.toml` does not work due to dependency version conflicts with
other modules), therefore removing it seems to be the easiest solution
to restore compatibility with new Nix versions while not creating any
major inconvenience for QMK users.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants