Python DevShells (for pip)

[phaer]

Hello,

This is a PR which implements devshells with support for console scripts and richer metadata then the approach in #912. Its currently only implemented for the pip builder, but the idea should be portable to PDM as well. Planning to look into this soon.

• users need to declare an the root package and/or dependencies of a given python project as editable. Consider a devshell for a package named my-tool:

{
  pip.editables.charset-normalizer = "/home/my-user/src/charset-normalizer";
  pip.editables.click = null;
  pip.editables.my-tool = null;
}

The first expression would link the editable charset-normalizer from the given absolute path, to .dream2nix/editables/charset-normalizer and install it.
The second line would default to copying click from the nix store to .dream2nix/editables/click and install it.
As the first one matches the root package, we would default to link our project root to .dream2nix/editables/my-tool and install it.

• We also implicitly adds dependencies which are "direct installs" in pips report to editables, e.g. "click @ git+https://github.com/pallets/click.git@main" for convenience.

• "Installing" here only affects the devshell, we don't touch the results of nix build, but instead run a python script (editable.py) in a shellHook to get the right PATH and PYTHONPATH for the shell setup and install wrapper scripts for console_scripts of our dependencies.

• We do change public.pyEnv by moving its upstream sitecustomize.py to _sitecustomize.py. It was recently re-introduced into nixpkgs as its removal caused a few pkgs to break, but it did work for the overwhelming majority without issues. Moving it, deactivates it by default but it can easily be re-included if needed. We do so in the devshell just in case in https://github.com/nix-community/dream2nix/compare/main...phaer:dream2nix:devshells?expand=1#diff-4f29741e77180e7c500ebe90ecaa8af18d8aede267e142bf2f83888073079c9eR213

• This PR includes commits from fetchPipMetadata: make mitmproxy optional #932. I can drop them if unwanted.

• The history includes my first attempt as a shell script. Should I squash this?

Give it a try

Just add some pip.editables as above to your package and include
${your-pkg.config.pip.editablesShellHook} in your shellHook or see
examples/packages/languages/python-local-development/default.nix and the devShell there.

[phaer]

Regarding pdm support: @DavHau mentioned that he might want to re-do the groups feature of the pdm module. For the editables we need a) a python environment and b) an attrset with one drv per package.

Another area which might need a bit more work, is garbage collection in .dream2nix. We probably don't want to drop .dream2nix/editables for each shell as it might contain actual worktrees. So we need to diff the declared editables and the existing ones.

[phaer]

maybe this can be removed? Looks like a left-over as well?