pip option to install PEP518 build dependencies to current environment

[matham]

What's the problem this feature will solve?

This is to address three problems, one of which has been raised here before.

1. You have a mixed Python/Cython project that you want to work on - this project lists its build dependencies (e.g. Cython) in pyproject.toml. You clone and install it with pip install -e ..

   Now, you make some changes to the project and want to recompile the cython files. So you try python setup.py build_ext --inplace. But that fails with e.g. ModuleNotFoundError: No module named 'Cython' because Cython was not installed to your current environment - it was only installed temporarily to the isolated environment when the project was initially installed.

   Other than manually installing the dependencies listed in pyproject.toml, which would be a little annoying, what can you do?

2. Say you have project foo with a pyproject.toml that lists bar and other packages as build time dependencies. So now you make some changes to bar and install foo to see if foo still builds/works with the new bar changes. Unfortunately, because of build isolation, the last published bar package from pypi is used when you install foo.

   What you'd have liked instead is for you to be able to make pip use the newly generated wheel for bar when installing to the isolated build env.

   Again, other than turning off build isolation and manually installing all the pyproject.toml deps and bar, I don't see a way around this.

3. Optional dependencies. This has been brought up before here so I'll quickly summarize it. They wanted a way to use pyproject.toml but also use whatever packages is installed in the current environment. The additional packages in the current environment would determine which optional dependencies are supported by the wheel being built.

Describe the solution you'd like

I'd like to request a new pip option where in addition to the --no-build-isolation option, we also have something like --install-build-deps. Also, maybe, when making an editable install it should default to using this option. What it does is it installs the build-time pyproject.toml listed dependencies to the current environment, rather than installing it into a isolated environment.

An alternate solution that was presented for #3 was a new --system-site-packages flag to be added so that whatever you have installed in the current environment is visible in the isolated build environment. While it does solve #3 and possibly #2, I think --install-build-deps is a better solution overall and resolves all three issues.

Specifically it'll allow you to install all the build time dependencies to the current environment so that you can later manually install/build the project as you like with e.g. --no-build-isolation.

[uranusjr]

Personally I tend to think this whole functionality belongs in a separate (project management) tool, not pip. The fact that pip does not have an equivalent to python setup.py build_ext is a tell that pip never intends to fill that need.

[uranusjr]

FYI I hacked together a tool that might help: https://github.com/uranusjr/setl

[matham]

Thanks, but alas I'm using python 3.7 and the project I am maintaining still supports 3.6, but setl seems to require 3.8+ :(

[uranusjr]

The tool needs to be installed on 3.7 (I lowered the requirements due to toolchain support), but can be used to develop a project against with older versions by using the --python option. I wrote some documentation around this. https://setl.readthedocs.io/en/latest/cmds.html#the-global-python-option

[pradyunsg]

Closing this out since I agree that this is not a role that pip needs to file.