pip install --global-option=... git+https... does not take options into account

[zerothi]

Environment

• pip version:
  19.0.3
• Python version:
  3.7.3
• OS:
  Debian (latest)

Description

When issuing:

pip install --global-option="build" --global-option="--fcompiler=nag" git+https://github.com/zerothi/sisl.git

I would expect the installation to take into account the build options such that the install command would look like (example): python setup.py build --fcompiler=nag install.

Remarkably this does not occur when installing from the cloned directory:

git clone git+https://github.com/zerothi/sisl.git
cd sisl
pip install --global-option="build" --global-option="--fcompiler=nag" .

then everything works as expected.

Expected behavior

I would expect a git+https install to work exactly like the local directory installation.

How to Reproduce

Since it is easy to check whether it works simply pass a compiler you don't have access to on your machine (I don't have the nag compiler). Then the build should stop immediately with an error numpy.distutils.fcompiler.CompilerNotFound.

1. Non-bug (correctly errors out due to intercepted option --fcompiler:

git clone git+https://github.com/zerothi/sisl.git
cd sisl
pip install --global-option="build" --global-option="--fcompiler=nag" .

2. Bug:

pip install --global-option="build" --global-option="--fcompiler=nag" git+https://github.com/zerothi/sisl.git

This will effectively execute:

pip install git+https://github.com/zerothi/sisl.git

which is not intended... :(

Output

In the first one you should find a numpy.distutils.fcompiler.CompilerNotFound error, the second one should not execute any error.

[cjerdonek]

It would help if you could include pip's log output when running the command that causes the problem -- preferably with verbose logging.

[zerothi]

You can find the log here.

Command issued is:

pip -v -v -v --log sisl.log install --global-option="build" --global-option="--fcompiler=nag" git+https://github.com/zerothi/sisl.git

which should fail!
But doesn't. I.e. the --fcompiler=nag flag is not passed to the setup.py build script.

[cjerdonek]

How about the log file for the success case (installing from the directory) for comparison?

[cjerdonek]

Also, here are two issues that seem the same or related: #5518 ("pip 10.0.1 ignores --install-options installing from git+git://") and #5576 ("pip does not respect --install-scripts when installing a library from git").

[cjerdonek]

Okay, I see what's happening now.

When --global-option is passed, pip disallows binaries for that requirement:

pip/src/pip/_internal/cli/cmdoptions.py

Lines 72 to 79 in 78744e8

	names = ["build_options", "global_options", "install_options"]
	if any(map(getname, names)):
	control = options.format_control
	control.disallow_binaries()
	warnings.warn(
	'Disabling all use of wheels due to the use of --build-options '
	'/ --global-options / --install-options.', stacklevel=2,
	)

However, when it gets to the point where pip decides whether to build a wheel or fall back to using setup.py, it looks at whether it's a VCS checkout before checking whether binaries were disallowed:

pip/src/pip/_internal/wheel.py

Lines 766 to 776 in 78744e8

	if req.link and not req.link.is_artifact:
	# VCS checkout. Build wheel just for this run.
	return True
	if "binary" not in format_control.get_allowed_formats(
	canonicalize_name(req.name)):
	logger.info(
	"Skipping bdist_wheel for %s, due to binaries "
	"being disabled for it.", req.name,
	)
	return None

And when building a wheel, the global options aren't used, so the build succeeds, and it never gets to trying the options with setup.py.

[cjerdonek]

Also, I believe the "Disabling all use of wheels" warning isn't making it to the log file because it's only being output to the console via stderr. So it's better to paste the console output into the issue rather than the log file contents.

[zerothi]

I agree that this probably is a duplicate of #5518, the only difference between #5518 and this is that here wheel is not used.

Feel free to close if you think it is superfluous to have this lying around :)

[cjerdonek]

@zerothi Okay, thanks. I'll close this as a duplicate of #5518 to respect the one that was filed earlier.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.