Inconsistent behaviour of installed wrappers in venv

[lewis6991]

Environment

• pip version: 20.0.2
• Python version: 3.7.6
• OS: rhe7

Description

I have no idea if this is a pip, setuptools or a venv issue.

The script wrappers generated installed in virtual environments (from venv) don't appear to install consistently and depend upon how the cache is built.

Sometimes the wrappers are built with easy_install and sometimes not. The problem for me is that the easy_install wrappers don't support virtual environments in long named paths (due always using a shebang). Whereas the wrappers not built with easy_install will use an exec if the path of the virutalenv is too long.

The problem appears to be with how the cache is built. If the package was installed globally with a clean cache, then the non-easy_install wrappers are built in subsequent venvs.

Expected behavior

Never to use easy_install, or at least build consistently regardless of the cache state.

How to Reproduce

Run this:

export XDG_CACHE_HOME=$(pwd)/myvenv.cache

clean() {
    rm -rf $XDG_CACHE_HOME
    rm -rf myvenv
}

install_venv() {
   python3 -m venv myvenv
   myvenv/bin/pip3 install --upgrade pip
   myvenv/bin/pip3 install lit
   head myvenv/bin/lit
}

clean
install_venv

myvenv/bin/lit has a wrapper that appears to be built from easy_install

Now run:

clean
pip3 install lit # Add lit to the cache from a global install
install_venv

Now the lit wrapper doesn't use easy_install.

Output

% python3 -m venv myvenv
% myvenv/bin/pip3 install --upgrade pip
Collecting pip
  Downloading https://files.pythonhosted.org/packages/54/0c/d01aa759fdc501a58f431eb594a17495f15b88da142ce14b5845662c13f3/pip-20.0.2-py2.py3-none-any.whl (1.4MB)
     |████████████████████████████████| 1.4MB 10.6MB/s
Installing collected packages: pip
  Found existing installation: pip 19.2.3
    Uninstalling pip-19.2.3:
      Successfully uninstalled pip-19.2.3
Successfully installed pip-20.0.2

% myvenv/bin/pip3 install lit
Collecting lit==0.9.0
  Downloading lit-0.9.0.tar.gz (87 kB)
     |████████████████████████████████| 87 kB 2.8 MB/s
Installing collected packages: lit
    Running setup.py install for lit ... done
Successfully installed lit-0.9.0
Found existing installation: lit 0.9.0
Uninstalling lit-0.9.0:
  Successfully uninstalled lit-0.9.0
Collecting lit==0.9.0
  Using cached lit-0.9.0.tar.gz (87 kB)
Installing collected packages: lit
    Running setup.py install for lit ... done
Successfully installed lit-0.9.0

% head myvenv/bin/lit
#!./myvenv/bin/python3.7
# EASY-INSTALL-ENTRY-SCRIPT: 'lit==0.9.0','console_scripts','lit'
__requires__ = 'lit==0.9.0'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(

% rm -rf $XDG_CACHE_HOME
% rm -rf myvenv
% pip3 install lit
Collecting lit
  Downloading lit-0.9.0.tar.gz (87 kB)
     |████████████████████████████████| 87 kB 2.7 MB/s
Building wheels for collected packages: lit
  Building wheel for lit (setup.py) ... done
  Created wheel for lit: filename=lit-0.9.0-py3-none-any.whl size=69762 sha256=04a8bb642b9cf046820d69e3c71a11dffdcd3eb5ed6720ec88d1c5637c8aa5d3
  Stored in directory: ./myvenv.cache/pip/wheels/8c/9c/a9/adb97c130a3ef201435efffc782f8665d8a57574752f8d44a7
Successfully built lit
Installing collected packages: lit
Successfully installed lit-0.9.0

% myvenv/bin/pip3 install --upgrade pip
Collecting pip
  Downloading https://files.pythonhosted.org/packages/54/0c/d01aa759fdc501a58f431eb594a17495f15b88da142ce14b5845662c13f3/pip-20.0.2-py2.py3-none-any.whl (1.4MB)
     |████████████████████████████████| 1.4MB 7.4MB/s
Installing collected packages: pip
  Found existing installation: pip 19.2.3
    Uninstalling pip-19.2.3:
      Successfully uninstalled pip-19.2.3
Successfully installed pip-20.0.2

% myvenv/bin/pip3 install lit
Processing ./myvenv.cache/pip/wheels/8c/9c/a9/adb97c130a3ef201435efffc782f8665d8a57574752f8d44a7/lit-0.9.0-py3-none-any.whl
Installing collected packages: lit
Successfully installed lit-0.9.0
Found existing installation: lit 0.9.0
Uninstalling lit-0.9.0:
  Successfully uninstalled lit-0.9.0
Processing ./myvenv.cache/pip/wheels/8c/9c/a9/adb97c130a3ef201435efffc782f8665d8a57574752f8d44a7/lit-0.9.0-py3-none-any.whl
Installing collected packages: lit
Successfully installed lit-0.9.0

% head myvenv/bin/lit
#!./myvenv/bin/python3.7
# -*- coding: utf-8 -*-
import re
import sys
from lit import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

[uranusjr]

pip needs wheel to build the package without setuptools, so a fresh venv wouldn’t be able to achieve your expectation. The behaviour can indeed be confusing, see #7709 for a proposal for improvements.

Also note that if you’re a maintainer of the package, one way to avoid this problem is to adopt PEP 518, by creating a pyproject.toml with the following content:

[build-system]
requires = ["setuptools", "wheel"]

[lewis6991]

Many thanks for the response. Installing wheel in my venv nicely works around the issue.