distutils.cfg Can Break venv

[nicksloan]

BPO	17732
Nosy	@birkenfeld, @vsajip, @larryhastings, @tarekziade, @carljm, @merwok
Files	distutilsvenv.patch: distutils/dist.py patch for venv distutilsvenv.patch: Updated patch. distutilsvenv.patch distutilsvenv.patch distutilsvenv.patch distutilsvenv.patch distutilsvenv.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/merwok'
closed_at = <Date 2013-05-12.11:23:17.029>
created_at = <Date 2013-04-14.18:10:32.241>
labels = ['type-bug', 'library', 'release-blocker']
title = 'distutils.cfg Can Break venv'
updated_at = <Date 2013-05-16.17:57:20.161>
user = 'https://bugs.python.org/nicksloan'

bugs.python.org fields:

activity = <Date 2013-05-16.17:57:20.161>
actor = 'georg.brandl'
assignee = 'eric.araujo'
closed = True
closed_date = <Date 2013-05-12.11:23:17.029>
closer = 'georg.brandl'
components = ['Distutils']
creation = <Date 2013-04-14.18:10:32.241>
creator = 'nicksloan'
dependencies = []
files = ['29870', '29873', '29888', '29889', '29939', '29940', '29976']
hgrepos = []
issue_num = 17732
keywords = ['patch']
message_count = 24.0
messages = ['186942', '186943', '186945', '186973', '186978', '187004', '187010', '187012', '187018', '187021', '187111', '187112', '187113', '187296', '187303', '187363', '187376', '187386', '187563', '188446', '188546', '189011', '189387', '189393']
nosy_count = 9.0
nosy_names = ['georg.brandl', 'vinay.sajip', 'larry', 'tarek', 'carljm', 'eric.araujo', 'python-dev', 'nicksloan', 'eruart']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'patch review'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue17732'
versions = ['Python 3.3', 'Python 3.4']

[nicksloan]

When distutils.cfg defines an install-lib, it will be used within virtual environments created using venv as well, which makes it impossible to install things with setup.py or similar.

Steps to reproduce:

Create a distutils.cfg file and set a value for install-lib.

Create a virtual environment: $pyvenv myvenv

Activate that environment: $source myvenv/bin/activate

Attempt to install something: $python distribute_setup.py

It will attempt to install in the lib-install directory, and should fail with an error that the directory is not on the python path. This issue affects python3 from the mac homebrew project, which bundles a distutil.cfg, thus breaking pyvenv by default.

[merwok]

There is a --no-user-cfg option to disable reading ~/.pydistutils.cfg, but I don’t recall an option to ignore the global config. How does virtualenv avoid this problem, if it does?

[nicksloan]

It looks like virtualenv includes an empty distutils.cfg at myvirtualenv/lib/python3.3/distutils/distutils.cfg. Most of myvirtualenv/lib/python3.3 is just symlinked to the base python version, but myvirtualenv/lib/python3.3/distutils/ is not, and I bet this is exactly why.

[vsajip]

This looks to me as if it will need a patch in distutils. Unlike virtualenv, which contains a patched copy of distutils (and hence allows having a .cfg adjacent to it), pyvenv does not create patched modules in the venv. It does not make sense to change this behaviour.

The correct solution would appear to be for distutils to ignore certain configuration options (install-lib, but also equivalent options for headers and scripts) when running in a venv.

[nicksloan]

That's along the lines of what I've been thinking as I dig into this. I'd love to take a stab at a patch for this if no one else has done so already.

[nicksloan]

Here is a patch that seems to fix the problem. It simply short-circuits distutils options that change directories. This is my first python patch ever, so I'm eager for comments. Is this the right approach?

[nicksloan]

On second thought, there is probably no good reason to ignore the build-* settings. Here is an updated patch.

[vsajip]

On second thought, there is probably no good reason to ignore the build-* settings.

I was just about to mention this. I'm not an expert on distutils internals and whether this is the best way to accomplish what's needed, but the approach seems reasonable to me. Why the empty list and the extend, though? Why not something like the following?

if sys.prefix == sys.base_prefix:
    # not in venv
    ignore_options = []
else:
    # in venv
    ignore_options = ['install-base', ...]

My other comments on the patch are generic, i.e. you need to look at the tests and docs, too.

Anyway, thanks for the effort you've spent to come up with a patch. Can I suggest that you sign a PSF contributor form to license your work to the PSF?

http://www.python.org/psf/contrib/contrib-form/