pip does not support spaces in directories names

[tarekziade]

under Mac OS X 10.7

tareks:tmp tarek$ mkdir "some space"
tareks:tmp tarek$ cd some\ space/
tareks:some space tarek$ virtualenv --no-site-packages .
The --no-site-packages flag is deprecated; it is now the default behavior.
New python executable in ./bin/python
Installing setuptools............done.
Installing pip...............done.
tareks:some space tarek$ bin/pip install docutils
-bash: bin/pip: "/private/tmp/some: bad interpreter: No such file or directory

Or maybe that's virtualenv ?

[pfmoore]

Looks like a problem with not quoting the path to the interpreter in the #! line in pip-script.py.

I can't reproduce on Windows because there is special code in virtualenv.py to use the "short name" which never has spaces.

First question would be, how should a #! line be written in Unix to handle an interpreter with spaces in the pathname? The #! processing is at such a low level that I wouldn't want to assume that quoting the filename simply works on all platforms without verification...

[pfmoore]

Sorry - should have mentioned, it's a virtualenv issue I'm talking about.

[tarekziade]

how should a #! line be written in Unix to handle an interpreter with spaces in the pathname?

I think you can simply escape it with ""

Notice that the python produced by virtualenv is a binary, and that binary alone produces the error. So yeah, definitely a virtualenv issue

[pfmoore]

Hmm, the python executable is simply copied/linked from the system one. Could this be a Python issue? What happens if you install Python into a directory with spaces in it? Note that the general rule is that Python doesn't like directories with spaces (that's one reason it's not installed in "Program Files" on Windows). I don't know if Python is expected to work when installed in a directory with spaces on OSX.

I don't have access to OSX, so I won't be able to put together or test a patch for this. Sorry.

[cellularmitosis]

I just got bit by this. Unfortunately, quoting in spaces nor escaping with backslashes fixes the problem.

I am incredibly dismayed to find that OS X does not support a shebang with spaces in the interpreter path! Gah!

I had to come up with an incredibly lame hack:

cd virtualenv/bin
mv pip .real_pip
vim pip

I then populated pip with the following:

#!/bin/bash

"/Users/jpepas/Developer/Peepcode/Meet Jquery/flask/virtualenv/bin/python" "/Users/jpepas/Developer/Peepcode/Meet Jquery/flask/virtualenv/bin/.real_pip" $@

So the only bullet-proof workaround I see is for each of the scripts which use a shebang (for my setup, that's pip, pip-2.7, easy_install, easy_install-2.7) to be replaced by this level-of-indirection hack.

I feel dirty now.

(update 2017/5/28: just to be clear, my dismay is directed at Apple, this isn't the virtualenv team's fault.)

[kot-behemoth]

Just encountered this problem on Mountain Lion 10.8.5, feels very strange.

[ionelmc]

Spent time fixing my bash scripts (quoting and escaping those damn paths) only to find out the pip/virtualenv tooling doesn't support spaces ... 😞

[Ivoz]

It's not pip/virtualenv that doesn't support spaces. It's fundamentally an issue of the shebang feature provided by *nixes. You just can't have a space in your interpreter path for the shebang line.

https://lists.gnu.org/archive/html/bug-bash/2008-05/msg00052.html

I found this workaround, I'm not sure if it would be universally applicable without creating any regressions though

#!/usr/bin/env /home/myname/tmp/a b/python

Then the python in ~/tmp/a b/ will be executed correctly, at least on ArchLinux.

[vsajip]

It's not pip/virtualenv that doesn't support spaces

It's not just a problem with spaces in shebang lines. For example, if you try and pip install something with C extensions (say ... a recent SQLAlchemy), and your Python headers are in a directory with a space in its full path, compilations fail because of unquoted arguments in the command line passed to the compiler :-(

[Ivoz]

compilations fail because of unquoted arguments in the command line passed to the compiler :-(

Is that a quoting failure on pip's or setuptool's part?

[vsajip]

Is that a quoting failure on pip's or setuptool's part?

Not sure - it might even be a bug in distutils. IMO one of these tools should paper over any cracks (seeing that distutils won't be seeing too many changes, especially on 2.x).

[AlexeyMK]

For folks looking for a simpler solution, I got #923 (comment) to work pretty well, in a simpler mode. I only ever needed to call pip once (during an install script), so instead of saying

virtualenv .env
.env/bin/activate
pip install -r requirements.txt

I did

virtualenv .env
.env/bin/activate
python .env/bin/pip install -r requirements.txt

Solved my my problem - still a challenge to call pip outside of the script, but the incision is more narrow.

[Josh-Tilles]

@AlexeyMK Thank you for posting that! It worked like a charm

[joeyespo]

Just ran into this with pip version 1.5.6.

pip intall ____ was not quoting #!C:\Program Files (x86)\Python\python.exe in ____-script.py leading to a "failed to create process" error when running ____.exe in scripts.

Seems it used to quote these paths, since older scripts installed fine. Not exactly sure what changed or when.

Edits:

• Using pip install -e . properly quotes the path. Installing from PyPI does not. I tried using pip 7.0.3 and am seeing the same problem.
• This setuptools fix might be the source of the problem

Update:

• I opened an issue here

[roskakori]

For the record: this is not limited to Unix, I encountered the same situation on Windows Server 2012 with (ancient) Python 2.6.6 and (current) pip 7.1.0 and setuptools 18.0.1. The generated pip-script.py started with

#!C:\Program Files\Python26\python.exe

which I manually fixed by adding quotes:

#!"C:\Program Files\Python26\python.exe"

[johnthagen]

I think this is related to #2783. I've been seeing a lot of problems on Windows pip installing packages with entry_points=['console_script'... when the user has a space in their user name. I'll put details on the Windows specific issue.

[amckinlay]

Any word on whether this will ever be fixed? I spent 15 minutes trying to figure out why pip wouldn't run inside my virtual environment.

first line of pip:
#!"/Users/andrewmckinlay/Documents/Triplett/PacBio problem/py-motifs-rerun/bin/python2.7"

yet:
bad interpreter: "/Users/andrewmckinlay/Documents/Triplett/PacBio: no such file or directory

[smoothmango]

I got @AlexeyMK's comment above to work, though what he did wasn't clear to me at first so I though I would rephrase it:

The problem here is that spaces in #! shebangs aren't allowed, and this is an OS level thing, not a bash quirk - therefore the standard practice of quoting a path won't work.

Rather than running pip directly (which itself is just a python script), and having it try to call the correct python interpreter implicitly (which fails), you can instead run it explicitly by calling python pip instead of plain pip. So python is just running the script pip and an implicit shebang interpreter path resolution isn't needed.

Summary:

With you virtualenv activated run python pip rather than pip to avoid errors if your virtualenv path has spaces in it.

virtualenv env    # create virtual env
source env/bin/activate  # activate it
python env/bin/pip {arguments}

[johnthagen]

@smoothmango Would another option be to run python -m pip (or python3 -m pip) rather than pip?

Seems to be one way the docs specify to run it: https://pip.pypa.io/en/stable/installing/#upgrading-pip

[smoothmango]

@johnthagen yes, the -m searches for modules in sys.path, so if you've activated your virtualenv, then it will find the correct pip [tested]

So you can shorten python .env/bin/pip to python -m pip

[yan12125]

@Ivoz's approach:

I found this workaround, I'm not sure if it would be universally applicable without creating any regressions though
#!/usr/bin/env /home/myname/tmp/a b/python

Does not work on Mac:

yen@PC951-Sierra:~/tmp$ virtualenv "v e n v"
New python executable in /Users/yen/tmp/v e n v/bin/python
Installing setuptools, pip, wheel...done.
yen@PC951-Sierra:~/tmp$ source ./v\ e\ n\ v/bin/activate
(v e n v) yen@PC951-Sierra:~/tmp$ vim v\ e\ n\ v/bin/pip
(v e n v) yen@PC951-Sierra:~/tmp$ cat ./v\ e\ n\ v/bin/pip
#!/usr/bin/env /Users/yen/tmp/v e n v/bin/python

# -*- coding: utf-8 -*-
import re
import sys

from pip import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())
(v e n v) yen@PC951-Sierra:~/tmp$ ./v\ e\ n\ v/bin/pip
env: /Users/yen/tmp/v: No such file or directory
(v e n v) yen@PC951-Sierra:~/tmp$

For those who are interested, see exec_shell_imgact() function in Mac's XNU kernel

[Zeex]

I had this problem on Windows 10 with Python 3.5.

I installed a package that provides a binary script but every time I run it I was getting failed to create process error.

Inserting quotes around the path to python.exe in package_name-script.py like suggested here solved the issue.

[vsajip]

@vsajip Could you cut a new release from the current master of distlib?

When time allows, so it might be sometime in early September. There are a couple of distlib issues I'm working on and would like to resolve before the next release.

[pradyunsg]

When time allows, so it might be sometime in early September.

That works well. ^.^

FTR - I just ran the test-suite of pip after vendoring the latest distlib and all tests passed.

---

I guess that means this would be shipping as a part of pip 10 -- adding to the milestone. :)

[Bilge]

I have this problem on Windows 7 and quoting the path in the shebang line does not help. I find it pretty shocking that in 2017 we still develop software that falls flat on its face when encountering a space.

[obestwalter]

I find it pretty shocking that after decades of agony developers still use spaces in paths and expect it to work :)

This is a long standing issue with many facets. The person who fixes it upstream will be revered and adored like no other, maybe something is happening already?

[pradyunsg]

Just to quote myself here...

I guess [...] this would be shipping as a part of pip 10

[Bilge]

But what is shipping as part of pip 10, and is it just for Mac users or will it support Windows also?

[pradyunsg]

is it just for Mac users or will it support Windows also?

I think the fix is for *nix systems, so Linux and Mac. I'm not sure about Windows; @pfmoore or @vsajip likely have a better idea about the state of the situation there than me.

what is shipping as part of pip 10

I'll quote @vsajip here:

special mechanisms to specify very long paths or paths with spaces in scripts installed using ScriptMaker

(apologies for being terse; I have a mild headache right now)

[pfmoore]

But what is shipping as part of pip 10, and is it just for Mac users or will it support Windows also?

If you want to help ensure that the next release solves your issue you should download and test the development version of pip from github, and report back here.

[Bilge]

I'm not really into Python and this just confirmed I don't want to be. It's taken 4 years to add support for a space, and only to some platforms.

[pradyunsg]

I can't reproduce on Windows because there is special code in virtualenv.py to use the "short name" which never has spaces.

FWIW this was never an issue on Windows apparently. So, yeah, this would fix it for all platforms.

---

There are a couple of distlib issues I'm working on and would like to resolve before the next release.

@vsajip ping! Any ETA on the next release of distlib? :)

[pradyunsg]

@vsajip Ping 2.0!

We're planning to do a pip 10 release soon -- in like another month or two at most ideally. It would be nice to get this fixed in pip 10. :)

[vsajip]

Some more issues came up, including ones that affect pip (example). Once resolved, I will cut a release.

[pradyunsg]

Awesome. :)

Just wanted to drop a reminder.

[pradyunsg]

Thanks for the release @vsajip!

This fix will be shipping as a part of pip 10.