Injected shebang does not escape whitespace in interpreter path

[j-hui]

Describe the bug

After I've installed a package, whitespace is not properly escaped in the shebang.

How to reproduce

$ pipx install --verbose pass2csv
pipx >(setup:853): pipx version is 1.4.2
pipx >(setup:854): Default python interpreter is '/opt/homebrew/opt/python@3.12/libexec/bin/python'
pipx >(package_name_from_spec:370): Determined package name: pass2csv
pipx >(package_name_from_spec:371): Package name determined in 0.0s
creating virtual environment...
pipx >(run_subprocess:168): running /opt/homebrew/opt/python@3.12/libexec/bin/python -m venv --without-pip /Users/j-hui/Library/Application Support/pipx/venvs/pass2csv
pipx >(run_subprocess:168): running <checking pip's availability>
pipx >(run_subprocess:168): running /Users/j-hui/Library/Application Support/pipx/venvs/pass2csv/bin/python -c import sysconfig; print(sysconfig.get_path('purelib'))
pipx >(run_subprocess:168): running /Users/j-hui/Library/Application Support/pipx/shared/bin/python-c import sysconfig; print(sysconfig.get_path('purelib'))
pipx >(run_subprocess:168): running /Users/j-hui/Library/Application Support/pipx/venvs/pass2csv/bin/python --version
pipx >(_parsed_package_to_package_or_url:137): cleaned package spec: pass2csv
installing pass2csv...
pipx >(run_subprocess:168): running /Users/j-hui/Library/Application Support/pipx/venvs/pass2csv/bin/python -m pip --no-input install pass2csv
pipx >(run_subprocess:168): running <fetch_info_in_venv commands>
pipx >(get_venv_metadata_for_package:339): get_venv_metadata_for_package: 21ms
pipx >(_parsed_package_to_package_or_url:137): cleaned package spec: pass2csv
pipx >(run_subprocess:168): running <checking pip's availability>
pipx >(needs_upgrade:78): Time since last upgrade of shared libs, in seconds: 555. Upgrade will be run by pipx if greater than 2592000.
pipx >(run_subprocess:168): running <checking pip's availability>
  installed package pass2csv 1.0.0, installed using Python 3.12.1
  These apps are now globally available
    - pass2csv
done! ✨ 🌟 ✨

$ pass2csv
zsh: /Users/j-hui/.local/bin/pass2csv: bad interpreter: /Users/j-hui/Library/Application: no such file or directory

This is due to the unescaped whitespace in the shebang:

$ head -n3 /Users/j-hui/.local/bin/pass2csv
#!/Users/j-hui/Library/Application Support/pipx/venvs/pass2csv/bin/python
import argparse
import csv

(But it seems like escaping whitespace in the shebang is not actually supported.)

Expected behavior

It should work.

[j-hui]

Related: conda/conda#186

[chrysle]

Duplicate of #1198. The workaround is now documented.

[Gitznik]

Does this actually fix it @j-hui @chrysle? /Users/j-hui/.local/bin/pass2csv is an entry point file that pipx generates and specifies the shebang I think. Symlinking won't change the file that pipx generates.

The docs are for using a pipx managed application within another shebang.

[chrysle]

Right, I missed that, sorry.

/Users/j-hui/.local/bin/pass2csv is an entry point file that pipx generates

Actually, it is the script created by the venv to run the application, copied into the local bin directory. I think we should patch the path given to the method exposing the script:

pipx/src/pipx/commands/common.py

Line 56 in 950c9b0

paths: List[Path],

Although that would require that the pipx home directory is always symlinked.

[Gitznik]

There is an issue in iterm2 where the resolution was to symlink to .config/iterm2 when install iterm. We could go down a similar route.

https://gitlab.com/gnachman/iterm2/-/issues/9362

Actually, it is the script created by the venv to run the application, copied into the local bin directory

The more you know 👍

[Gitznik]

I've had another look into this.

While the new pipx installation location on mac surfaces this issue, it is not a pipx issue. It's the packages use of setup.py scripts that causes this.

For reproducing this, I created a dir called test dir with spaces on my machine (Ubuntu).

For the entrypoint of a python application, pip performs some magic with the shebang of the executable:

> test dir with spaces head -n7 "$(which visidata)"

#!/bin/sh
'''exec' "/home/robert/private/code/test dir with spaces/.ven/bin/python3.11" "$0" "$@"
' '''
# -*- coding: utf-8 -*-
import re
import sys
from visidata.main import vd_cli

It does not do this on the scripts of the installed python application:

> test dir with spaces head -n3 "$(which vd)"

#!/home/robert/private/code/test dir with spaces/.ven/bin/python3.11

import visidata.main

The application mentioned in this issue does not define an entrypoint, so it cannot be invoked if the venv lives in a path with spaces in it. Same for ranger. visidata defines visidata as entrypoint, which works, and vd as script, which does not.

This was using just regular pip install into a venv.

I've also spun up a mac VM and used pipx to do the installation, which shows the same behavior:

IMO this should be reraised on the packages where you face this issue or on pip. The only thing we can do on pipx side about this, is catching the error and providing a more helpful error message.

[chrysle]

Thanks for reproducing! Sounds good. Since the author recommends installing pass2csv via pipx, we should probably nudge them towards providing a pipx-compatible entry point.

Also, the linked entry point section is somewhat outdated, since it describes setup.py usage (while you can of course also define entry points in the today recommended pyproject.toml). We should update it.

[Gitznik]

Sweet. I'll split this into separate issues and close this one.