Understanding binstubs

Binstubs are wrapper scripts around executables (sometimes referred to as "binaries", although they don't have to be compiled) whose purpose is to prepare the environment before dispatching the call to the original executable.

In the Python world, the most common binstubs are the ones that setuptools generates after installing a package that contains executables. But binstubs can be written in any language, and it often makes sense to create them manually.

setuptools

Let's see what happens when we pip install nose. [nose ships with an executable][nosetests]. After the installation, setuptools will provide us with the following executables:

1. <python-prefix>/bin/nosetests (binstub generated by setuptools)
2. <python-prefix>/lib/python2.7/site-packages/nose/__init__.py (original)

The first file is a binstub created to wrap the second. setuptools put it in <python-prefix>/bin because that directory is considered to already be in our $PATH.

The directory where setuptools installed the second file (the original) isn't in our $PATH, but even if it was, it wouldn't be safe to run it directly because executables in Python projects often aren't meant to be called directly without any setup.

The generated binstub <python-prefix>/bin/nosetests is a short Python script, presented in a slightly simplified form here:

#!/usr/bin/env python
# EASY-INSTALL-ENTRY-SCRIPT: 'nose==1.3.0','console_scripts','nosetests'
__requires__ = 'nose==1.3.0'
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.exit(
        load_entry_point('nose==1.3.0', 'console_scripts', 'nosetests')()
    )

The purpose of every setuptools binstub is to use setuptools to prepare the sys.path before calling the original executable.

pyenv

pyenv adds its own "shims" directory to $PATH which contains binstubs for every executable related to Python. There are binstubs for python, pip, and for all setuptools binstubs across each installed Python version.

When you call nosetests on the command-line, it results in this call chain:

1. $PYENV_ROOT/shims/nosetests (pyenv shim)
2. $PYENV_ROOT/versions/2.7.5/bin/nosetests (setuptools binstub)
3. $PYENV_ROOT/versions/2.7.5/lib/python2.7/site-packages/nose/__init__.py (original)

An pyenv shim, presented here in a slightly simplified form, is a short shell script:

#!/usr/bin/env bash
export PYENV_ROOT="$HOME/.pyenv"
exec pyenv exec "$(basename "$0")" "$@"

The purpose of pyenv's shims is to route every call to a python executable through pyenv exec, which ensures it gets executed with the right Python version.