Install fails on Ubuntu precise

[mhw]

I've just run into problems trying to get a working version of simp_le on an up-to-date Ubuntu precise (12.04) machine. The initial failure is the installation of setuptools into the venv directory:

cert@ip-10-0-0-6:~/simp_le$ ./venv.sh
The --no-site-packages flag is deprecated; it is now the default behavior.
New python executable in venv/bin/python
Installing distribute.............................................................................................................................................................................................done.
Installing pip...............done.
Downloading/unpacking distribute from https://pypi.python.org/packages/5f/ad/1fde06877a8d7d5c9b60eff7de2d452f639916ae1d48f0b8f97bf97e570a/distribute-0.7.3.zip#md5=c6c59594a7b180af57af8a0cc0cf5b4a
  Downloading distribute-0.7.3.zip (145Kb): 145Kb downloaded
  Running setup.py egg_info for package distribute

Downloading/unpacking setuptools>=0.7 (from distribute)
  Downloading setuptools-34.1.1.zip (621Kb): 621Kb downloaded
  Running setup.py egg_info for package setuptools
    Traceback (most recent call last):
      File "<string>", line 3, in <module>
      File "setuptools/__init__.py", line 10, in <module>
        from six.moves import filter, map
    ImportError: No module named six.moves
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 3, in <module>

  File "setuptools/__init__.py", line 10, in <module>

    from six.moves import filter, map

ImportError: No module named six.moves

----------------------------------------
Command python setup.py egg_info failed with error code 1 in /home/cert/simp_le/venv/build/setuptools
Storing complete log in /home/cert/.pip/pip.log
Downloading/unpacking pip from https://pypi.python.org/packages/11/b6/abcb525026a4be042b486df43905d6893fb04f05aac21c32c638e939e447/pip-9.0.1.tar.gz#md5=35f01da33009719497f01a4ba69d63c9
  Downloading pip-9.0.1.tar.gz (1.2Mb): 1.2Mb downloaded
[...]
cert@ip-10-0-0-6:~/simp_le$ simp_le
Traceback (most recent call last):
  File "/home/cert/simp_le/venv/bin/simp_le", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/home/cert/simp_le/venv/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 2707, in <module>
    working_set.require(__requires__)
  File "/home/cert/simp_le/venv/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 686, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/cert/simp_le/venv/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 584, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: mock
cert@ip-10-0-0-6:~/simp_le$~/simp_le$

It seems both pip and distribute need to be upgraded from the versions that ship with virtualenv on Precise, before setuptools is installed. I made the following change to my copy:

diff --git a/venv.sh b/venv.sh
index 72394f3..f03a3d7 100755
--- a/venv.sh
+++ b/venv.sh
@@ -2,7 +2,7 @@

 virtualenv --no-site-packages venv
 export PATH="$PWD/venv/bin:$PATH"  # #49, activate script requires bash
-for pkg in setuptools pip wheel
+for pkg in pip distribute setuptools wheel
 do
   pip install -U "${pkg?}"
 done

and the installation then produced a working version of simp_le. I don't know if that change would cause issues for more modern distributions and Python installs though.

[mhw]

See also pypa/setuptools#940

[zenhack]

Did you try it with just upgrading pip (not distribute)? I don't see any reference to distribute in the issue you linked, and it wouldn't make a lot of sense; distribute was a fork of setuptools that has sense been merged back in, and the distribute package on pypi is supposedly just a shim that pulls in setuptools>=0.7.

Do you get the same error when upgrading just pip?

[mhw]

I did try without upgrading distribute, yes. Upgrading pip and then setuptools gets the venv.sh script to complete successfully, but simp_le would still fail with the error about pkg_resources.DistributionNotFound: mock.

I eventually found that distribute was the key when I managed to stumble on a working version by interrupting the venv.sh script part way through. I've just worked out the steps I went through and arrived at this:

rm -rf venv
# First two steps that would be run by venv.sh script
virtualenv venv
pip install -U setuptools
# Interrupt venv.sh, then run following manually
pip install -U pip
pip install -U setuptools
# Re-run venv.sh script
virtualenv venv
pip install -U setuptools

So I stumbled on a working version by running venv.sh, interrupting after the setuptools install error, then manually upgrading pip and then setuptools and finally rerunning the venv.sh again. In this case the final upgrade of setuptools looks a bit different:

Downloading/unpacking distribute from https://pypi.python.org/packages/5f/ad/1fde06877a8d7d5c9b60eff7de2d452f639916ae1d48f0b8f97bf97e570a/distribute-0.7.3.zip#md5=c6c59594a7b180af57af8a0cc0cf5b4a
  Downloading distribute-0.7.3.zip (145Kb): 145Kb downloaded
  Running setup.py egg_info for package distribute

Downloading/unpacking setuptools>=0.7 (from distribute)
  Downloading setuptools-34.1.1.zip (621Kb): 621Kb downloaded
  Running setup.py egg_info for package setuptools
    warning: no files found matching '*' under directory 'setuptools/_vendor'
Downloading/unpacking packaging>=16.8 (from setuptools>=0.7->distribute)
  Downloading packaging-16.8.tar.gz (44Kb): 44Kb downloaded
  Running setup.py egg_info for package packaging

    warning: no previously-included files found matching '.travis.yml'
    warning: no previously-included files found matching 'dev-requirements.txt'
    no previously-included directories found matching 'docs/_build'
    no previously-included directories found matching 'tasks'
Downloading/unpacking six>=1.6.0 (from setuptools>=0.7->distribute)
  Downloading six-1.10.0.tar.gz
  Running setup.py egg_info for package six

    no previously-included directories found matching 'documentation/_build'
Downloading/unpacking appdirs>=1.4.0 (from setuptools>=0.7->distribute)
  Downloading appdirs-1.4.0.tar.gz
  Running setup.py egg_info for package appdirs

Downloading/unpacking pyparsing (from packaging>=16.8->setuptools>=0.7->distribute)
  Downloading pyparsing-2.1.10.zip (1.4Mb): 1.4Mb downloaded
  Running setup.py egg_info for package pyparsing

Installing collected packages: distribute, setuptools, packaging, six, appdirs, pyparsing
  Found existing installation: distribute 0.6.24
    Uninstalling distribute:
      Successfully uninstalled distribute
  Running setup.py install for distribute

  Running setup.py install for setuptools
 [...]

That is, going through these steps results in distribute 0.6.24 being replaced with 0.7.3. This proves to be the necessary step to get a working version of simp_le. For some reason none of the previous upgrades of setuptools will replace the version of distribute that the Ubuntu python-virtualenv package installs, and the presence of the older version of distribute seems to be what prevents simp_le from starting up.

[zenhack]

When I apply your patch I get this:

Collecting distribute
  Using cached distribute-0.7.3.zip
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-bz38ia3i/distribute/setuptools/__init__.py", line 2, in <module>
        from setuptools.extension import Extension, Library
      File "/tmp/pip-build-bz38ia3i/distribute/setuptools/extension.py", line 5, in <module>
        from setuptools.dist import _get_unpatched
      File "/tmp/pip-build-bz38ia3i/distribute/setuptools/dist.py", line 7, in <module>
        from setuptools.command.install import install
      File "/tmp/pip-build-bz38ia3i/distribute/setuptools/command/__init__.py", line 8, in <module>
        from setuptools.command import install_scripts
      File "/tmp/pip-build-bz38ia3i/distribute/setuptools/command/install_scripts.py", line 3, in <module>
        from pkg_resources import Distribution, PathMetadata, ensure_directory
      File "/tmp/pip-build-bz38ia3i/distribute/pkg_resources.py", line 1518, in <module>
        register_loader_type(importlib_bootstrap.SourceFileLoader, DefaultProvider)
    AttributeError: module 'importlib._bootstrap' has no attribute 'SourceFileLoader'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-bz38ia3i/distribute/

simp_le still installs correctly, but distribute doesn't.

I'm going to see if I can reproduce this on a CentOS 6 box; if not I'm inclined to just close this. 12.04 is only supported by Canonical for another 3 months, and I'd rather not be pulling in compatibility goop (especially if it causes error messages, real or not) if we don't have to.

[mhw]

Yeah, I wouldn't worry too much about it if the fix looks like a backwards step.

My main reason for reporting the issue was to get the details somewhere that they could be found by others who hit the problem. 12.04 is old enough that getting any Let's Encrypt tools running on it at all is a bit of a challenge (or at least was when I first tried a few months ago) and simp_le was the tool that worked for me. But then I ran into the issue with the original kuba/simp_le repository not working with the current Let's Encrypt API and updated to your fork to rectify that (for which, thanks very much). And here we are.

[zenhack]

CentOS 6 doesn't even have a distro package for virtualenv, so I'm going to go ahead and close.