FixedEggMetadata._zipinfo_name breakage

[kwlzn]

after testing the 1.1.3 pex release in pants, I encountered the following issue:

  ...
  File "/Users/kwilson/dev/pants/src/python/pants/backend/python/tasks/python_task.py", line 198, in _build_chroot
    builder.freeze()
  File "/Users/kwilson/dev/pants/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/pex_builder.py", line 407, in freeze
    self._prepare_bootstrap()
  File "/Users/kwilson/dev/pants/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/pex_builder.py", line 361, in _prepare_bootstrap
    x = self._interpreter.get_location('setuptools')

Exception message: Failed to extract pkg_resources from setuptools.  Perhaps pants was linked with an incompatible setuptools.

after a closer look, it seems related to a recent change to FixedEggMetadata._zipinfo_name in this commit:

1fe5a4e#diff-3ffb3bc5eb80cbb36374158a2dbbc433R140

which causes the following symptom:

[illuminati pex (master)]$ pex .
...
>>> from pex.util import DistributionHelper
>>> egg_path = '/Users/kwilson/.cache/pants/python_cache/interpreters/CPython-2.7.9/setuptools-5.4.1-py2.7.egg'
>>> d = DistributionHelper.distribution_from_path(egg_path, name='setuptools')
>>> d.resource_listdir('/')
[]

where the following is expected:

>>> d.resource_listdir('/')
['setuptools', 'pkg_resources.py', '_markerlib', 'EGG-INFO', 'easy_install.pyc', 'pkg_resources.pyc', 'easy_install.py']

a pdb trace before and after reveals a working _zipinfo_name() call pre-change:

> /Users/kwilson/dev/release/pants-release/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/finders.py(141)_zipinfo_name()
-> if fspath[:len(zip_pre)] == zip_pre:
(Pdb) fspath
['', 'Users', 'kwilson', '.cache', 'pants', 'python_cache', 'interpreters', 'CPython-2.7.9', 'setuptools-5.4.1-py2.7.egg']
(Pdb) fspath[:len(zip_pre)]
['', 'Users', 'kwilson', '.cache', 'pants', 'python_cache', 'interpreters', 'CPython-2.7.9', 'setuptools-5.4.1-py2.7.egg']
(Pdb) zip_pre
['', 'Users', 'kwilson', '.cache', 'pants', 'python_cache', 'interpreters', 'CPython-2.7.9', 'setuptools-5.4.1-py2.7.egg']
(Pdb) s
> /Users/kwilson/dev/release/pants-release/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/finders.py(142)_zipinfo_name()
-> return '/'.join(fspath[len(zip_pre):])
(Pdb) s
--Return--
> /Users/kwilson/dev/release/pants-release/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/finders.py(142)_zipinfo_name()->''
-> return '/'.join(fspath[len(zip_pre):])
(Pdb) s
--Return--
> /Users/kwilson/dev/release/pants-release/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pkg_resources.py(1765)_listdir()->['setuptools', 'pkg_resources.py', '_markerlib', 'EGG-INFO', 'easy_install.pyc', 'pkg_resources.pyc', ...]
-> return list(self._index().get(self._zipinfo_name(fspath), ()))

and then one after the change showcasing the branch miss in if fspath.startswith(self.zip_pre) due to the trailing slash in self.zip_pre that is absent in fspath:

--Call--
> /Users/kwilson/dev/pants/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/finders.py(138)_zipinfo_name()
-> def _zipinfo_name(self, fspath):
(Pdb) s
> /Users/kwilson/dev/pants/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/finders.py(140)_zipinfo_name()
-> if fspath.startswith(self.zip_pre):
(Pdb) fspath
'/Users/kwilson/.cache/pants/python_cache/interpreters/CPython-2.7.9/setuptools-5.4.1-py2.7.egg'
(Pdb) self.zip_pre
'/Users/kwilson/.cache/pants/python_cache/interpreters/CPython-2.7.9/setuptools-5.4.1-py2.7.egg/'
(Pdb) s
> /Users/kwilson/dev/pants/build-support/pants_dev_deps.venv/lib/python2.7/site-packages/pex/finders.py(142)_zipinfo_name()
-> assert "%s is not a subpath of %s" % (fspath, self.zip_pre)

@mikekap any chance you could take a look?

[kwlzn]

added a test to reveal this breakage and went ahead and reverted the breaking commit to unblock master.

feel free to resubmit a PR that makes this work for windows but doesn't break this code path elsewhere.