testpath conda package breaks pip

[ostrokach]

Issue:

Installing testpath>=0.4 into a conda environment breaks pip.

The error message we get when we try to use pip is:

pip._vendor.pkg_resources.RequirementParseError: Invalid requirement, parse error at "'; extra '"

Related Github Issues:

• Added metrics for currently running terminals and labeled by type kernels jupyter/notebook#4036
• Couldnt set it up on Ubuntu Server 18.04  fastai/fastai#799

Related StackOverflow questions:

• https://stackoverflow.com/questions/52582563/pip-install-attributeerror-distinfodistribution-dep-map

Full error message:

Exception:
Traceback (most recent call last):
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2869, in _dep_map
    return self.__dep_map
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2663, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 93, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1632, in parseString
    raise exc
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1622, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3395, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1383, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3183, in parseImpl
    raise ParseException(instring, loc, self.errmsg, self)
pip._vendor.pyparsing.ParseException: Expected stringEnd (at char 33), (line:1, col:34)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2949, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 97, in __init__
    requirement_string[e.loc:e.loc + 8]))
pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'; extra '"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_internal/basecommand.py", line 141, in main
    status = self.run(options, args)
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 330, in run
    self._warn_about_conflicts(to_install)
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 456, in _warn_about_conflicts
    package_set, _dep_info = check_install_conflicts(to_install)
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 98, in check_install_conflicts
    package_set = create_package_set_from_installed()
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 41, in create_package_set_from_installed
    package_set[name] = PackageDetails(dist.version, dist.requires())
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2607, in requires
    dm = self._dep_map
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2871, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2881, in _compute_dependencies
    reqs.extend(parse_requirements(req))
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2942, in parse_requirements
    yield Requirement(line)
  File "/home/username/anaconda3/envs/show_error/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2951, in __init__
    raise RequirementParseError(str(e))
pip._vendor.pkg_resources.RequirementParseError: Invalid requirement, parse error at "'; extra '"

Environment (conda list):

$ conda list
# packages in environment at /home/kimlab1/strokach/anaconda3/envs/show_error:
#
# Name                    Version                   Build  Channel
bzip2                     1.0.6                h470a237_2    conda-forge
ca-certificates           2018.8.24            ha4d7672_0    conda-forge
certifi                   2018.8.24             py36_1001    conda-forge
libffi                    3.2.1                hfc679d8_5    conda-forge
libgcc-ng                 7.2.0                hdf63c60_3    conda-forge
libstdcxx-ng              7.2.0                hdf63c60_3    conda-forge
ncurses                   6.1                  hfc679d8_1    conda-forge
openssl                   1.0.2p               h470a237_0    conda-forge
pip                       18.0                     py36_1    conda-forge
python                    3.6.6                h5001a0f_0    conda-forge
readline                  7.0                  haf1bffa_1    conda-forge
setuptools                40.4.0                py36_1000    conda-forge
sqlite                    3.25.2               hb1c47c0_0    conda-forge
testpath                  0.4.1                    py36_0    conda-forge
tk                        8.6.8                ha92aebf_0    conda-forge
wheel                     0.31.1                py36_1001    conda-forge
xz                        5.2.4                h470a237_1    conda-forge
zlib                      1.2.11               h470a237_3    conda-forge

Details about conda and system ( conda info ):

$ conda info
     active environment : show_error
    active env location : /home/kimlab1/strokach/anaconda3/envs/show_error
            shell level : 2
       user config file : /home/kimlab1/strokach/.condarc
 populated config files : /home/kimlab1/strokach/.condarc
          conda version : 4.5.11
    conda-build version : 3.12.0
         python version : 3.6.4.final.0
       base environment : /home/kimlab1/strokach/anaconda3  (writable)
           channel URLs : https://conda.anaconda.org/pytorch/linux-64
                          https://conda.anaconda.org/pytorch/noarch
                          https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/linux-64
                          https://repo.anaconda.com/pkgs/pro/noarch
                          https://conda.anaconda.org/ostrokach-forge/linux-64
                          https://conda.anaconda.org/ostrokach-forge/noarch
                          https://conda.anaconda.org/kimlab/linux-64
                          https://conda.anaconda.org/kimlab/noarch
                          https://conda.anaconda.org/bioconda/linux-64
                          https://conda.anaconda.org/bioconda/noarch
                          https://conda.anaconda.org/salilab/linux-64
                          https://conda.anaconda.org/salilab/noarch
          package cache : /home/strokach/conda-cache
       envs directories : /home/kimlab1/strokach/anaconda3/envs
                          /home/kimlab1/strokach/.conda/envs
               platform : linux-64
             user-agent : conda/4.5.11 requests/2.18.4 CPython/3.6.4 Linux/4.15.0-34-generic linuxmint/19 glibc/2.27
                UID:GID : 8002:7300
             netrc file : None
           offline mode : False

[ocefpaf]

I can confirm this and I moved the latest testpath to broken. Let's wait for a new release. I guess that #6 (comment) the culprit.

[takluyver]

FWIW, I think this should be reported to pip as a bug, if it hasn't already been. There was certainly a bug in Flit causing this, but it shouldn't be possible to cripple pip by installing a package with bad metadata.

[takluyver]

nvm, it looks like it has been reported: pypa/pip#5839

[ltupin]

Instead of remove it you can do testpath>=0.4.

[ltupin]

I've opened an issue on testpath git repo.

[sorenwacker]

how did you find out this was caused by testpath? I had the same error and by chance I found this thread, though the error message does not say anything about testpath. So, how did you know?

[ostrokach]

I put a pdb breakpoint in setuptools before the line that throws the error and then googled the string that was causing the error. I forgot what that string was, but it led me to testpath in one of the top google hits.