GitHub dependency with hashed pins fails on Windows for Python < 3.10

[nedbat]

Issue

After upgrading to tox 4, one of my GitHub-installed dependencies is failing, but only on Windows, and only for Python versions 3.7, 3.8, and 3.9. Other OS's, and Python 3.10 or 3.11 on Windows are successful.

The run with three OS's, and Windows failing, is here: https://github.com/nedbat/coveragepy/actions/runs/3772843444/jobs/6413996847

The failure is here: https://github.com/nedbat/coveragepy/actions/runs/3772843444/jobs/6413996895#step:6:569

Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\packaging\requirements.py", line 35, in __init__
    parsed = parse_requirement(requirement_string)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\packaging\_parser.py", line 64, in parse_requirement
    return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\packaging\_parser.py", line 82, in _parse_requirement
    url, specifier, marker = _parse_requirement_details(tokenizer)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\packaging\_parser.py", line 111, in _parse_requirement_details
    marker = _parse_requirement_marker(
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\packaging\_parser.py", line 143, in _parse_requirement_marker
    tokenizer.raise_syntax_error(
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\packaging\_tokenizer.py", line 161, in raise_syntax_error
    raise ParserSyntaxError(
packaging._tokenizer.ParserSyntaxError: Expected end or semicolon (after URL and whitespace)
    pycontracts @ https://github.com/slorg1/contracts/archive/c5a6da27d4dc9985f68e574d20d86000880919c3.zip    
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\python\pip\req\file.py", line 37, in __init__
    self._requirement: Requirement | Path | str = Requirement(req)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\packaging\requirements.py", line 37, in __init__
    raise InvalidRequirement(str(e)) from e
packaging.requirements.InvalidRequirement: Expected end or semicolon (after URL and whitespace)
    pycontracts @ https://github.com/slorg1/contracts/archive/c5a6da27d4dc9985f68e574d20d86000880919c3.zip    
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\session\cmd\run\single.py", line 45, in _evaluate
    tox_env.setup()
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\api.py", line 242, in setup
    self._setup_env()
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\python\runner.py", line 99, in _setup_env
    self._install_deps()
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\python\runner.py", line 103, in _install_deps
    self._install(requirements_file, PythonRun.__name__, "deps")
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\api.py", line 96, in _install
    self.installer.install(arguments, section, of_type)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\python\pip\pip_install.py", line 83, in install
    self._install_requirement_file(arguments, section, of_type)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\python\pip\pip_install.py", line 92, in _install_requirement_file
    new_options, new_reqs = arguments.unroll()
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\python\pip\req_file.py", line 104, in unroll
    opts_dict = vars(self.options)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\python\pip\req\file.py", line 157, in options
    self._ensure_requirements_parsed()
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\python\pip\req\file.py", line 177, in _ensure_requirements_parsed
    self._requirements = self._parse_requirements(opt=self._opt, recurse=True)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\python\pip\req_file.py", line 92, in _parse_requirements
    requirements = super()._parse_requirements(opt, recurse)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\python\pip\req\file.py", line 183, in _parse_requirements
    parsed_req = self._handle_requirement_line(parsed_line)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\python\pip\req\file.py", line 287, in _handle_requirement_line
    return ParsedRequirement(line.requirement, req_options, line.filename, line.lineno)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tox\tox_env\python\pip\req\file.py", line 61, in __init__
    rel_path = str(path.resolve().relative_to(root))
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\pathlib.py", line 1181, in resolve
    s = self._flavour.resolve(self, strict=strict)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\pathlib.py", line 206, in resolve
    s = self._ext_to_normal(_getfinalpathname(s))
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'D:\\a\\coveragepy\\coveragepy\\requirements\\pycontracts @ https:\\github.com\\slorg1\\contracts\\archive\\c5a6da27d4dc9985f68e574d20d86000880919c3.zip    '

The dependency is in dev.pip:

pycontracts @ https://github.com/slorg1/contracts/archive/c5a6da27d4dc9985f68e574d20d86000880919c3.zip \
    --hash=sha256:2b889cbfb03b43dc811b5879248ac5c7e209ece78f03be9633de76a6b21a5a89

Environment

Provide at least:

  python -m pip freeze --all
  shell: C:\Program Files\Git\bin\bash.EXE --noprofile --norc -e -o pipefail {0}
  env:
    PIP_DISABLE_PIP_VERSION_CHECK: 1
    COVERAGE_IGOR_VERBOSE: 1
    FORCE_COLOR: 1
    pythonLocation: C:\hostedtoolcache\windows\Python\3.8.10\x64
    PKG_CONFIG_PATH: C:\hostedtoolcache\windows\Python\3.8.10\x64/lib/pkgconfig
    Python_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.8.10\x64
    Python2_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.8.10\x64
    Python3_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.8.10\x64
cachetools==5.2.0
chardet==5.1.0
colorama==0.4.6
distlib==0.3.6
filelock==3.8.2
importlib-metadata==5.2.0
packaging==22.0
pip==22.3.1
platformdirs==2.6.0
pluggy==1.0.0
pyproject_api==1.2.1
setuptools==56.0.0
tomli==2.0.1
tox==4.0.16
typing_extensions==4.4.0
virtualenv==20.17.1
zipp==3.11.0

Output of running tox

Provide the output of tox -rvv: see the action run link above.

Minimal example

I tried to remove other factors (especially tox-gh-actions, which I think is fully removed from this run).

[gaborbernat]

  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\packaging\_tokenizer.py", line 161, in raise_syntax_error
    raise ParserSyntaxError(
packaging._tokenizer.ParserSyntaxError: Expected end or semicolon (after URL and whitespace)
    pycontracts @ https://github.com/slorg1/contracts/archive/c5a6da27d4dc9985f68e574d20d86000880919c3.zip    
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

As the trace indicates the issue is with packaging and not tox 👍

[nedbat]

I saw that in the stack trace, but the version of packaging is the same with tox 3 or with tox 4. What should I do to try to narrow this down?

[gaborbernat]

I saw that in the stack trace, but the version of packaging is the same with tox 3 or with tox 4.

Irrelevant. tox 3 does not parse your requirement files, per https://tox.wiki/en/latest/faq.html#new-features-in-tox-4 We now discover your package dependency... this is a new feature, and is what's breaking here because a dependency specified seems invalid:

from packaging.requirements import Requirement

req = "pycontracts @ https://github.com/slorg1/contracts/archive/c5a6da27d4dc9985f68e574d20d86000880919c3.zip "
Requirement(req)

From some quick local testing seems the trailing whitespace is causing the issue, so perhaps report that to packaging?

[gaborbernat]

The fact that this fails only on Windows is actually a tox bug 😆 it should fail everywhere 😆

[gaborbernat]

@nedbat seems master branch https://github.com/nedbat/coveragepy/blob/master/requirements/dev.pip no longer contains this line...

[nedbat]

Yes, separately I got tired of keeping this dependency going. If you don't want to pursue this, I understand.

[gaborbernat]

https://github.com/tox-dev/tox/pull/2779/files should handle this edge case that's failing with packaging.

[gaborbernat]

https://github.com/tox-dev/tox/releases/tag/4.0.18

[nedbat]

Wow, thanks!