_parse_version_many does not match PEP 508 or documentation

[matt-phylum]

PEP 508 says that a version_many looks like version_one (wsp* ',' version_one)* and the documentation comment in _parse_version_many says that a version_many looks like (SPECIFIER (WS? COMMA WS? SPECIFIER)*)? (essentially the same thing, but optional).

However, the implementation of _parse_version_many actually parses something like version_one (wsp* ',' version_one)* wsp* (',' wsp*)? or (SPECIFIER (WS? COMMA WS? SPECIFIER)* WS? (COMMA WS?)?)?. An extra comma, possibly surrounded by whitespace, is accepted.

It's probably too late to fix the implementation. Packages like this one contain malformed requirements that are accepted by the implementation but not the documentation. The documentation should probably be changed to reflect reality instead.

from packaging._tokenizer import DEFAULT_RULES, Tokenizer
from packaging._parser import _parse_version_many

def parse(input: str) -> str:
    tokenizer = Tokenizer(input, rules=DEFAULT_RULES)
    _parse_version_many(tokenizer)
    return (tokenizer.source[:tokenizer.position], tokenizer.source[tokenizer.position:])

print(repr(parse(">=1.0")))
print(repr(parse(">=1.0 ")))
print(repr(parse(">=1.0,")))
print(repr(parse(">=1.0 ,")))
print(repr(parse(">=1.0 , ")))

produces

('>=1.0', '')
('>=1.0 ', '')
('>=1.0,', '')
('>=1.0 ,', '')
('>=1.0 , ', '')

It looks like the extra, empty specifier is accidentally removed by SpecifierSet when it tries to handle converting an empty string into an empty set.

from packaging.specifiers import SpecifierSet

print(SpecifierSet("")._specs)
print(SpecifierSet(">=1.0")._specs)
print(SpecifierSet(">=1.0,")._specs)

produces

frozenset()
frozenset({<Specifier('>=1.0')>})
frozenset({<Specifier('>=1.0')>})

Together, these behaviors cause trailing commas in requirements to be ignored, allowing invalid requirements to be installed by setuptools.

from packaging.requirements import Requirement

print(Requirement("a>=1.0,"))

produces

a>=1.0

[uranusjr]

The canonical spec source is on packaging.python.org: https://packaging.python.org/en/latest/specifications/version-specifiers/#compatible-release

So the most important thing would be to raise a PR to pypa/packaging.python.org if we don’t decide to change the implementation. (Personally I feel it’s not worthwhile to break people’s things for this.)

[matt-phylum]

It looks like this is the relevant spec from packaging.python.org: https://packaging.python.org/en/latest/specifications/dependency-specifiers/ It's the same as the PEP.