Package update not found when name contains .

[wwuck]

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us what version of Renovate you run.

renovate/renovate:32.68.1-slim

Please select which platform you are using if self-hosting.

Bitbucket Server

If you're self-hosting Renovate, tell us what version of the platform you run.

7.21.1

Was this something which used to work for you, and then stopped?

I never saw this working

Describe the bug

Renovate is not finding any python package updates where the package name contains . but the wheel .whl filename has the . replaced with _.

The relevant spec: https://packaging.python.org/specifications/binary-distribution-format/#escaping-and-unicode

I have a python PEP420 namespace package with the name testing.test-namespace-package and I can upload it successfully to our internal private Nexus repository.

If I follow the URL that renovate is looking at on our Nexus instance, I get these filenames:

testing_test_namespace_package-1.0.0-py3-none-any.whl
testing_test_namespace_package-1.0.1-py3-none-any.whl
testing_test_namespace_package-1.0.2-py3-none-any.whl
testing_test_namespace_package-1.0.3-py3-none-any.whl

Looking at

renovate/lib/modules/datasource/pypi/index.ts

Line 170 in 13d0255

private static extractVersionFromLinkText(

it seems like extractVersionFromLinkText is correctly replacing - with _ but is not replacing the . with _ in the filename as per the packaging spec.

Reproduction repository: https://github.com/wwuck/renovate-pep420-namespace-package
This repository is an example PEP420 namespace package that contains . in the package name but when built into a .whl file the . are replaced with _ in the filename.

There is currently a bug on pypi.org warehouse software that prevents uploading of packages in this format, but we are using them successfully with our internal private Nexus repository, so I can't link to any test package on https://test.pypi.org for this bug.

pypi/warehouse#10072
pypa/flit#442
pypa/flit#473

Relevant debug logs

Logs

DEBUG: Found no results from datasource that look like a version (repository=RT/test-renovate-bug, dependency=testing.test-namespace-package)
       "result": {"releases": [], "isPrivate": true}

...

DEBUG: packageFiles with updates (repository=RT/test-renovate-bug)
       "config": {
         "pip_requirements": [
           {
             "packageFile": "requirements-dev.txt",
             "deps": [
               {
                 "depName": "testing.test-namespace-package",
                 "currentValue": "==1.0.0",
                 "datasource": "pypi",
                 "currentVersion": "1.0.0",
                 "depIndex": 0,
                 "updates": [],
                 "warnings": [],
                 "versioning": "pep440"
               }
             ],
             "registryUrls": ["https://nexus.example.com/repository/pypi-group/simple"]
           }
         ]
       }

Have you created a minimal reproduction repository?

I have linked to a minimal reproduction repository in the bug description

[wwuck]

There are some packages on pypi.org that still have . in the package name section of the filename, such as https://pypi.org/project/backend.ai/#files, so the current check for wheelPrefix will need to handle all cases of substituting or not substituting ..

From the spec linked above:

Tools consuming wheels must be prepared to accept . (FULL STOP) and uppercase letters, however, as these were allowed by an earlier version of this specification.

[wwuck]

Would it be better to change the check to a regex, or to compare against two different wheelPrefix values (eg. (wheelText.startsWith(wheelPrefixDot) || wheelText.startsWith(wheelPrefixNoDot)))?

The check against two different prefix values seems simpler to me, if I'm not missing any special cases?

I can attempt a PR on this if you think that is a good way to go.

[renovate-release]

🎉 This issue has been resolved in version 32.89.2 🎉

The release is available on:

• GitHub release
• 32.89.2

Your semantic-release bot 📦🚀