[6.5.0] Packages needlessly uninstalled during sync due to non-canonicalized names (dots vs dashes)

[Rogdham]

Packages with dots in them are not compiled to the same file in version 6.4.0 and 6.5.0, where dots are replaced with dashes.

E.g. zope.event becomes zope-event

Environment Versions

1. OS Type: Arch Linux
2. Python version: Python 3.10.2
3. pip version: pip 20.3.4 from /usr/lib/python3.10/site-packages/pip (python 3.10)
4. pip-tools version: pip-compile, version 6.5.0

Steps to replicate

On version 6.4.0

• requirements.in

pip-tools==6.4.0
gevent==21.12.0

• requirements.txt (after pip-compile)

#
# This file is autogenerated by pip-compile with python 3.10
# To update, run:
#
#    pip-compile
#
click==8.0.3
    # via pip-tools
gevent==21.12.0
    # via -r requirements.in
greenlet==1.1.2
    # via gevent
pep517==0.12.0
    # via pip-tools
pip-tools==6.4.0
    # via -r requirements.in
tomli==2.0.0
    # via pep517
wheel==0.37.1
    # via pip-tools
zope.event==4.5.0
    # via gevent
zope.interface==5.4.0
    # via gevent

# The following packages are considered to be unsafe in a requirements file:
# pip
# setuptools

On version 6.5.0

• requirements.in

pip-tools==6.5.0
gevent==21.12.0

• requirements.txt (after pip-compile)

#
# This file is autogenerated by pip-compile with python 3.10
# To update, run:
#
#    pip-compile
#
click==8.0.3
    # via pip-tools
gevent==21.12.0
    # via -r requirements.in
greenlet==1.1.2
    # via gevent
pep517==0.12.0
    # via pip-tools
pip-tools==6.5.0
    # via -r requirements.in
tomli==2.0.0
    # via pep517
wheel==0.37.1
    # via pip-tools
zope-event==4.5.0
    # via gevent
zope-interface==5.4.0
    # via gevent

# The following packages are considered to be unsafe in a requirements file:
# pip
# setuptools

Expected result

• The zope.event package being listed as zope.event in generated requirements.txt
• Running pip-sync twice in a raw will not do anything on second run (except printing Everything up-to-date

Actual result

• The zope.event package has been renamed to zope-event in generated requirements.txt
• Running pip-sync does the following even if nothing has changed (i.e. it does so in two consecutive runs)

Found existing installation: zope.event 4.5.0
Uninstalling zope.event-4.5.0:
  Successfully uninstalled zope.event-4.5.0
Found existing installation: zope.interface 5.4.0
Uninstalling zope.interface-5.4.0:
  Successfully uninstalled zope.interface-5.4.0
Collecting zope-event==4.5.0
  Using cached zope.event-4.5.0-py2.py3-none-any.whl (6.8 kB)
Collecting zope-interface==5.4.0
  Using cached zope.interface-5.4.0-cp310-cp310-linux_x86_64.whl
Requirement already satisfied: setuptools in ./env/lib/python3.10/site-packages (from zope-event==4.5.0->-r /tmp/tmpcg_uqvf4 (line 1)) (58.1.0)
Installing collected packages: zope-interface, zope-event
Successfully installed zope-event-4.5.0 zope-interface-5.4.0
WARNING: You are using pip version 21.2.4; however, version 22.0.3 is available.
You should consider upgrading via the '/tmp/foo/env/bin/python -m pip install --upgrade pip' command.

[webknjaz]

This is expected and explained at #1567 (comment)

[AndydeCleyre]

@webknjaz

This is introduced there but can (I think) easily be resolved by using canonicalize_name in key_from_req. I have done so just now in the process of merging the new release back into #1329, and I think this particular valid issue is resolved in that PR.

[AndydeCleyre]

Oh to be clear, I'm not referring to the issue as titled, but this part

Running pip-sync does the following even if nothing has changed (i.e. it does so in two consecutive runs)

I may re-title this.

[AndydeCleyre]

The fix specifically, in key_from_req:

-    key = key.replace("_", "-").lower()
-    return key
+    return str(canonicalize_name(key))

[Rogdham]

This is expected and explained at #1567 (comment)

Thank you for the pointer. I hope that having created this issue will point users that are searching for keywords like dot or dash will arrive here now! 😃

---

I'm confirming that #1329 is fixing the pip-sync consecutive runs issue.

The . are still converted into - but you know better than I do about this one, so it's ok with me.

Many thanks!

[di]

Given how large #1329 is, I think it probably makes sense to pull 1eac955 out into a separate PR (and add a test 😉) in order to get it out sooner than later.

[AndydeCleyre]

I'll take it out and rebase if someone merges the tiny change alone, it should have been part of the recent merge.

If no one does by the time I get back to this, I'll submit a small PR myself.

[AndydeCleyre]

Please see the small PR at #1572.

[atugushev]

Released as part of pip-tools v6.5.1.

[Rogdham]

Confirmed fixed with v6.5.1, thank you! 🎉

[alexlitvinenko]

I can still reproduce.

[Rogdham]

@alexlitvinenko this is normal: . are replaced to - since v6.5.0.

What was not normal and is now fixed, was that running pip-sync twice in a raw would try to perform some tasks instead of displaying Everything up-to-date.

[alexlitvinenko]

thanks @Rogdham
I've raised this question because Pycharm had raised it.

But, I see that pip correctly interprets zope-interface in requirements.txt.

Please, consider this as just a warning.

[di]

That would be a bug in Pycharm I think.

[AndydeCleyre]

FWIW: the Pycharm bug tracker