pip breaks when pip installs and conda uninstalls the same package

[stas00]

Environment

• pip version: 20.2.4
• conda version: 4.9.2
• Python version: 3.8.6
• OS: linux

Description

This is a bug in pip, but it gets triggered by conda.

When installing a package explicitly it creates an empty file REQUESTED, which conda can't delete and everything blows up.

Expected behavior

Since conda removes explicitly the files in its manifest unlike pip which removes the whole dir, adding new files that are not in manifest breaks things.

How to Reproduce

# cleanup
conda uninstall -y pynvml
pip uninstall -y pynvml

# must install/uninstall things in this exact order
conda install -y -c conda-forge pynvml
pip uninstall -y pynvml
pip install pynvml
conda uninstall -y pynvml

# check: not good:
ls -lt /mnt/nvme1/anaconda3/envs/main-38/lib/python3.8/site-packages/pynvml-8.0.4.dist-info/
total 0
-rw-rw-r-- 1 stas stas 0 Nov 12 22:03 REQUESTED

This file wasn't in conda-forge build, but was added by pip - conda doesn't know to remove it since it removes files explicitly only the ones it installed, but pip removes the dir.

So now the site-packages is broken and we can't use pip anymore:

pip install pynvml
Requirement already satisfied: pynvml in /mnt/nvme1/anaconda3/envs/main-38/lib/python3.8/site-packages (8.0.4)
WARNING: No metadata found in /mnt/nvme1/anaconda3/envs/main-38/lib/python3.8/site-packages
ERROR: Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/mnt/nvme1/anaconda3/envs/main-38/lib/python3.8/site-packages/pynvml-8.0.4.dist-info/METADATA'

"Requirement already satisfied" is bogus since there is just one REQUESTED file that wasn't cleaned up in the dist dir.

this fails too:

pip uninstall -y pynvml
Found existing installation: pynvml 8.0.4
ERROR: Exception:
Traceback (most recent call last):
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 228, in _main
    status = self.run(options, args)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_internal/commands/uninstall.py", line 89, in run
    uninstall_pathset = req.uninstall(
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 685, in uninstall
    uninstalled_pathset = UninstallPathSet.from_dist(dist)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_internal/req/req_uninstall.py", line 535, in from_dist
    for path in uninstallation_paths(dist):
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_internal/req/req_uninstall.py", line 67, in unique
    for item in fn(*args, **kw):
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_internal/req/req_uninstall.py", line 85, in uninstallation_paths
    r = csv.reader(FakeFile(dist.get_metadata_lines('RECORD')))
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1432, in get_metadata_lines
    return yield_lines(self.get_metadata(name))
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1420, in get_metadata
    value = self._get(path)
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1616, in _get
    with open(path, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/nvme1/anaconda3/envs/main-38/lib/python3.8/site-packages/pynvml-8.0.4.dist-info/RECORD'

The only fix at the moment is to manually remove the dir by hand.

I have a package whose build process validates that the conda/pip installs worked correctly and so it happens to hit a similar sequence. But this can easily happen in real life, except install / uninstall commands can be called days apart...

[uranusjr]

Ah, so this is the cause to #8676, eh? We’ve had trouble reproducing this and you’re the first to identify a likely cause. Thanks for filling in the missing piece!

This looks like a Conda bug. pip needs RECORD to know what files to uninstall. If Conda removes the file, it should also remove the .dist-info directory, which pip relies on to discover installed packages. It should not uninstall the package’s metadata half-way.

pip can’t really do anything here since uninstallation cannot be reasonably done without RECORD. IMO this is a Conda bug and should be fixed by Conda.

[pfmoore]

However, note that PEP 376 says that "Third-party installation tools also should not overwrite or delete files that are not in a RECORD file without prompting or warning."

I would interpret that as saying that pip should include REQUESTED in the RECORD file. (The same would apply to INSTALLER, BTW).

I do agree that conda should not respond to conda uninstall xxx by leaving a dist-info directory for xxx present. That's invalid, because with a dist-info, the project is not uninstalled. So there's also a bug in conda, in that it should either abort the uninstall with an error because there are dist-info files that it's not going to remove (so it can't actually remove the package properly), or it should remove the files and prompt or warn as described in PEP 376.

[uranusjr]

I would interpret that as saying that pip should include REQUESTED in the RECORD file. (The same would apply to INSTALLER, BTW).

I believe pip does include them. The problem is Conda does not read RECORD; it maintains its own manifest, and only removes what it thinks need to be removed.

[pradyunsg]

Yea, this is almost definitely a conda bug. Could you file an issue on their tracker and crosslink to here?

[uranusjr]

In the meantime, we can probably implement a safety net in uninstall that errors out with a message like

foo appears to be installed, by its metadata is corrupt, pip doesn’t know what happened and will not touch it.

• If you think foo is installed, try repairing it with --force-reinstall.
• If you think foo is NOT installed, manually remove path/to/foo.dist-info.

[pfmoore]

I believe pip does include them.

Apologies, I didn't check (I misunderstood the OP's report). Yes, it does - so this is definitely a conda bug, if it's not respecting the standard metadata.

[stas00]

Yea, this is almost definitely a conda bug. Could you file an issue on their tracker and crosslink to here?

Are you asking me to do so? Or would it be better for one of you to do that, since you better understand the issue and will know how to present it correctly to the other side.

[pfmoore]

As we're not conda users, we can't really help the conda devs to identify and address the issue. And as you're the one seeing the issue, I'd assume you're motivated to push for a fix, so you would be a better person to create a conda issue. (I don't even know where I'd file an issue against conda!)

[pfmoore]

As a simple bug report, you could simply put

conda does not remove all files listed in the RECORD metadata file when asked to remove a package. This leaves a corrupt distribution present, which other tools such as pip are unable to process.

Add your own example of how to reproduce the issue, and feel free to link back here for more details on why this isn't a pip bug.

[stas00]

Thank you for that additional bit that was missing from my investigation, @pfmoore.

I will file the report and link back here. conda/conda#10357

[pfmoore]

No problem! Thank you for taking the time to dig into this - the "METADATA issue" #8676 has been a long-running problem for us, and it's great to understand what was going on here and finally identify how it needs to be fixed!