Don't fail uninstallation if easy-install.pth doesn't exist

[deveshks]

Fixes and Closes #7856

I have moved the raise UninstallationError exception from the constructor of UninstallPathEntries to the start of the remove method, and I am logging a warning and returning from the remove method if the file does not exist

[deveshks]

Hi @uranusjr and @pradyunsg

Please take a look at the PR whenever you can. Also will there be a unit test which would need to be added in https://github.com/pypa/pip/blob/master/tests/functional/test_uninstall.py to ensure nothing else is affected?

If yes, I would assume the test would be:

1. Create a dummy package and install in it editable mode
2. Manually remove the file easy-install.pth
3. Attempt to uninstall the editable package. The uninstall should pass successfully

Which existing test can I refer to in test_uninstall.py in order to implement such a test

[uranusjr]

The implementation looks good. A test would be needed, and test_uninstall.py is definitely where is should go. Your test structure sounds like a good idea. Sound thoughts:

• You can use the script fixture to install FSPkg. See the beginning of test_uninstall_editable_and_pip_install to get an idea how you can do that.
• There should be a warning message when the uninstall call succeeds. See the last statement in test_basic_uninstall_distutils to get an idea how this can be checked.
• Use an pip list --format=json call to make sure the package is uninstalled (despite the missing file). Various tests in test_uninstall.py has a similar check you can copy.

[uranusjr]

Also, you will need to provide some description in the news fragmemt file. One simple sentence describing the issue you fixed would be enough. The line would end up in pip’s release notes when the fix is included in a new version.

[deveshks]

Hi @uranusjr,

Thank you for the suggestion. I wrote the following test function to try the logic out but I am facing an issue.

def test_uninstall_editable_and_pip_install_and_easy_install_remove(script, data):

    script.environ['SETUPTOOLS_SYS_PATH_TECHNIQUE'] = 'raw'
    pkg_path = data.packages.joinpath("FSPkg")
    script.pip('install', '-e', '.',
               expect_stderr=True, cwd=pkg_path)
    # ensure both are installed with --ignore-installed:
    script.pip('install', '--ignore-installed', '.',
               expect_stderr=True, cwd=pkg_path, use_module=False)
    list_result = script.pip('list', '--format=json')
    assert {"name": "FSPkg", "version": "0.1.dev0"} \
           in json.loads(list_result.stdout)

    # Remove easy-install.pth
    easy_install_pth = join(script.site_packages_path, 'easy-install.pth')
    os.remove(easy_install_pth)

    # Uninstall both develop and install
    uninstall = script.pip('uninstall', 'FSPkg', '-y')

I am able to find easy-install.pth inside script.site_packages_path, but when I remove the file using os.remove and then run the uninstall command, I get the following error:

------------------ Captured stdout call ---------------------
Script result: python -m pip uninstall FSPkg -y
  return code: 1
-- stderr: --------------------
/private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pytest-of-devesh/pytest-1/test_uninstall_editable_and_pi0/workspace/venv/bin/python: No module named pip

==== short test summary info ===
FAIL tests/functional/test_uninstall.py::test_uninstall_editable_and_pip_install_and_easy_install_remove

I saw that the contents of easy-install.pth look as follows:

/private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pytest-of-devesh/pytest-1/setuptools0/install
/private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pytest-of-devesh/pytest-1/pip0/pip/src
/private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pytest-of-devesh/pytest-1/test_uninstall_editable_and_pi0/data/packages/FSPkg

So when I remove easy-install.pth as part of the unit test, pytest is unable to find pip since I think it tries to look it up in easy-install.pth but can't find it.

Is there any other way to install FSPkg instead of script.pip('install', '-e', '.', expect_stderr=True, cwd=pkg_path) which can avoid such an issue?

[pradyunsg]

@deveshks and I discussed about this over IRC; which resulted in filing of #7893.

[deveshks]

Hi @uranusjr ,

Could you please the queries at #7891 (comment) had about how to write the test case for this PR.

I tried it the way you mentioned but came across some difficulties.

[uranusjr]

Ah, because pip itself is also installed as an editable install for tests… This sucks.

One way I can think of to avoid this problem is to rename easy-install.pth to something else for the test (or maybe for the whole test suite). Python loads all .pth files when it configures sys.path, so the file can be named whatever as long as it is <something>.pth. This can be done with a context manager (with rename_easy_install_path('pip-test.pth')) if we only want to change the name for one test.

Another way would be to use python path/to/pip/source/code to invoke pip instead. But I feel less confortable about it since this might cause more consequences.

[deveshks]

Hi @uranusjr

Thanks for the suggestion. I tried two possible scenarios of renaming easy-install.pth to pip-test.pth

1. In the first scenario, I performed the renaming before I use the script fixture to install FSPkg. I saw that the FSPkg was not listed on pip-test.pth, but list --format=json listed out FSPkg after I performed install. But when running the uninstall FSPkg -y command didn't throw any warnings about a missing file

2. In the second scenario, I performed the renaming after I use the script fixture to install FSPkg. I saw that FSPkg was listed in easy-install.pth and list --format=json listed out FSPkg after I performed the install. But when running uninstall FSPkg -y, it threw the warning about the missing .pth file, the uninstall didn't succeed and list --format=json listed out FSPkg

I am unsure when you said to rename the file, but I tied both possible scenarios and neither worked. Could you please expand on how exactly do I achieve your suggestion.

Also I observed that the install and uninstall are done twice in a uninstall test, once for develop and once for install. Do I do the same for this test, or I perform install and uninstall only once?

[uranusjr]

pip always writes to easy-install.pth, creating it if necessary. So in scenario 1, after installing FSPkg, you would have both easy-install.pth and pip-test.pth, thus uninstallation succeeds. Scenario 2 is closer to what we’re looking for, but we don’t want pip install -e FSPkg to edit a pre-existing easy-install.pth (since we can’t undo the edit easily).

I think the correct test flow would be

1. Rename easy-install.pth to pip-test.pth
2. Install FSPkg
3. Rename easy-install.pth generated by the previous step to (say) pip-test-fspkg.pth
4. pip list --format=json should successfully list FSPkg
5. pip uninstall FSPkg should fail because it can’t locate easy-install.pth (we will fix this part to let it skip the missing easy-install.pth)
6. Delete pip-test-fspkg.pth (which is left behind because pip uninstall does not know it should look for it).
7. Rename pip-test.pth back to easy-install.pth (restoring the state before the test).

Does this make sense?

[deveshks]

Thanks @uranusjr ,

Let me create a test case based on what you said and create a commit for it.

But I had a question. After doing pip uninstall FSPkg, do I just check that the warning message is present in stderr, or do I also check that FSPkg is not present in list --format=json, because I do see that the warning message is present, but FSPkg is also present in list --format=json, as listed in scenario 2 above.

Is that the expected behavior?

[uranusjr]

The entry will still be shown as installed after uninstallation because Python can still find the metadata in pip-test-fspkg.pth. So I would check for the existence of the warning first, and check for FSPkg after deleting the pth file.

[deveshks]

Thanks for the suggestion @uranusjr Your explanation makes sense since pip will try to find the package in all pth files, including the one I renamed earlier.

I have fixed that issue and added a commit for the same. Please take a look whenever possible :)

[deveshks]

Pinging @uranusjr to take a look at the updated PR addressing suggestions made :)

[uranusjr]

One extra thing I realised when reading the news entry. The implementation and test LGTM!

[uranusjr]

In case you missed it, I also had a quetion about testing .egg-link removal in the previous message.

[deveshks]

Hi @uranusjr

Actually I thought that you were referring for fixing the news entry only. Let me address your other question about asserting the removal of .egg-link

[uranusjr]

Thanks for the hard work @deveshks!

[deveshks]

Thanks for all the review comments which made this easier @uranusjr :)
I will wait for it to get merged now.

[pradyunsg]

Wheee! This is great!

Thanks for all the hard work here @deveshks and @uranusjr! ^>^