Packages installed from local sources appear also subject to dependency confusion attacks

[cp936]

Description

Hello,

I'd like to report what may be a variant of the dependency confusion attack. I reported it to security@python.org and it was suggested I file an issue, with approval to make it public.

In a typical dependency confusion attack, a package installed from one index might get upgraded from another index if the package name is found on the other index and given higher priority. Some public examples I found have exploited conditions where a package exists on an organization's internal index server but the package name is unclaimed on PyPI.

We've found the same attack appears possible against packages installed from standalone, offline sources like local .whl files, Git repositories, or folders with setup.py. The following install methods all appear vulnerable:

pip install path/to/package.whl
pip install git+http://github.com/<username>/<project>
pip install .

When users check for outdated packages with pip install --outdated, pip submits each installed package name to PyPI, including packages installed from local sources. However, locally sourced packages are not guaranteed to exist in PyPI. If a locally installed package's name is unclaimed on PyPI, a malicious actor could attempt to register a package in PyPI with the same name. If successful, pip may end up downloading and installing the malicious package as an upgrade.

The general solution for dependency confusion also works for this variant - users should only submit package names to PyPI for upgrade if the user trusts PyPI to get those packages. In practice though many users just ask pip to upgrade everything from PyPI via something like pip list --outdated | awk 'NR>2 {print $1}' | xargs -n1 pip install -U https://stackoverflow.com/questions/2720014/how-to-upgrade-all-python-packages-with-pip.

Dependency confusion is a known attack but I haven't seen any research explicitly stating packages installed from local sources are affected too. I'm submitting this issue to clarify a user does not appear to need to download a package from an HTTP package index to become vulnerable to dependency confusion. A user could develop a custom package and install it, all on one system, and still appears to be vulnerable if the chosen package name doesn't exist on PyPI.

The security team pointed me to PEP 708 and it seems like it would resolve this issue. Packages installed from local files wouldn't have an associated online package index but I assume the mitigation process would account for that scenario (treat the local file system as an index?).

If you have any questions feel free to let me know.

Cheers,
Carl

Expected behavior

I would expect pip to skip/ignore packages installed from local sources when checking online indexes for available package upgrades.

pip version

24.2

Python version

3.12.7

OS

Ubuntu and Windows 10

How to Reproduce

1. Create a package with a name not currently claimed on PyPI. Alternately download this barebones test package myawesomepkg.zip and unzip it.
2. Install the package (if you unzipped myawesomepkg.zip then cd myawesomepkg && pip install .)
3. Start an HTTP proxy listener.
4. Run pip list --outdated --trusted-host pypi.org --proxy http://<your proxy addr:port>
5. The proxy should reflect that pip requests https://pypi.org/simple/<test package name>
   
6. Pip checked pypi for available upgrade for a package installed offline. If you run pip install myawesomepkg -U --trusted-host pypi.org --proxy http://<your proxy addr:port> and spoof the response, pip does download and install the "new" package.

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[notatallshaw]

pip list --outdated | awk 'NR>2 {print $1}' | xargs -n1 pip install -U

This is a user workflow involving multiple pip commands, there will always be ways in which a user can call multiple commands unsafely, I think there's only so much that can be done to encourage safe behavior.

I would expect pip to skip/ignore packages installed from local sources when checking online indexes for available package upgrades.

A pip maintainer could correct me, but I believe pip currently treats all sources are equally trusted, so if a source is available and has a newer version of that package it is no less trusted than another source.

Perhaps the behavior of pip list --outdated could be changed here, if the provenance of the package is kept (I'm not sure?), but this doesn't really stop dependency confusion attacks as the user have pip list in their workflow instead of pip list --outdated.

And it might be user intentional, perhaps the user had network issues and they manually copied a wheel to their machine to install, but after resolving those network issues they expect to be able to upgrade directly from the index.

And it opens up other questions about similar scenarios, what if a user installed from one index, and then switches to another index, how is pip supposed to know if the new index is more or less trusted?

I’m not against improving the situation here (in fact I suggest pip adopts uv’s index-strategy to help mitigate dependency confision: #8606 (comment)) but I think such changes need to be considered at a design level, not on a case by case scenario level.