index-url extra-index-url install priority order

[roipoussiere]

This is totally related to #5045, but I can not comment on it.

For me this should be re-opened, because it's a security issue.

I provide the package xxx to the private repository my-company.com. Then I gives installation instructions to collaborators, configure CI to install this package, etc.

The problem is if someone add a package with the same name on pypi.org, then users and CI will get this package instead of mine, which is bad: and I want to be sure that the installation will not change.

It's not about naming package: if I name my packe my_company.xxx, a malicious user could chose the exact same name and publish it to pypi, letting my collaborators installing a malware instead of my package.

[pfmoore]

This is how index behaviour and package uniqueness is defined. It's not a security issue. If you don't trust PyPI, you should exclude it altogether from your package index searches.

[roipoussiere]

Yes but Pypi is used to download my project dependencies and I don't want to include all of them in our private repo.

[pfmoore]

Understood - so you're trusting PyPI, for better or worse. I don't have a solution for you, but treating all indexes as equal is by design. So if you want a solution for your use case, you need to describe what you'd want to happen as a new feature, without changing existing behaviour. It can then be considered - but would likely need someone to submit a PR once the approach is agreed.

[uranusjr]

There is a discussion on the philosophy around this design choice: https://discuss.python.org/t/4045

In concrete words, pip is consistently treating pypi.org as a global namespace registry, while your mental model is treating it as a package repository. This is still an actively debated topic, but at this point neither view is “wrong”. To use this feature as pip designs it, you should name-squat on PyPI to get what you want.

[pfmoore]

Philosophical issues to one side, I'd note that devpi fixes this problem, and is available as a solution right now. From the documentation of "index inheritance":

All privately uploaded packages will by default inhibit lookups from pypi, allowing to stay safe from an attacker who could otherwise upload malicious release files to the public PyPI index.

I don't use devpi myself, so I don't know the details, but I would strongly recommend that @roipoussiere and anyone else who feels that they would like "priority order" for indexes in pip, look at whether devpi solves their issue.

(Yes, I know it has the problem of "but it means I have to set up a server in my local environment". I have some sympathy for this, as I work in an environment where installing local servers is frustratingly painful thanks to internal policies and bureaucracy. But conversely, please remember that asking a group of volunteer maintainers to make significant code changes to address an issue that already has a solution doesn't come without cost either...)

[roipoussiere]

Thank you for these detailed responses.

So if you want a solution for your use case, you need to describe what you'd want to happen as a new feature, without changing existing behaviour.

Ok I understand. Here my suggestion:

Create a new pip boolean option prefer-extra, defaulted to False. If set to True, pip will look for the packages in the repositories defined in extra-index-url, in the definition order.

[pfmoore]

-1 (you shouldn't be too surprised to hear...)

It's way too easy for people to set a flag in config and forget they have it set, and then be confused when legitimate upgrades don't work. Or we'd get people wanting to set it in requirements files, but its effect wouldn't be limited to the requirement file. Also, the proposal doesn't explain how this would interact with --find-links.

I don't think you fully understood my reservations - this is not a simple thing to change, and it will take some work to flesh out a viable proposal.

I did a quick test with devpi, and setting up a devpi server with your local packages stored in a private index backed by the default PyPI mirror index does precisely what you want. Given that devpi solves this problem out of the box, I'm going to be very hard to convince that this needs a solution within pip.

[ivanst0]

@roipoussiere, have you considered using this setup?
--index-url <your private feed> --extra-index-url https://pypi.org/simple
I believe this would make pip search for a package in your private feed first and then in the public repository.

[pfmoore]

@ivanst0 It doesn't. The indexes are treated equally.

[jsiverskog]

https://www.bleepingcomputer.com/news/security/researcher-hacks-over-35-tech-firms-in-novel-supply-chain-attack/

[mwarkentin]

Here's the security researchers post on it: https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610

[SaschaKrug]

So, it is proven now, that treating indexes equally is a huge security risk. Are there any plans to changing this behaviour? At least optionally?

Currently we can't use alternative index urls without creating empty packages in pypi, which is not a nice solution.

[pradyunsg]

Current "top running idea" is to... drum roll remove --extra-index-url entirely. See #9612 (comment) for more details.

[bennuttall]

So, it is proven now, that treating indexes equally is a huge security risk

If you're referring to the alex birsan article, the security risk was not due to treating indexes equally. pip did not, at the time, treat indexes equally. pip looks for the "best" file it can find, between all available indexes. That means highest version, tag match (ABI tag, python version tag, platform tag) and only if both indexes provide an equivalent file would it choose between the two. Prior to pip 21, index-url would take precedence in the case of equivalent files available, and since pip 21, it seems that either additional-index-url takes precedence or this choice is arbitrary (see #9610).

If both indexes provide the same set of versions of a package, and PyPI only provides an sdist of the latest version, but the additional index provides a wheel (compatible with your environment e.g. a pure python wheel or matching abi/py/platform tags) then pip will choose the wheel.

If PyPI provides a pure python wheel and so does the additional index, then pip 20 would download it from PyPI but pip 21 downloads it from the additional index.

I raised #9610 to ask for clarification since this change in behaviour wasn't listed in the changelog. IMO it should be deterministic which index takes precedence in the case of equivalent files (and should be documented).

I run a repository which only provides wheels (compiled for raspberry pi). The Raspberry Pi OS is configured to set extra-index-url to this repo. That means all pip requests go pypi.org and piwheels.org and if piwheels provides a wheel, the wheel trumps the sdist from pypi. If we don't have the wheel, pip chooses the sdist from pypi.

I don't see how changing the priority order of indexes or removing the extra-index-url option (#9612) would remove the security risk. The solution surely is to find a way of explicitly installing packages from particular indexes. If you're Microsoft and you have dependencies on internal private projects microsoft-alice, microsoft-bob and microsoft-charlie which let's say between them depend on upstream (PyPI) numpy, scipy and pandas. You need a way of telling pip to install these packages and their requirements but only install the microsoft- packages from the internal private repo.

[vanschelven]

The solution surely is to find a way of explicitly installing packages from particular indexes.

This way already exists (even without --extra-index-url, albeit not elegantly:

pip install -r public-requirements.txt
pip install -r private-requirements.txt --index-url=https://my.private.index/

Although this "solution" probably comes with the significant downside of being mostly broken when the private packages themselves depend on public packages.

[pfmoore]

If you're referring to the alex birsan article, the security risk was not due to treating indexes equally. pip did not, at the time, treat indexes equally. pip looks for the "best" file it can find, between all available indexes.

Specifically, the security risk comes from the fact that pip is selecting the "best" match to the user's request, from multiple sources which are not equally secure. The exploit is to put a higher version of a "private" package in a place that pip looks, but which the attacker has access to.

That's fundamentally down to pip's ability to use multiple indexes. It's got absolutely nothing to do with index priority, in fact - because if foo-666.666.666 is the latest version of a package, it will be chosen regardless of any priorities between indexes, simply because it's the latest version.

There really is no "good" way of securing --extra-index-url if you look at it that way. Allow pip to look in 2 locations and you have to accept that all of your packages are now being served as securely as the least secure index. And the evidence of the "dependency confusion" article is that people simply aren't aware of that reality. So what the pip developers need to decide is whether our responsibility ends with having documented how multiple indexes work, or whether we should view the ability to have multiple indexes as an "attractive nuisance" and remove it to ensure that people aren't tempted to use it in an insecure manner.

The clamour of voices arguing "this is a security flaw", plus the sheer stress on the maintainers that would be involved in arguing that this isn't our problem, suggests that we should remove the feature. But there's no doubt that it would penalise people who use the ability correctly - and it feels wrong to be penalising those people for the sake of the group who didn't properly assess the risks.

The solution surely is to find a way of explicitly installing packages from particular indexes.

This way already exists (even without --extra-index-url, albeit not elegantly:

Tools already exist to do this, as well. There's devpi and simpleindex, and probably others, which allow you to serve whatever explicitly curated subset of PyPI you choose, along with your own private packages, and be in total control of what is available. If you have any sort of business requirement for control over what pip installs, you really ought to be using one of these solutions.

[pradyunsg]

I know there's going to be users chiming in with their opinions here, but this is specifically a question for @pypa/pip-team -- do we want to do the regular 6 month removal plan, or something faster (as our deprecation policy allows that)?

[uranusjr]

I think the regular deprecation period would work, there's no real hurry to this.

One thing I want though, is a replacement of the feature for legitimate usages. One issue with devpi, simpleindex etc. is the index can no loner be static, since the "extra links" nature of the feature means all links much be fetched and merged at runtime. Simply removing the feature kind of feels like "punishing the good guys" to me.

Another thing to consider is --find-links, which has all the same "features" (good and bad), plus being non-standard and having more obscure issues. I can imagine some of the existing --extra-links-url users migrating to it because "it works", which would be the opposite of what we want.

[pfmoore]

do we want to do the regular 6 month removal plan, or something faster

Definitely not faster than that, although my main concern is that I don't think we're ready to even start the process yet. I think we need to be extremely careful here not to let the yelling that it's a "security hole" and so needs to be fixed urgently, push us into rushing a solution before we understand what's best.

I think @uranusjr has some very good points that we need to consider.

For me, the key point here is that pip's behaviour is not hard to describe or understand - pip treats all package sources equally. That's it. The "security problem" here seems to come down to a few things:

1. Some users seem to have a problem understanding that simple rule. That may be down to option naming, or documentation, or simply blind unwillingness on the part of the users to check how the feature works before using it.
2. Users don't understand or appreciate that if you merge different package sources, your trust level on the combined source should be the minimum of the trust levels assigned to the individual sources.
3. People don't seem to be willing to accept that they should not blindly trust PyPI.

In reality, I think it's more about people not thinking about these issues until there's a scare, rather than about any explicit fault here. But when there is a scare, instead of people taking it as a reminder to consider their development processes, they yell at tools like pip demanding a "magic wand" that makes the problem go away again.

In other areas, we are very reluctant to make changes for fear of breaking users' workflows¹. Why are we so willing to remove a feature without any replacement in this case, just because some people have characterised it as a "security problem"? Yes, it is a security problem, but I don't think the case has been made that it's a security problem with pip.

As a strawman, maybe we should drop the behaviour of using PyPI by default, and make all users opt into using PyPI? We could even add a warning to every use of --index-url (including for PyPI) reminding users that it is their responsibility to verify the security of any package source they use.

Of course, I don't actually believe that this is a valid option. But I do think that we're trying to apply a technology solution to a people problem here, and that never goes well :-(

This is almost certainly a case where some UX research would be extremely valuable - getting consensus from a representative group of users on the best approach here seems like the right course of action. Maybe some of the companies who paid out bounties to the researchers who found this issue could fund some research into the best solution? Or require that the bounties already paid out be used to fund such research, rather than just paying people for pressing the panic button?

¹ Look at the fuss over removing the fact that the error given by pip install foo== stopped printing the list of available versions of foo 🙁

[mwarkentin]

@pfmoore I think some stronger guidance from pip on how to handle private packages would be helpful in the docs / user guide. The recommendation to use devpi keeps coming up in these threads, but I'm not seeing anywhere in the docs where this is a recommended pattern. I found one reference to a "private index server" under the "Hash-checking mode" headline:

Hash-checking mode is a labor-saving alternative to running a private index server containing approved packages: it removes the need to upload packages, maintain ACLs, and keep an audit trail (which a VCS gives you on the requirements file for free).

Maybe there could be a section dedicated to handling private and public package installs securely? My gut says that many or most companies doing any significant amount of Python development will have the need for both private and public packages but I only have my own experience to go off of here, agreed that some user research could be helpful here. From our company's perspective, it's preferable not to have to own and operate our own internal package repository since if it goes down it will impact basically everything including local development and deployments.

In our case, we've used Gemfury as a managed solution for several years for private packages (they don't support pull-through caching from pypi), so it appears we may need to find and migrate to an alternative solution (if anyone knows of any managed services which do support a more seamless private/public package configuration I'd be interested to hear!). From their docs:

We use --extra-index-url to allow pip to keep the original Index URL. This allows pip to implicitly install public packages that your private package may depend on.

Do the other tools like pipenv and poetry handle public/private installations more securely? If so perhaps those could be referenced along with devpi or other repository based solutions when you hit these problems that can't be handled by pip on its own?

[pfmoore]

I think some stronger guidance from pip on how to handle private packages would be helpful in the docs / user guide.

That's a very good suggestion. Probably the biggest issue here is that the pip developers don't actually have any experience in these matters (or at least I don't, I can't speak for the others). So while we can offer generalities like "use devpi", we don't really have any "best practices" or recommendations to suggest here.

It would be fantastic if someone who did have expertise in the types of "business critical" environments where these issues are significant could contribute something here in the form of a PR for the documentation.

[uranusjr]

Do the other tools like pipenv and poetry handle public/private installations more securely?

I don't think so. They do make it clearer to the user there is no such feature though.

[mwarkentin]

@uranusjr it sounds like poetry does support priorities: https://python-poetry.org/docs/repositories/

Any custom repository will have precedence over PyPI. If you still want PyPI to be your primary source for your packages you can declare custom repositories as secondary.

It seems like pipenv supports specifying specific repositories for a package as well: https://pipenv-fork.readthedocs.io/en/latest/advanced.html#specifying-package-indexes

If you’d like a specific package to be installed with a specific package index, you can do the following:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[[source]]
url = "http://pypi.home.kennethreitz.org/simple"
verify_ssl = false
name = "home"

[dev-packages]

[packages]
requests = {version="*", index="home"}
maya = {version="*", index="pypi"}
records = "*"

[dhirtysix]

Is there any idea if/when this issue will be fixed?

[pfmoore]

Assuming you mean when will PEP 708 be implemented in pip, it's planned but we don't have an expected date - as it's volunteer effort, the best we can say is that it will happen when someone has the time.

[HarryCaveMan]

This is crazy... Just use pip correctly, if you are really concerned, pin your versions (which you should be doing anyway if you care about the stability of your products), or provide a direct path to the wheel/tgz you want to install IE

pip install https://myindexhost.com/mypackage.1.2.3-py2.py3-none-any.whl
or, if you need extras:
pip install mypackage[extrafeature] @ https://myindexhost.com/mypackage.1.2.3-py2.py3-none-any.whl

If you are in an enterprise setting where security is a major concern, you should not be just blindly using FOSS without any background knowledge. There are plenty of safe ways to use pip, the authors are not at fault because y'all refuse to do the requisite reading to enable yourselves to follow best practices. There is a reason this "CVE" was yanked from NVD.

[ryukinix]

A good ad-hoc way to handle that it's using a package prefix as part of its name. For instance, in our company, we declare and publish the package to our gitlab package registry as: <company>-package-name. For instance, the company name is yandex and the package name is catboost

So:

pip install yandex-catboost

The chance of collision is really low and you can still import the package properly without renaming, import catboost should work.

We have been using this for the last 2 years and it is totally ok without any collision event.

[ericchansen]

I think it's worth reiterating that the only secure and practical enterprise solution is to host your own package index. See https://packaging.python.org/en/latest/guides/hosting-your-own-index for details on how to do so.

[ryukinix]

I think it's worth reiterating that the only secure and practical enterprise solution is to host your own package index. See https://packaging.python.org/en/latest/guides/hosting-your-own-index for details on how to do so.

Yes, of course.

[...] we declare and publish the package to our gitlab package registry

[pergardebrink]

We have been using this for the last 2 years and it is totally ok without any collision event.

@ryukinix, Unintentional collisions are very unlikely if using a company prefix, but you are instead highly likely to be targeted with an intentional collision attack instead (which is exactly what this thread is all about and no good way to protect against with pip out of the box).

Just like mentioned in the original post here:

It's not about naming package: if I name my packe my_company.xxx, a malicious user could chose the exact same name and publish it to pypi, letting my collaborators installing a malware instead of my package.

[jcozar87]

It is like @pergardebrink says.

Totally agree with hosting our own python index. However, using just 1 private index means that we need to copy in our private index all pypi package dependencies, which in case of hosted indices means more money.

If we use extra indices (one for private and one for pypi) there is a security risk, if someone creates a package in pypi with the same name of one of my packages in the private index and release a version like 99, then the resolution would install the malware instead of my private package.

[pergardebrink]

This is crazy... Just use pip correctly, if you are really concerned, pin your versions (which you should be doing anyway if you care about the stability of your products), or provide a direct path to the wheel/tgz you want to install IE

pip install mypackage @ https://myindexhost.com/mypackage.1.2.3-py2.py3-none-any.whl

If you are in an enterprise setting where security is a major concern, you should not be just blindly using FOSS without any background knowledge. There are plenty of safe ways to use pip, the authors are not at fault because y'all refuse to do the requisite reading to enable yourselves to follow best practices. There is a reason this "CVE" was yanked from NVD.

I beg to differ, it's hard to use pip in a safe way. Most/all the other package ecosystems (Go Modules, NuGet, npm/yarn/pnpm, etc) have a way to pin versions by default, and also to lock transitive dependencies. And also have protection against dependency confusion attacks. With pip, you cannot even pin transitive dependencies at all (without treating them as direct dependencies).

The defaults should be safe and not require people to ".. just use pip correctly", it should instead be the default and help the user to do the right thing. Having to specify the url directly to the wheel to be secure is exactly the opposite of that (how would you even manage that, which whl do you specify for different architectures etc?). Or setting up a private repository cloning all PyPi packages needed is not that either.

[HarryCaveMan]

This is crazy... Just use pip correctly, if you are really concerned, pin your versions (which you should be doing anyway if you care about the stability of your products), or provide a direct path to the wheel/tgz you want to install IE
pip install mypackage @ https://myindexhost.com/mypackage.1.2.3-py2.py3-none-any.whl
If you are in an enterprise setting where security is a major concern, you should not be just blindly using FOSS without any background knowledge. There are plenty of safe ways to use pip, the authors are not at fault because y'all refuse to do the requisite reading to enable yourselves to follow best practices. There is a reason this "CVE" was yanked from NVD.

I beg to differ, it's hard to use pip in a safe way. Most/all the other package ecosystems (Go Modules, NuGet, npm/yarn/pnpm, etc) have a way to pin versions by default, and also to lock transitive dependencies. And also have protection against dependency confusion attacks. With pip, you cannot even pin transitive dependencies at all (without treating them as direct dependencies).

The defaults should be safe and not require people to ".. just use pip correctly", it should instead be the default and help the user to do the right thing. Having to specify the url directly to the wheel to be secure is exactly the opposite of that (how would you even manage that, which whl do you specify for different architectures etc?). Or setting up a private repository cloning all PyPi packages needed is not that either.

Pip also has a simple and very straightforward way to pin versions... mypackage==1.2.3, mypackage<1.2.4, mypackage<=1.2.3 would all work to prevent name squatters from slipping in via a higher version. AFAIK, no package manager has provided a feature which reads the user's mind to determine which version of a package they want from a public index. They all require this to be provided in text format one way or another, and any that support no pin at all will grab the latest (highest) version in the absence of a pin. If you are not okay with --extra-index-url working the way it does, there multiple other ways to fetch the package from your private index without using it, so don't. There is nothing hard about providing the URI of the package you want from a private index.

As for transitive deps, unless those dependencies are themselves on a private index, they can't get squatted on, because they already exist on the public index. If they are on a private index, you have to specify them explicitly by full URI, because pip/setuptools does not support using extra-index-url internally for transitive deps, for security reasons ¯\_(ツ)_/¯. What all that means is, if you follow the very simple best practice I've outlined (pinning versions and using full URI for private deps) at the top level of the dependency tree, you won't get transitive deps from an unexpected index. And again, if you care at all about the stability of your products, you should be pinning transitive deps regardless of security concerns because breaking changes are a thing.

[ryukinix]

But not a version from a specific pypi index server from multiple provided. That's something that we really had a headache with and solved based on what I said before. No clear and perfect way of doing this right now. Em seg., 24 de jun. de 2024 20:39, HarryCaveMan ***@***.***> escreveu:

…

This is crazy... Just use pip correctly, if you are really concerned, pin your versions (which you should be doing anyway if you care about the stability of your products), or provide a direct path to the wheel/tgz you want to install IE pip install mypackage @ https://myindexhost.com/mypackage.1.2.3-py2.py3-none-any.whl If you are in an enterprise setting where security is a major concern, you should not be just blindly using FOSS without any background knowledge. There are plenty of safe ways to use pip, the authors are not at fault because y'all refuse to do the requisite reading to enable yourselves to follow best practices. There is a reason this "CVE" was yanked from NVD. I beg to differ, it's hard to use pip in a safe way. Most/all the other package ecosystems (Go Modules, NuGet, npm/yarn/pnpm, etc) have a way to pin versions by default, and also to lock transitive dependencies. And also have protection against dependency confusion attacks. With pip, you cannot even pin transitive dependencies at all (without treating them as direct dependencies). The defaults should be safe and not require people to ".. just use pip correctly", it should instead be the default and help the user to do the right thing. Having to specify the url directly to the wheel to be secure is exactly the opposite of that (how would you even manage that, which whl do you specify for different architectures etc?). Or setting up a private repository cloning all PyPi packages needed is not that either. pip also has a way to pin versions... ==<version> — Reply to this email directly, view it on GitHub <#8606 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB2J57SMNQ3PJPQJTHTVE7TZJCU23AVCNFSM6AAAAABJXP6CBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBXGYZTMMJSGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[HarryCaveMan]

But not a version from a specific pypi index server from multiple provided. That's something that we really had a headache with and solved based on what I said before. No clear and perfect way of doing this right now. Em seg., 24 de jun. de 2024 20:39, HarryCaveMan @.> escreveu:
…
This is crazy... Just use pip correctly, if you are really concerned, pin your versions (which you should be doing anyway if you care about the stability of your products), or provide a direct path to the wheel/tgz you want to install IE pip install mypackage @ https://myindexhost.com/mypackage.1.2.3-py2.py3-none-any.whl If you are in an enterprise setting where security is a major concern, you should not be just blindly using FOSS without any background knowledge. There are plenty of safe ways to use pip, the authors are not at fault because y'all refuse to do the requisite reading to enable yourselves to follow best practices. There is a reason this "CVE" was yanked from NVD. I beg to differ, it's hard to use pip in a safe way. Most/all the other package ecosystems (Go Modules, NuGet, npm/yarn/pnpm, etc) have a way to pin versions by default, and also to lock transitive dependencies. And also have protection against dependency confusion attacks. With pip, you cannot even pin transitive dependencies at all (without treating them as direct dependencies). The defaults should be safe and not require people to ".. just use pip correctly", it should instead be the default and help the user to do the right thing. Having to specify the url directly to the wheel to be secure is exactly the opposite of that (how would you even manage that, which whl do you specify for different architectures etc?). Or setting up a private repository cloning all PyPi packages needed is not that either. pip also has a way to pin versions... == — Reply to this email directly, view it on GitHub <#8606 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2J57SMNQ3PJPQJTHTVE7TZJCU23AVCNFSM6AAAAABJXP6CBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBXGYZTMMJSGY . You are receiving this because you were mentioned.Message ID: @.>

Both of these methods work to do what you describe, because of the way pip works, the specific version and specific index must be included in the URI anyway:

pip install https://myindexhost.com/mypackage.1.2.3-py2.py3-none-any.whl
or, if you need extras:
pip install mypackage[extrafeature] @ https://myindexhost.com/mypackage.1.2.3-py2.py3-none-any.whl

I just used myindexhost.com as a fake example index host and 1.2.3 as a fake example version.

[pfmoore]

Can we drop this discussion, please? It doesn't add anything to the proposal, and if you want to know how to work with pip as it currently behaves, please take that discussion to a more general help forum such as https://discuss.python.org/c/users/7.

If you simply want to complain about pip's current behaviour, then please don't. We have heard all the arguments many times by now, and taken them under consideration. PEP 708 is the community's agreed way forward, and it will be implemented when someone has the time to do so (remember, everyone involved here is a volunteer, working on pip in their free time).

[HarryCaveMan]

Can we drop this discussion, please? It doesn't add anything to the proposal, and if you want to know how to work with pip as it currently behaves, please take that discussion to a more general help forum such as https://discuss.python.org/c/users/7.

If you simply want to complain about pip's current behaviour, then please don't. We have heard all the arguments many times by now, and taken them under consideration. PEP 708 is the community's agreed way forward, and it will be implemented when someone has the time to do so (remember, everyone involved here is a volunteer, working on pip in their free time).

The real issue is that Prisma cloud scans link this issue/thread in their "CVE" PRISMA-2022-0168 (if you can even call it that since it was pulled from NVD and no CNA will own it).

[notatallshaw]

Now uv has been out for a little while and has trying to be implementing generally "safe by default" options I think it's worth looking at what they uv did here: https://docs.astral.sh/uv/pip/compatibility/#packages-that-exist-on-multiple-indexes. The quote describing the default behavior:

When uv searches for a package across multiple indexes, it will iterate over the indexes in order (preferring the --extra-index-url over the default index), and stop searching as soon as it finds a match. This means that if a package exists on multiple indexes, uv will limit its candidate versions to those present in the first index that contains the package.

But they also add a flag --index-strategy which includes a couple of "unsafe" options that more closely match pip's behavior.

IMO, this would be safer for all users even with PEP 708, as it introduces the presumption that a particular package should only be found from a particular index. And the advise to users who have issue with this once PEP 708 is available would to to use that and not an "unsafe" option.

Ofc, someone would need to implement it and there would need to be a transition period changing defaults.