pip reports conflict between identical versions of the same package when installed both as editable package and and dependency of an editable package

[bersbersbers]

Description

Sorry, I cannot do it more concisely than in the title :)

Expected behavior

Installation succeeds without error

pip does not seem to understand that the two instances of "humanize 4.9.1.dev23 (from git+https://github.com/python-humanize/humanize#egg=humanize)" refer to exactly the same package.

pip version

24.0

Python version

3.12.2

OS

Windows 11 23H2

How to Reproduce

pyproject.toml

[project]
name = "Bug"
version = "0"
dependencies = [
    "humanize @ git+https://github.com/python-humanize/humanize#egg=humanize",
]

Then run

pip install -e . -e git+https://github.com/python-humanize/humanize#egg=humanize

Output

INFO: pip is looking at multiple versions of bug to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install bug==0 and humanize 4.9.1.dev23 (from git+https://github.com/python-humanize/humanize#egg=humanize) because these package versions have conflicting dependencies.

The conflict is caused by:
The user requested humanize 4.9.1.dev23 (from git+https://github.com/python-humanize/humanize#egg=humanize)
bug 0 depends on humanize 4.9.1.dev23 (from git+https://github.com/python-humanize/humanize#egg=humanize)

To fix this you could try to:

1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[bersbersbers]

(This may be a duplicate of #10216)

[pfmoore]

The problem is that if we were to install humanize from the URL you give, that would work now. But suppose someone updates humanize in a way that breaks compatibility with your bug package. The editable install means that the code change would be picked up and your environment would be broken.

In general, you shouldn't depend on code at a "changeable" URL like a git branch head, but you should rather depend on a specific commit. That avoids your dependencies breaking unexpectedly. It's hard to say more with only an artificial example to go on. But the basic answer is that "you shouldn't be doing this". Either you should be avoiding dependencies on URLs that don't point to a fixed commit, or you shouldn't be using editable installs.

Making this "work" (in some sense) would be a rather significant change to how pip models dependencies, and I don't think it's something we'd want to do just to support a usage which is at best questionable.

[bersbersbers]

The problem is that if we were to install humanize from the URL you give, that would work now. But suppose someone updates humanize in a way that breaks compatibility with your bug package. The editable install means that the code change would be picked up and your environment would be broken.

I agree - but isn't that the point of editable install from Git?

In general, you shouldn't depend on code at a "changeable" URL like a git branch head, but you should rather depend on a specific commit. That avoids your dependencies breaking unexpectedly. It's hard to say more with only an artificial example to go on. But the basic answer is that "you shouldn't be doing this". Either you should be avoiding dependencies on URLs that don't point to a fixed commit, or you shouldn't be using editable installs.

Point taken, but the URL being "changeable" is not the issue here, I believe. I can reproduce the same behavior with more specific URLs:

pyproject.toml

[project]
name = "Bug"
version = "0"
dependencies = [
    "humanize @ git+https://github.com/python-humanize/humanize@b1e5d43c6fd44dbaf0ad86339f63bfa04982d707#egg=humanize",
]

and

pip install -e . -e git+https://github.com/python-humanize/humanize@b1e5d43c6fd44dbaf0ad86339f63bfa04982d707#egg=humanize

Making this "work" (in some sense) would be a rather significant change to how pip models dependencies

Accepted. In that case, I think the error message might use improvements.

I don't think it's something we'd want to do just to support a usage which is at best questionable.

(I am not sure that argument still hold in light of commit-based URLs.) But again, if pip does not support that, it could be make more transparent to the user. I think that is my main point.

[pfmoore]

In that case, I think the error message might use improvements.

Sure. In which case, a PR would be welcomed.

[bersbersbers]

One more comment now that I have read #10216 (comment), where you wrote "two repositories": my expectation was that pip recognizes the two repositories being the same, and applying -e from the command-line override to the to-be-installed dependency. Hence installing humanize only once, in editable mode, which should satisfy "both" requirements (the pyproject.toml one and the command-line one).

[bersbersbers]

(And I think #11899 would give me exactly that.)

[pfmoore]

Yeah, there's a lot of weird interactions here between different features that were never really intended to be used together. If we had a clean slate, a lot of these things would simply never have been allowed - but it's too late for that now as people are using them, and we have to do the best we can to find sane and maintainable solutions...

[dimbleby]

Presumably duplicate #12502, #12520

Did something change or did everyone just decide this was a good time to try this?

Or perhaps folk have just got worse at looking for duplicates...!