Pip-compile upgrades a package even it doesn't exist in a requrements.*

[atugushev]

Pip-compile should raise an error while trying to upgrade a package which doesn't exist in a requirements.in.

Environment Versions

1. OS Type: any
2. Python version: any
3. pip version: any
4. pip-tools version: any

Steps to replicate

1. echo "click" > requirements.in
2. pip-compile
3. pip-compile -P django

Expected result

Raise a error/warning or skip non-existing package.

Actual result

#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile -P pytz
#
click==7.0
django==1.11.20
pytz==2018.9

[pawciobiel]

It's not clear if -P should or shouldn't work with SRC *in and/or DST *.txt requirements files. Would it make sense if pip-compile -P django requirements.in raise error but not pip-compile -P django ?

[atugushev]

Hello @pawciobiel,

if pip-compile -P django requirements.in raise error but not pip-compile -P django

What's the difference between those commands? Why they should have different behavior?

[AndydeCleyre]

IMO it would be best for this to skip non-existing packages with a warning, rather than an error. That way I could safely loop through different .in files, upgrading a specified package if and only if it exists, without needing to check if a failure is a "real" problem or just an inappropriate upgrade attempt.

FWIW, I'm using these Zsh functions to wrap pip-compile -P and pip-compile -U:

# recompile *requirements.txt with upgraded versions of all or specified packages (upgrade)
pipu () {  # [req...]
    local reqs=($@)
    for reqsin in *requirements.in(N); do
        print -rP "%F{cyan}> upgrading ${reqsin:r}.txt <- $reqsin . . .%f"
        if [[ $# -gt 0 ]]; then
            pip-compile --no-header ${${@/*/-P}:^reqs} $reqsin 2>&1 | hpype
            pipc $reqsin  # can remove if https://github.com/jazzband/pip-tools/issues/759 gets fixed
        else
            pip-compile --no-header -U $reqsin 2>&1 | hpype
        fi
    done
}
# upgrade with hashes
pipuh () {  # [req...]
    local reqs=($@)
    for reqsin in *requirements.in(N); do
        print -rP "%F{cyan}> upgrading ${reqsin:r}.txt <- $reqsin . . .%f"
        if [[ $# -gt 0 ]]; then
            pip-compile --no-header --generate-hashes ${${@/*/-P}:^reqs} $reqsin 2>&1 | hpype
            pipch $reqsin  # can remove if https://github.com/jazzband/pip-tools/issues/759 gets fixed
        else
            pip-compile --no-header -U --generate-hashes $reqsin 2>&1 | hpype
        fi
    done
}

[atugushev]

@AndydeCleyre, probably, warning+skip is better than an error indeed. Thanks for the suggestion!

[tuukkamustonen]

I second the need for not raising error. And in my case (#950), I wouldn't want even warning.

Maybe --upgrade-package and --upgrade-only-package or something?

[AndydeCleyre]

@atugushev I've implemented a very small change that seems to do "the right thing," with a simple test. I don't know that there's a strong case for errors or warnings in this case, so there are none raised in my PR.

You mentioned in your WIP PR that there was a complication with "upgrading subdependencies" -- can you elaborate with an example to demonstrate the expectation?

[atugushev]

@AndydeCleyre

You mentioned in your WIP PR that there was a complication with "upgrading subdependencies" -- can you elaborate with an example to demonstrate the expectation?

There were a few edge cases with sub-deps. Some of them, probably, wouldn't relevant if we don't want to raise a error/warning. I'll elaborate on these cases in your PR.

[atugushev]

FTR, I've updated the "Expected result" in the issue description, since both variants fine by me.