--keep-outdated while switching dependency git location results in conflicting Pipfile and lock

[nicholasserra]

Issue description

Switching a dependency from a fork (eg https://github.com/nicholasserra/twine) to the source (eg https://github.com/pypa/twine) with the --keep-outdated flag results in conflicting Pipfile and Pipfile.lock

Expected result

Expected both the Pipfile and Pipfile.lock to reference https://github.com/pypa/twine

Actual result

Pipfile references the new source given (https://github.com/pypa/twine) but the Pipfile.lock still references the old fork url and commit (https://github.com/nicholasserra/twine).

Steps to replicate

• Starting with a blank project, no Pipfile or lock.
• Install a github dependency at a commit from a fork

pipenv install -e git+https://github.com/nicholasserra/twine.git@668f4db44d2965d9009308d96e13776069d050db#egg=twine

• Pipfile shows the following:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
twine = {editable = true, ref = "668f4db44d2965d9009308d96e13776069d050db", git = "https://github.com/nicholasserra/twine.git"}

• Pipfile.lock shows the following (clutter removed)

{
    "_meta": {
        "hash": {
            "sha256": "027966d540aa935f91d54fd1141002805ec638da730d755661de05b233ca879e"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
    ...
        "twine": {
            "editable": true,
            "git": "https://github.com/nicholasserra/twine.git",
            "ref": "668f4db44d2965d9009308d96e13776069d050db"
        },
    ...
    },
    "develop": {}
}

• Update the dependency to reference the source github url at a commit:

pipenv install -e git+https://github.com/pypa/twine.git@8f5e5d6d42d582ef3ea6ef07da277e0cabd22fd2#egg=twine --keep-outdated

• The Pipfile correctly gets updated:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
twine = {editable = true, ref = "8f5e5d6d42d582ef3ea6ef07da277e0cabd22fd2", git = "https://github.com/pypa/twine.git"}

• The Pipfile.lock still references the old url and commit:

{
    "_meta": {
        "hash": {
            "sha256": "dc1ca95272a1bbd8a7fe0c22e1f5e9a7b4fab405c67d32b96a3f9534b833fbaf"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
    ...
        "twine": {
            "editable": true,
            "git": "https://github.com/nicholasserra/twine.git",
            "ref": "668f4db44d2965d9009308d96e13776069d050db"
        },
    ...
    },
    "develop": {}
}

---

$ pipenv --support

Pipenv version: '2022.6.7'

Pipenv location: '/Users/nick/Code/pipenv-bug/venv/lib/python3.8/site-packages/pipenv'

Python location: '/Users/nick/Code/pipenv-bug/venv/bin/python'

Python installations found:

• 3.8.2: /Users/nick/Code/pipenv-bug/venv/bin/python3
• 3.8.2: /Users/nick/Code/pipenv-bug/venv/bin/python
• 3.8.2: /Users/nick/Code/pipenv-bug/venv/bin/python3
• 3.8.2: /Users/nick/Code/pipenv-bug/venv/bin/python
• 3.8.2: /Users/nick/.pyenv/versions/3.8.2/bin/python3
• 3.7.3: /usr/bin/python3
• 3.7.0: /Users/nick/.pyenv/versions/3.7.0/bin/python3
• 3.6.1: /Users/nick/.pyenv/versions/3.6.1/bin/python3
• 3.4.3: /usr/local/bin/python3
• 3.4.3: /usr/local/bin/python3.4m
• 3.4.3: /usr/local/bin/python3.4
• 2.7.16: /usr/bin/python
• 2.7.16: /usr/bin/python2
• 2.7.16: /usr/bin/python2.7
• 2.7.13: /usr/local/bin/python
• 2.7.13: /usr/local/bin/python2
• 2.7.13: /usr/local/bin/python2.7
• 2.7.13: /Users/nick/.pyenv/versions/2.7.13/bin/python
• 2.7.10: /Users/nick/.pyenv/versions/2.7.10/bin/python

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.8.2',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '19.6.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 19.6.0: Mon Apr 18 21:50:40 PDT '
                     '2022; root:xnu-6153.141.62~1/RELEASE_X86_64',
 'python_full_version': '3.8.2',
 'python_version': '3.8',
 'sys_platform': 'darwin'}

System environment variables:

• MANPATH
• rvm_bin_path
• TERM_PROGRAM
• NVM_CD_FLAGS
• GEM_HOME
• TERM
• SHELL
• IRBRC
• TMPDIR
• NVM_PATH
• TERM_PROGRAM_VERSION
• OLDPWD
• MY_RUBY_HOME
• TERM_SESSION_ID
• NVM_DIR
• USER
• _system_type
• rvm_path
• SSH_AUTH_SOCK
• __CF_USER_TEXT_ENCODING
• VIRTUAL_ENV
• rvm_prefix
• PATH
• NVM_NODEJS_ORG_MIRROR
• PWD
• LANG
• ITERM_PROFILE
• _system_arch
• XPC_FLAGS
• PS1
• PYTHONDONTWRITEBYTECODE
• _system_version
• XPC_SERVICE_NAME
• rvm_version
• PYENV_SHELL
• SHLVL
• HOME
• COLORFGBG
• LC_TERMINAL_VERSION
• ITERM_SESSION_ID
• LOGNAME
• GEM_PATH
• GOPATH
• NVM_BIN
• NVM_IOJS_ORG_MIRROR
• LC_TERMINAL
• RUBY_VERSION
• _system_name
• COLORTERM
• _
• PIP_SHIMS_BASE_MODULE
• PIP_DISABLE_PIP_VERSION_CHECK
• PIP_PYTHON_PATH
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /Users/nick/Code/pipenv-bug/venv/bin:/Users/nick/.pyenv/shims:/Users/nick/.nvm/versions/node/v10.15.1/bin:/Applications/Postgres.app/Contents/Versions/9.4/bin:/Users/nick/.rvm/gems/ruby-2.3.0/bin:/Users/nick/.rvm/gems/ruby-2.3.0@global/bin:/Users/nick/.rvm/rubies/ruby-2.3.0/bin:/usr/local/heroku/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/usr/local/share/dotnet:/Applications/Wireshark.app/Contents/MacOS:/usr/local/git/bin:/Users/nick/.rvm/bin:/Users/nick/.rvm/bin:/Users/nick/bin
• SHELL: /bin/bash
• LANG: en_US.UTF-8
• PWD: /Users/nick/Code/pipenv-bug
• VIRTUAL_ENV: /Users/nick/Code/pipenv-bug/venv

---

Contents of Pipfile ('/Users/nick/Code/pipenv-bug/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
twine = {editable = true, ref = "8f5e5d6d42d582ef3ea6ef07da277e0cabd22fd2", git = "https://github.com/pypa/twine.git"}

[dev-packages]

[requires]
python_version = "3.8"

Contents of Pipfile.lock ('/Users/nick/Code/pipenv-bug/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "dc1ca95272a1bbd8a7fe0c22e1f5e9a7b4fab405c67d32b96a3f9534b833fbaf"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "bleach": {
            "hashes": [
                "sha256:085f7f33c15bd408dd9b17a4ad77c577db66d76203e5984b1bd59baeee948b2a",
                "sha256:0d03255c47eb9bd2f26aa9bb7f2107732e7e8fe195ca2f64709fcf3b0a4a085c"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==5.0.1"
        },
        "certifi": {
            "hashes": [
                "sha256:84c85a9078b11105f04f3036a9482ae10e4621616db313fe045dd24743a0820d",
                "sha256:fe86415d55e84719d75f8b69414f6438ac3547d2078ab91b67e779ef69378412"
            ],
            "markers": "python_version >= '3.6'",
            "version": "==2022.6.15"
        },
        "charset-normalizer": {
            "hashes": [
                "sha256:2857e29ff0d34db842cd7ca3230549d1a697f96ee6d3fb071cfa6c7393832597",
                "sha256:6881edbebdb17b39b4eaaa821b438bf6eddffb4468cf344f09f89def34a8b1df"
            ],
            "markers": "python_full_version >= '3.5.0'",
            "version": "==2.0.12"
        },
        "colorama": {
            "hashes": [
                "sha256:854bf444933e37f5824ae7bfc1e98d5bce2ebe4160d46b5edf346a89358e99da",
                "sha256:e6c6b4334fc50988a639d9b98aa429a0b57da6e17b9a44f0451f930b6967b7a4"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==0.4.5"
        },
        "commonmark": {
            "hashes": [
                "sha256:452f9dc859be7f06631ddcb328b6919c67984aca654e5fefb3914d54691aed60",
                "sha256:da2f38c92590f83de410ba1a3cbceafbc74fee9def35f9251ba9a971d6d66fd9"
            ],
            "version": "==0.9.1"
        },
        "docutils": {
            "hashes": [
                "sha256:23010f129180089fbcd3bc08cfefccb3b890b0050e1ca00c867036e9d161b98c",
                "sha256:679987caf361a7539d76e584cbeddc311e3aee937877c87346f31debc63e9d06"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==0.18.1"
        },
        "idna": {
            "hashes": [
                "sha256:84d9dd047ffa80596e0f246e2eab0b391788b0503584e8945f2368256d2735ff",
                "sha256:9d643ff0a55b762d5cdb124b8eaa99c66322e2157b69160bc32796e824360e6d"
            ],
            "markers": "python_full_version >= '3.5.0'",
            "version": "==3.3"
        },
        "importlib-metadata": {
            "hashes": [
                "sha256:637245b8bab2b6502fcbc752cc4b7a6f6243bb02b31c5c26156ad103d3d45670",
                "sha256:7401a975809ea1fdc658c3aa4f78cc2195a0e019c5cbc4c06122884e9ae80c23"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==4.12.0"
        },
        "keyring": {
            "hashes": [
                "sha256:372ff2fc43ab779e3f87911c26e6c7acc8bb440cbd82683e383ca37594cb0617",
                "sha256:3ac00c26e4c93739e19103091a9986a9f79665a78cf15a4df1dba7ea9ac8da2f"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==23.6.0"
        },
        "pkginfo": {
            "hashes": [
                "sha256:848865108ec99d4901b2f7e84058b6e7660aae8ae10164e015a6dcf5b242a594",
                "sha256:a84da4318dd86f870a9447a8c98340aa06216bfc6f2b7bdc4b8766984ae1867c"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'",
            "version": "==1.8.3"
        },
        "pygments": {
            "hashes": [
                "sha256:5eb116118f9612ff1ee89ac96437bb6b49e8f04d8a13b514ba26f620208e26eb",
                "sha256:dc9c10fb40944260f6ed4c688ece0cd2048414940f1cea51b8b226318411c519"
            ],
            "markers": "python_version >= '3.6'",
            "version": "==2.12.0"
        },
        "readme-renderer": {
            "hashes": [
                "sha256:73b84905d091c31f36e50b4ae05ae2acead661f6a09a9abb4df7d2ddcdb6a698",
                "sha256:a727999acfc222fc21d82a12ed48c957c4989785e5865807c65a487d21677497"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==35.0"
        },
        "requests": {
            "hashes": [
                "sha256:bc7861137fbce630f17b03d3ad02ad0bf978c844f3536d0edda6499dafce2b6f",
                "sha256:d568723a7ebd25875d8d1eaf5dfa068cd2fc8194b2e483d7b1f7c81918dbec6b"
            ],
            "markers": "python_version >= '3.7' and python_version < '4'",
            "version": "==2.28.0"
        },
        "requests-toolbelt": {
            "hashes": [
                "sha256:380606e1d10dc85c3bd47bf5a6095f815ec007be7a8b69c878507068df059e6f",
                "sha256:968089d4584ad4ad7c171454f0a5c6dac23971e9472521ea3b6d49d610aa6fc0"
            ],
            "version": "==0.9.1"
        },
        "rfc3986": {
            "hashes": [
                "sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd",
                "sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==2.0.0"
        },
        "rich": {
            "hashes": [
                "sha256:4c586de507202505346f3e32d1363eb9ed6932f0c2f63184dea88983ff4971e2",
                "sha256:d2bbd99c320a2532ac71ff6a3164867884357da3e3301f0240090c5d2fdac7ec"
            ],
            "markers": "python_version < '4' and python_full_version >= '3.6.3'",
            "version": "==12.4.4"
        },
        "setuptools": {
            "hashes": [
                "sha256:990a4f7861b31532871ab72331e755b5f14efbe52d336ea7f6118144dd478741",
                "sha256:c1848f654aea2e3526d17fc3ce6aeaa5e7e24e66e645b5be2171f3f6b4e5a178"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==62.6.0"
        },
        "six": {
            "hashes": [
                "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926",
                "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==1.16.0"
        },
        "tqdm": {
            "hashes": [
                "sha256:40be55d30e200777a307a7585aee69e4eabb46b4ec6a4b4a5f2d9f11e7d5408d",
                "sha256:74a2cdefe14d11442cedf3ba4e21a3b84ff9a2dbdc6cfae2c34addb2a14a5ea6"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==4.64.0"
        },
        "twine": {
            "editable": true,
            "git": "https://github.com/nicholasserra/twine.git",
            "ref": "668f4db44d2965d9009308d96e13776069d050db"
        },
        "typing-extensions": {
            "hashes": [
                "sha256:6657594ee297170d19f67d55c05852a874e7eb634f4f753dbd667855e07c1708",
                "sha256:f1c24655a0da0d1b67f07e17a5e6b2a105894e6824b92096378bb3668ef02376"
            ],
            "markers": "python_version < '3.9'",
            "version": "==4.2.0"
        },
        "urllib3": {
            "hashes": [
                "sha256:44ece4d53fb1706f667c9bd1c648f5469a2ec925fcf3a776667042d645472c14",
                "sha256:aabaf16477806a5e1dd19aa41f8c2b7950dd3c746362d7e3223dbe6de6ac448e"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'",
            "version": "==1.26.9"
        },
        "webencodings": {
            "hashes": [
                "sha256:a0af1213f3c2226497a97e2b3aa01a7e4bee4f403f95be16fc9acd2947514a78",
                "sha256:b36a1c245f2d304965eb4e0a82848379241dc04b865afcc4aab16748587e1923"
            ],
            "version": "==0.5.1"
        },
        "zipp": {
            "hashes": [
                "sha256:56bf8aadb83c24db6c4b577e13de374ccfb67da2078beba1d037c17980bf43ad",
                "sha256:c4f6e5bbf48e74f7a38e7cc5b0480ff42b0ae5178957d564d18932525d5cf099"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==3.8.0"
        }
    },
    "develop": {}
}

[matteius]

I am guessing that it has something to do with the version number not changing in the fork, so it keeps the outdated requirement. To be honest, I find the --keep-outdated implementation to be very much a hack that can lead to unexpected requirement update behaviors. I recommend not using it if possible and allowing your requirements to update to the specifiers you have outlined in your Pipfile by simply using pipenv lock or pipenv install without the --keep-outdated flag.

[matteius]

@nicholasserra --keep-outdated and likely --selective-upgrade are deprecated/slated for removal. I've been working on an alternative command that solves the kind of problems these other flags caused. Checkout out: #5617

Any feedback appreicated!