Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python installers (pyenv and asdf) should not need to be present on PATH to be detected by Pipenv #4245

Closed
zbentley opened this issue May 6, 2020 · 2 comments · Fixed by #4246
Labels
Category: Development Issue affects development workflow. PR: awaiting-merge The PR related to this issue has been reviewed and is awaiting merge. Priority: Low This item is low priority and may not be looked at in the next few release cycles. Type: Enhancement 💡 This is a feature or enhancement request.

Comments

@zbentley
Copy link
Contributor

zbentley commented May 6, 2020

Is your feature request related to a problem? Please describe.

We run Linux systems that contain multiple Pipenv projects which span multiple versions of Python. We use pyenv to manage those versions. We set PYENV_ROOT to a common directory in /opt so that they can share Python builds. We set that PYENV_ROOT=/opt/pyenv in an .env` file checked into each project. This has two benefits:

  • We can refer to that .env file as a common "vars that should be set for this project" from other programs that understand such files, e.g. EnvironmentFile directives for Systemd services.
  • We can invoke pipenv from shells that are not specially configured, e.g. human users, service launcher scripts and makefiles.

However, when Pipenv wants to use Pyenv, it expects it to be on PATH. Users of these systems then have to modify their PATH (by whatever means they wish) to point at the /opt/pyenv directory. Despite respecting PYENV_ROOT, Pipenv does not use that variable to resolve the pyenv binary itself.

Describe the solution you'd like

First change: When Pipenv wants to install Python via pipenv or asdf, it should try additional locations to find an installer executable. At present, it searches PATH. I would like it to search the following, in order:

  • PATH
  • The respective "installer root" environment variable + /bin/{installer_name}. The variables in question are ASDF_DIR and PYENV_ROOT.
  • The default recommended source-control based install location for the installer + /bin/{installer_name}.

If an executable file of the expected name is found in any of those locations it should be used.

Second change: whether or not an executable installer can be found according to the above logic should be used to decide whether or not to try the other installer (the pyenv -> asdf fallback). At present, pythonfinder's methods which only check for whether or not an installer's environment variables are configured will be used. On users' systems with both pyenv and asdf, where only asdf is activated, this will be a behavior change; pyenv will now be used for installation.

Describe alternatives you've considered

  • Manual workaround: every human user could ensure pyenv/asdf was on their PATH. Every launcher script could appripriately modify PATH before launching. Pros: requires no Pipenv changes. Cons: all code which wants to do a hands-free pipenv install on a system without an installer on the path will have to be aware of this and be modified.
  • Common shell hacks: a common PATH modification could be added to systemwide shell profile, sourced by all users, to make the installers available. Pros: Requires no Pipenv changes. Cons: doesn't work for programs which invoke Pipenv which don't run in a shell (e.g. Systemd); doesn't work for users with esoteric shells.
  • Make .env files dynamic. If we could write PATH=/opt/pyenv/bin:$PATH in an .env file and have the $PATH reference be substituted, the problem would be solved. Libraries exist to add this dynamic behavior to .env files. Pros: solves this at a lower level than the "find an installer binary" logic. Cons: many other programs consume .env files and do not perform variable substitution. Only Laravel performs that substitution explicitly to my knowledge (and source .env does as well, but that's not treating it as an .env file but rather a shell script).

Additional context

When running pipenv shell without a pyenv binary on the PATH, even with PYENV_ROOT set properly, the following trace is emitted:

Loading .env environment variables...
Warning: Python 3.8 was not found on your system...
Traceback (most recent call last):
  File "/home/zacbentley/.local/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/cli/command.py", line 390, in shell
    pypi_mirror=state.pypi_mirror,
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/core.py", line 2156, in do_shell
    three=three, python=python, validate=False, pypi_mirror=pypi_mirror,
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/core.py", line 574, in ensure_project
    pypi_mirror=pypi_mirror,
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/core.py", line 494, in ensure_virtualenv
    python = ensure_python(three=three, python=python)
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/core.py", line 419, in ensure_python
    version = pyenv.find_version_to_install(python)
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/pyenv.py", line 99, in find_version_to_install
    ), key=operator.attrgetter('cmpkey'))
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/pyenv.py", line 96, in <genexpr>
    inst_version
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/pyenv.py", line 79, in iter_installable_versions
    for name in self._pyenv('install', '--list').out.splitlines():
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/pyenv.py", line 70, in _pyenv
    c = delegator.run(args, block=False, timeout=timeout)
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/delegator.py", line 336, in run
    c.run(block=block, binary=binary, cwd=cwd, env=env)
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/delegator.py", line 204, in run
    s = PopenSpawn(self._popen_args, **pexpect_kwargs)
  File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/pexpect/popen_spawn.py", line 53, in __init__
    self.proc = subprocess.Popen(cmd, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

$ pipenv --support

Pipenv version: '2018.11.26'

Pipenv location: '/home/zacbentley/.local/lib/python2.7/site-packages/pipenv'

Python location: '/usr/bin/python'

Python installations found:

  • 3.5.2: /usr/bin/python3.5
  • 3.5.2: /usr/bin/python3.5m
  • 2.7.12: /usr/bin/python

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.4.0-1102-aws',
 'platform_system': 'Linux',
 'platform_version': '#113-Ubuntu SMP Wed Jan 29 14:54:54 UTC 2020',
 'python_full_version': '2.7.12',
 'python_version': '2.7',
 'sys_platform': 'linux2'}

System environment variables:

  • LC_TERMINAL_VERSION
  • PYTHONDONTWRITEBYTECODE
  • LESSOPEN
  • SSH_CLIENT
  • LOGNAME
  • USER
  • PATH
  • HOME
  • LANG
  • LC_TERMINAL
  • TERM
  • SHELL
  • SHLVL
  • PYTHONFINDER_IGNORE_UNSUPPORTED
  • XDG_RUNTIME_DIR
  • SSH_AUTH_SOCK
  • TMUX
  • PIP_PYTHON_PATH
  • XDG_SESSION_ID
  • _
  • SSH_CONNECTION
  • LESSCLOSE
  • SSH_TTY
  • OLDPWD
  • XDG_DATA_DIRS
  • PWD
  • PIP_SHIMS_BASE_MODULE
  • PIP_DISABLE_PIP_VERSION_CHECK
  • MAIL
  • LS_COLORS
  • TMUX_PANE

Pipenv–specific environment variables:

Debug–specific environment variables:

  • PATH: /home/zacbentley/bin:/home/zacbentley/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
  • SHELL: /bin/bash
  • LANG: en_US.UTF-8
  • PWD: /home/zacbentley/cloudwatch_test

Contents of Pipfile ('/home/zacbentley/cloudwatch_test/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
flask = "*"
pyopenssl = "*"
validatesns = "*"
boto3 = "*"

[requires]
python_version = "3.8"

Contents of Pipfile.lock ('/home/zacbentley/cloudwatch_test/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "965842e951cb3f46daf1ce0b8f0203fdef361b2da8c54d8f27b99cc13b9d32c0"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "asn1crypto": {
            "hashes": [
                "sha256:5a215cb8dc12f892244e3a113fe05397ee23c5c4ca7a69cd6e69811755efc42d",
                "sha256:831d2710d3274c8a74befdddaf9f17fcbf6e350534565074818722d6d615b315"
            ],
            "version": "==1.3.0"
        },
        "boto3": {
            "hashes": [
                "sha256:989ede38b9f69743e2536ccf371941354e77103b47b37d5ba90f77718368a248",
                "sha256:dc4d17a9b0bd6fb03b2650a0f7c7b6a458583fe7bebbcd6bbefd299d7169fb5b"
            ],
            "index": "pypi",
            "version": "==1.13.3"
        },
        "botocore": {
            "hashes": [
                "sha256:d291035e643c353029df8985cbc0bbdcdf9117fff81c715dd688aadd51816f41",
                "sha256:e2c384f378d96c09079d9fa44be4f6a849c4d5295de68c9c8afc8783f7a5a2a2"
            ],
            "version": "==1.16.3"
        },
        "cffi": {
            "hashes": [
                "sha256:001bf3242a1bb04d985d63e138230802c6c8d4db3668fb545fb5005ddf5bb5ff",
                "sha256:00789914be39dffba161cfc5be31b55775de5ba2235fe49aa28c148236c4e06b",
                "sha256:028a579fc9aed3af38f4892bdcc7390508adabc30c6af4a6e4f611b0c680e6ac",
                "sha256:14491a910663bf9f13ddf2bc8f60562d6bc5315c1f09c704937ef17293fb85b0",
                "sha256:1cae98a7054b5c9391eb3249b86e0e99ab1e02bb0cc0575da191aedadbdf4384",
                "sha256:2089ed025da3919d2e75a4d963d008330c96751127dd6f73c8dc0c65041b4c26",
                "sha256:2d384f4a127a15ba701207f7639d94106693b6cd64173d6c8988e2c25f3ac2b6",
                "sha256:337d448e5a725bba2d8293c48d9353fc68d0e9e4088d62a9571def317797522b",
                "sha256:399aed636c7d3749bbed55bc907c3288cb43c65c4389964ad5ff849b6370603e",
                "sha256:3b911c2dbd4f423b4c4fcca138cadde747abdb20d196c4a48708b8a2d32b16dd",
                "sha256:3d311bcc4a41408cf5854f06ef2c5cab88f9fded37a3b95936c9879c1640d4c2",
                "sha256:62ae9af2d069ea2698bf536dcfe1e4eed9090211dbaafeeedf5cb6c41b352f66",
                "sha256:66e41db66b47d0d8672d8ed2708ba91b2f2524ece3dee48b5dfb36be8c2f21dc",
                "sha256:675686925a9fb403edba0114db74e741d8181683dcf216be697d208857e04ca8",
                "sha256:7e63cbcf2429a8dbfe48dcc2322d5f2220b77b2e17b7ba023d6166d84655da55",
                "sha256:8a6c688fefb4e1cd56feb6c511984a6c4f7ec7d2a1ff31a10254f3c817054ae4",
                "sha256:8c0ffc886aea5df6a1762d0019e9cb05f825d0eec1f520c51be9d198701daee5",
                "sha256:95cd16d3dee553f882540c1ffe331d085c9e629499ceadfbda4d4fde635f4b7d",
                "sha256:99f748a7e71ff382613b4e1acc0ac83bf7ad167fb3802e35e90d9763daba4d78",
                "sha256:b8c78301cefcf5fd914aad35d3c04c2b21ce8629b5e4f4e45ae6812e461910fa",
                "sha256:c420917b188a5582a56d8b93bdd8e0f6eca08c84ff623a4c16e809152cd35793",
                "sha256:c43866529f2f06fe0edc6246eb4faa34f03fe88b64a0a9a942561c8e22f4b71f",
                "sha256:cab50b8c2250b46fe738c77dbd25ce017d5e6fb35d3407606e7a4180656a5a6a",
                "sha256:cef128cb4d5e0b3493f058f10ce32365972c554572ff821e175dbc6f8ff6924f",
                "sha256:cf16e3cf6c0a5fdd9bc10c21687e19d29ad1fe863372b5543deaec1039581a30",
                "sha256:e56c744aa6ff427a607763346e4170629caf7e48ead6921745986db3692f987f",
                "sha256:e577934fc5f8779c554639376beeaa5657d54349096ef24abe8c74c5d9c117c3",
                "sha256:f2b0fa0c01d8a0c7483afd9f31d7ecf2d71760ca24499c8697aeb5ca37dc090c"
            ],
            "version": "==1.14.0"
        },
        "click": {
            "hashes": [
                "sha256:d2b5255c7c6349bc1bd1e59e08cd12acbbd63ce649f2588755783aa94dfb6b1a",
                "sha256:dacca89f4bfadd5de3d7489b7c8a566eee0d3676333fbb50030263894c38c0dc"
            ],
            "version": "==7.1.2"
        },
        "cryptography": {
            "hashes": [
                "sha256:091d31c42f444c6f519485ed528d8b451d1a0c7bf30e8ca583a0cac44b8a0df6",
                "sha256:18452582a3c85b96014b45686af264563e3e5d99d226589f057ace56196ec78b",
                "sha256:1dfa985f62b137909496e7fc182dac687206d8d089dd03eaeb28ae16eec8e7d5",
                "sha256:1e4014639d3d73fbc5ceff206049c5a9a849cefd106a49fa7aaaa25cc0ce35cf",
                "sha256:22e91636a51170df0ae4dcbd250d318fd28c9f491c4e50b625a49964b24fe46e",
                "sha256:3b3eba865ea2754738616f87292b7f29448aec342a7c720956f8083d252bf28b",
                "sha256:651448cd2e3a6bc2bb76c3663785133c40d5e1a8c1a9c5429e4354201c6024ae",
                "sha256:726086c17f94747cedbee6efa77e99ae170caebeb1116353c6cf0ab67ea6829b",
                "sha256:844a76bc04472e5135b909da6aed84360f522ff5dfa47f93e3dd2a0b84a89fa0",
                "sha256:88c881dd5a147e08d1bdcf2315c04972381d026cdb803325c03fe2b4a8ed858b",
                "sha256:96c080ae7118c10fcbe6229ab43eb8b090fccd31a09ef55f83f690d1ef619a1d",
                "sha256:a0c30272fb4ddda5f5ffc1089d7405b7a71b0b0f51993cb4e5dbb4590b2fc229",
                "sha256:bb1f0281887d89617b4c68e8db9a2c42b9efebf2702a3c5bf70599421a8623e3",
                "sha256:c447cf087cf2dbddc1add6987bbe2f767ed5317adb2d08af940db517dd704365",
                "sha256:c4fd17d92e9d55b84707f4fd09992081ba872d1a0c610c109c18e062e06a2e55",
                "sha256:d0d5aeaedd29be304848f1c5059074a740fa9f6f26b84c5b63e8b29e73dfc270",
                "sha256:daf54a4b07d67ad437ff239c8a4080cfd1cc7213df57d33c97de7b4738048d5e",
                "sha256:e993468c859d084d5579e2ebee101de8f5a27ce8e2159959b6673b418fd8c785",
                "sha256:f118a95c7480f5be0df8afeb9a11bd199aa20afab7a96bcf20409b411a3a85f0"
            ],
            "version": "==2.9.2"
        },
        "docutils": {
            "hashes": [
                "sha256:6c4f696463b79f1fb8ba0c594b63840ebd41f059e92b31957c46b74a4599b6d0",
                "sha256:9e4d7ecfc600058e07ba661411a2b7de2fd0fafa17d1a7f7361cd47b1175c827",
                "sha256:a2aeea129088da402665e92e0b25b04b073c04b2dce4ab65caaa38b7ce2e1a99"
            ],
            "version": "==0.15.2"
        },
        "flask": {
            "hashes": [
                "sha256:4efa1ae2d7c9865af48986de8aeb8504bf32c7f3d6fdc9353d34b21f4b127060",
                "sha256:8a4fdd8936eba2512e9c85df320a37e694c93945b33ef33c89946a340a238557"
            ],
            "index": "pypi",
            "version": "==1.1.2"
        },
        "itsdangerous": {
            "hashes": [
                "sha256:321b033d07f2a4136d3ec762eac9f16a10ccd60f53c0c91af90217ace7ba1f19",
                "sha256:b12271b2047cb23eeb98c8b5622e2e5c5e9abd9784a153e9d8ef9cb4dd09d749"
            ],
            "version": "==1.1.0"
        },
        "jinja2": {
            "hashes": [
                "sha256:89aab215427ef59c34ad58735269eb58b1a5808103067f7bb9d5836c651b3bb0",
                "sha256:f0a4641d3cf955324a89c04f3d94663aa4d638abe8f733ecd3582848e1c37035"
            ],
            "version": "==2.11.2"
        },
        "jmespath": {
            "hashes": [
                "sha256:695cb76fa78a10663425d5b73ddc5714eb711157e52704d69be03b1a02ba4fec",
                "sha256:cca55c8d153173e21baa59983015ad0daf603f9cb799904ff057bfb8ff8dc2d9"
            ],
            "version": "==0.9.5"
        },
        "markupsafe": {
            "hashes": [
                "sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473",
                "sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161",
                "sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
                "sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
                "sha256:13d3144e1e340870b25e7b10b98d779608c02016d5184cfb9927a9f10c689f42",
                "sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
                "sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
                "sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
                "sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
                "sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
                "sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66",
                "sha256:596510de112c685489095da617b5bcbbac7dd6384aeebeda4df6025d0256a81b",
                "sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
                "sha256:6788b695d50a51edb699cb55e35487e430fa21f1ed838122d722e0ff0ac5ba15",
                "sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
                "sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
                "sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
                "sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
                "sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
                "sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
                "sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
                "sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
                "sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
                "sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
                "sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
                "sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
                "sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
                "sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
                "sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
                "sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
                "sha256:cdb132fc825c38e1aeec2c8aa9338310d29d337bebbd7baa06889d09a60a1fa2",
                "sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7",
                "sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be"
            ],
            "version": "==1.1.1"
        },
        "oscrypto": {
            "hashes": [
                "sha256:2f53f50813ecc5114fd126b89766fabdb9448e0bb080814451cc4bb92acafca4",
                "sha256:e9ea9401294b20ee17225502cc308660814ba8de386f94bc9b1dcf518d5399d9"
            ],
            "version": "==1.2.0"
        },
        "pycparser": {
            "hashes": [
                "sha256:2d475327684562c3a96cc71adf7dc8c4f0565175cf86b6d7a404ff4c771f15f0",
                "sha256:7582ad22678f0fcd81102833f60ef8d0e57288b6b5fb00323d101be910e35705"
            ],
            "version": "==2.20"
        },
        "pyopenssl": {
            "hashes": [
                "sha256:621880965a720b8ece2f1b2f54ea2071966ab00e2970ad2ce11d596102063504",
                "sha256:9a24494b2602aaf402be5c9e30a0b82d4a5c67528fe8fb475e3f3bc00dd69507"
            ],
            "index": "pypi",
            "version": "==19.1.0"
        },
        "python-dateutil": {
            "hashes": [
                "sha256:73ebfe9dbf22e832286dafa60473e4cd239f8592f699aa5adaf10050e6e1823c",
                "sha256:75bb3f31ea686f1197762692a9ee6a7550b59fc6ca3a1f4b5d7e32fb98e2da2a"
            ],
            "version": "==2.8.1"
        },
        "s3transfer": {
            "hashes": [
                "sha256:2482b4259524933a022d59da830f51bd746db62f047d6eb213f2f8855dcb8a13",
                "sha256:921a37e2aefc64145e7b73d50c71bb4f26f46e4c9f414dc648c6245ff92cf7db"
            ],
            "version": "==0.3.3"
        },
        "six": {
            "hashes": [
                "sha256:236bdbdce46e6e6a3d61a337c0f8b763ca1e8717c03b369e87a7ec7ce1319c0a",
                "sha256:8f3cd2e254d8f793e7f3d6d9df77b92252b52637291d0f0da013c76ea2724b6c"
            ],
            "version": "==1.14.0"
        },
        "urllib3": {
            "hashes": [
                "sha256:3018294ebefce6572a474f0604c2021e33b3fd8006ecd11d62107a5d2a963527",
                "sha256:88206b0eb87e6d677d424843ac5209e3fb9d0190d0ee169599165ec25e9d9115"
            ],
            "markers": "python_version != '3.4'",
            "version": "==1.25.9"
        },
        "validatesns": {
            "hashes": [
                "sha256:8596c0d0df88e9ba01da657b784262381ac0ff87097808d408bd883c36710493"
            ],
            "index": "pypi",
            "version": "==0.1.1"
        },
        "werkzeug": {
            "hashes": [
                "sha256:2de2a5db0baeae7b2d2664949077c2ac63fbd16d98da0ff71837f7d1dea3fd43",
                "sha256:6c80b1e5ad3665290ea39320b91e1be1e0d5f60652b964a3070216de83d2e47c"
            ],
            "version": "==1.0.1"
        }
    },
    "develop": {}
}
@frostming frostming added the Type: Enhancement 💡 This is a feature or enhancement request. label May 7, 2020
@triage-new-issues triage-new-issues bot removed the triage label May 7, 2020
@techalchemy
Copy link
Member

Does this problem actually still occur on the master branch of pipenv? I'm fairly sure it doesnt

@techalchemy techalchemy added Status: Needs More Information This issue does not provide enough information to take further action. and removed Type: Enhancement 💡 This is a feature or enhancement request. labels May 7, 2020
@zbentley
Copy link
Contributor Author

zbentley commented May 8, 2020

This still occurs on master. The steps to reproduce the issue are here: #4246 (comment)

@frostming frostming added Category: Development Issue affects development workflow. PR: awaiting-merge The PR related to this issue has been reviewed and is awaiting merge. Priority: Low This item is low priority and may not be looked at in the next few release cycles. Type: Enhancement 💡 This is a feature or enhancement request. and removed Status: Needs More Information This issue does not provide enough information to take further action. labels May 9, 2020
@frostming frostming linked a pull request May 16, 2020 that will close this issue
2 tasks
fwojciak pushed a commit to fwojciak/pipenv that referenced this issue May 29, 2020
2020.5.28 (2020-05-28)
======================

Features & Improvements
-----------------------

-   `pipenv install` and `pipenv sync` will no longer attempt to install satisfied dependencies during installation. pypa#3057, pypa#3506
-   Added support for resolution of direct-url dependencies in `setup.py` files to respect `PEP-508` style URL dependencies. pypa#3148
-   Added full support for resolution of all dependency types including direct URLs, zip archives, tarballs, etc.
    -   Improved error handling and formatting.
    -   Introduced improved cross platform stream wrappers for better `stdout` and `stderr` consistency. pypa#3298
-   For consistency with other commands and the `--dev` option description, `pipenv lock --requirements --dev` now emits both default and development dependencies. The new `--dev-only` option requests the previous behaviour (e.g. to generate a `dev-requirements.txt` file). pypa#3316
-   Pipenv will now successfully recursively lock VCS sub-dependencies. pypa#3328
-   Added support for `--verbose` output to `pipenv run`. pypa#3348
-   Pipenv will now discover and resolve the intrinsic dependencies of **all** VCS dependencies, whether they are editable or not, to prevent resolution conflicts. pypa#3368
-   Added a new environment variable, `PIPENV_RESOLVE_VCS`, to toggle dependency resolution off for non-editable VCS, file, and URL based dependencies. pypa#3577
-   Added the ability for Windows users to enable emojis by setting `PIPENV_HIDE_EMOJIS=0`. pypa#3595
-   Allow overriding `PIPENV_INSTALL_TIMEOUT` environment variable (in seconds). pypa#3652
-   Allow overriding `PIP_EXISTS_ACTION` evironment variable (value is passed to pip install). Possible values here: <https://pip.pypa.io/en/stable/reference/pip/#exists-action-option> Useful when you need to `PIP\_EXISTS\_ACTION=i` (ignore existing packages) - great for CI environments, where you need really fast setup. pypa#3738
-   Pipenv will no longer forcibly override `PIP_NO_DEPS` on all vcs and file dependencies as resolution happens on these in a pre-lock step. pypa#3763
-   Improved verbose logging output during `pipenv lock` will now stream output to the console while maintaining a spinner. pypa#3810
-   Added support for automatic python installs via `asdf` and associated `PIPENV_DONT_USE_ASDF` environment variable. pypa#4018
-   Pyenv/asdf can now be used whether or not they are available on PATH. Setting `PYENV_ROOT`/`ASDF_DIR` in a `.env` file allows Pipenv to install an interpreter without any shell customizations, so long as pyenv/asdf is installed. pypa#4245
-   Added `--key` command line parameter for including personal PyUp.io API tokens when running `pipenv check`. pypa#4257

Behavior Changes
----------------

-   Make conservative checks of known exceptions when subprocess returns output, so user won\'t see the whole traceback - just the error. pypa#2553
-   Do not touch Pipfile early and rely on it so that one can do `pipenv sync` without a Pipfile. pypa#3386
-   Re-enable `--help` option for `pipenv run` command. pypa#3844
-   Make sure `pipenv lock -r --pypi-mirror {MIRROR_URL}` will respect the pypi-mirror in requirements output. pypa#4199

Bug Fixes
---------

-   Raise `PipenvUsageError` when \[\[source\]\] does not contain url field. pypa#2373
-   Fixed a bug which caused editable package resolution to sometimes fail with an unhelpful setuptools-related error message. pypa#2722
-   Fixed an issue which caused errors due to reliance on the system utilities `which` and `where` which may not always exist on some
    systems.
-   Fixed a bug which caused periodic failures in python discovery when executables named `python` were not present on the target `$PATH`. pypa#2783
-   Dependency resolution now writes hashes for local and remote files to the lockfile. pypa#3053
-   Fixed a bug which prevented `pipenv graph` from correctly showing all dependencies when running from within `pipenv shell`. pypa#3071
-   Fixed resolution of direct-url dependencies in `setup.py` files to respect `PEP-508` style URL dependencies. pypa#3148
-   Fixed a bug which caused failures in warning reporting when running pipenv inside a virtualenv under some circumstances.
-   Fixed a bug with package discovery when running `pipenv clean`. pypa#3298
-   Quote command arguments with carets (`^`) on Windows to work around unintended shell escapes. pypa#3307
-   Handle alternate names for UTF-8 encoding. pypa#3313
-   Abort pipenv before adding the non-exist package to Pipfile. pypa#3318
-   Don\'t normalize the package name user passes in. pypa#3324
-   Fix a bug where custom virtualenv can not be activated with pipenv shell pypa#3339
-   Fix a bug that `--site-packages` flag is not recognized. pypa#3351
-   Fix a bug where `pipenv --clear` is not working pypa#3353
-   Fix unhashable type error during `$ pipenv install --selective-upgrade` pypa#3384
-   Dependencies with direct `PEP508` compliant VCS URLs specified in their `install_requires` will now be successfully locked during the resolution process. pypa#3396
-   Fixed a keyerror which could occur when locking VCS dependencies in
    some cases. pypa#3404
-   Fixed a bug that `ValidationError` is thrown when some fields are missing in source section. pypa#3427
-   Updated the index names in lock file when source name in Pipfile is changed. pypa#3449
-   Fixed an issue which caused `pipenv install --help` to show duplicate entries for `--pre`. pypa#3479
-   Fix bug causing `[SSL: CERTIFICATE_VERIFY_FAILED]` when Pipfile `[[source]]` has `verify_ssl=false` and url with custom port. pypa#3502
-   Fix `sync --sequential` ignoring `pip install` errors and logs. pypa#3537
-   Fix the issue that lock file can\'t be created when `PIPENV_PIPFILE` is not under working directory. pypa#3584
-   Pipenv will no longer inadvertently set `editable=True` on all vcs dependencies. pypa#3647
-   The `--keep-outdated` argument to `pipenv install` and `pipenv lock` will now drop specifier constraints when encountering editable dependencies.
    -   In addition, `--keep-outdated` will retain specifiers that would otherwise be dropped from any entries that have not been updated. pypa#3656
-   Fixed a bug which sometimes caused pipenv to fail to respect the `--site-packages` flag when passed with `pipenv install`. pypa#3718
-   Normalize the package names to lowercase when comparing used and in-Pipfile packages. pypa#3745
-   `pipenv update --outdated` will now correctly handle comparisons between pre/post-releases and normal releases. pypa#3766
-   Fixed a `KeyError` which could occur when pinning outdated VCS dependencies via `pipenv lock --keep-outdated`. pypa#3768
-   Resolved an issue which caused resolution to fail when encountering poorly formatted `python_version` markers in `setup.py` and `setup.cfg` files. pypa#3786
-   Fix a bug that installation errors are displayed as a list. pypa#3794
-   Update `pythonfinder` to fix a problem that `python.exe` will be mistakenly chosen for virtualenv creation under WSL. pypa#3807
-   Fixed several bugs which could prevent editable VCS dependencies from being installed into target environments, even when reporting
    successful installation. pypa#3809
-   `pipenv check --system` should find the correct Python interpreter when `python` does not exist on the system. pypa#3819
-   Resolve the symlinks when the path is absolute. pypa#3842
-   Pass `--pre` and `--clear` options to `pipenv update --outdated`. pypa#3879
-   Fixed a bug which prevented resolution of direct URL dependencies which have PEP508 style direct url VCS sub-dependencies with
    subdirectories. pypa#3976
-   Honor `PIPENV_SPINNER` environment variable pypa#4045
-   Fixed an issue with `pipenv check` failing due to an invalid API key from `pyup.io`. pypa#4188
-   Fixed a bug which caused versions from VCS dependencies to be included in `Pipfile.lock` inadvertently. pypa#4217
-   Fixed a bug which caused pipenv to search non-existent virtual environments for `pip` when installing using `--system`. pypa#4220
-   `Requires-Python` values specifying constraint versions of python starting from `1.x` will now be parsed successfully. pypa#4226
-   Fix a bug of `pipenv update --outdated` that can\'t print output correctly. pypa#4229
-   Fixed a bug which caused pipenv to prefer source distributions over wheels from `PyPI` during the dependency resolution phase. Fixed an issue which prevented proper build isolation using `pep517` based builders during dependency resolution. pypa#4231
-   Don\'t fallback to system Python when no matching Python version is found. pypa#4232

Vendored Libraries
------------------

- Updated `pip_shims` to support `--outdated` with new pip versions. pypa#3766
- Update vendored dependencies and invocations
  - Update vendored and patched dependencies
  - Update patches on `piptools`, `pip`, `pip-shims`, `tomlkit`
  - Fix invocations of dependencies
  - Fix custom `InstallCommand` instantiation
  - Update `PackageFinder` usage
  - Fix `Bool` stringify attempts from `tomlkit`
  - Updated vendored dependencies:
    -   **attrs**: `18.2.0 => `19.1.0`
    -   **certifi**: `2018.10.15 => `2019.3.9`
    -   **cached\_property**: `1.4.3 => `1.5.1`
    -   **cerberus**: `1.2.0 => `1.3.1`
    -   **click**: `7.0.0 => `7.1.1`
    -   **click-completion**: `0.5.0 => `0.5.1`
    -   **colorama**: `0.3.9 => `0.4.3`
    -   **contextlib2**: `(new) => `0.6.0.post1`
    -   **distlib**: `0.2.8 => `0.2.9`
    -   **funcsigs**: `(new) => `1.0.2`
    -   **importlib\_metadata** `1.3.0 => `1.5.1`
    -   **importlib-resources**: `(new) => `1.4.0`
    -   **idna**: `2.7 => `2.9`
    -   **jinja2**: `2.10.0 => `2.11.1`
    -   **markupsafe**: `1.0 => `1.1.1`
    -   **more-itertools**: `(new) => `5.0.0`
    -   **orderedmultidict**: `(new) => `1.0`
    -   **packaging**: `18.0 => `19.0`
    -   **parse**: `1.9.0 => `1.15.0`
    -   **pathlib2**: `2.3.2 => `2.3.3`
    -   **pep517**: `(new) => `0.5.0`
    -   **pexpect**: `4.6.0 => `4.8.0`
    -   **pip-shims**: `0.2.0 => `0.5.1`
    -   **pipdeptree**: `0.13.0 => `0.13.2`
    -   **pyparsing**: `2.2.2 => `2.4.6`
    -   **python-dotenv**: `0.9.1 => `0.10.2`
    -   **pythonfinder**: `1.1.10 => `1.2.2`
    -   **pytoml**: `(new) => `0.1.20`
    -   **requests**: `2.20.1 => `2.23.0`
    -   **requirementslib**: `1.3.3 => `1.5.4`
    -   **scandir**: `1.9.0 => `1.10.0`
    -   **shellingham**: `1.2.7 => `1.3.2`
    -   **six**: `1.11.0 => `1.14.0`
    -   **tomlkit**: `0.5.2 => `0.5.11`
    -   **urllib3**: `1.24 => `1.25.8`
    -   **vistir**: `0.3.0 => `0.5.0`
    -   **yaspin**: `0.14.0 => `0.14.3`
    -   **zipp**: `0.6.0`
    - Removed vendored dependency **cursor**. pypa#4169

-   Add and update vendored dependencies to accommodate `safety` vendoring:
    -   **safety** `(none)` => `1.8.7`
    -   **dparse** `(none)` => `0.5.0`
    -   **pyyaml** `(none)` => `5.3.1`
    -   **urllib3** `1.25.8` => `1.25.9`
    -   **certifi** `2019.11.28` => `2020.4.5.1`
    -   **pyparsing** `2.4.6` => `2.4.7`
    -   **resolvelib** `0.2.2` => `0.3.0`
    -   **importlib-metadata** `1.5.1` => `1.6.0`
    -   **pip-shims** `0.5.1` => `0.5.2`
    -   **requirementslib** `1.5.5` => `1.5.6` pypa#4188

-   Updated vendored `pip` => `20.0.2` and `pip-tools` => `5.0.0`. pypa#4215
-   Updated vendored dependencies to latest versions for security and bug fixes:
    -   **requirementslib** `1.5.8` => `1.5.9`
    -   **vistir** `0.5.0` => `0.5.1`
    -   **jinja2** `2.11.1` => `2.11.2`
    -   **click** `7.1.1` => `7.1.2`
    -   **dateutil** `(none)` => `2.8.1`
    -   **backports.functools\_lru\_cache** `1.5.0` => `1.6.1`
    -   **enum34** `1.1.6` => `1.1.10`
    -   **toml** `0.10.0` => `0.10.1`
    -   **importlib\_resources** `1.4.0` => `1.5.0` pypa#4226
-   Changed attrs import path in vendored dependencies to always import from `pipenv.vendor`. pypa#4267

Improved Documentation
----------------------

-   Added documenation about variable expansion in `Pipfile` entries. pypa#2317
-   Consolidate all contributing docs in the rst file pypa#3120
-   Update the out-dated manual page. pypa#3246
-   Move CLI docs to its own page. pypa#3346
-   Replace (non-existant) video on docs index.rst with equivalent gif. pypa#3499
-   Clarify wording in Basic Usage example on using double quotes to escape shell redirection pypa#3522
-   Ensure docs show navigation on small-screen devices pypa#3527
-   Added a link to the TOML Spec under General Recommendations & Version Control to clarify how Pipfiles should be written. pypa#3629
-   Updated the documentation with the new `pytest` entrypoint. pypa#3759
-   Fix link to GIF in README.md demonstrating Pipenv\'s usage, and add descriptive alt text. pypa#3911
-   Added a line describing potential issues in fancy extension. pypa#3912
-   Documental description of how Pipfile works and association with Pipenv. pypa#3913
-   Clarify the proper value of `python_version` and `python_full_version`. pypa#3914
-   Write description for `--deploy` extension and few extensions differences. pypa#3915
-   More documentation for `.env` files pypa#4100
-   Updated documentation to point to working links. pypa#4137
-   Replace docs.pipenv.org with pipenv.pypa.io pypa#4167
-   Added functionality to check spelling in documentation and cleaned up existing typographical issues. pypa#4209
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Category: Development Issue affects development workflow. PR: awaiting-merge The PR related to this issue has been reviewed and is awaiting merge. Priority: Low This item is low priority and may not be looked at in the next few release cycles. Type: Enhancement 💡 This is a feature or enhancement request.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants