Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pipenv install -e . fails due to dependency resolution for packages installed through direct URL #3148

Closed
haizaar opened this issue Nov 2, 2018 · 19 comments · Fixed by #3298
Labels
Category: Dependency Resolution Issue relates to dependency resolution. Type: Bug 🐛 This issue is a bug.

Comments

@haizaar
Copy link

haizaar commented Nov 2, 2018

Issue description

The title may be a bit confusing, thus I hope description will be clearer.

My packages section of Pipfile is as follows:

[packages]
prometheus-client = "*"
uberlogging = {file = "https://github.com/haizaar/uberlogging/archive/v0.0.2.tar.gz"}
private_pkg = {file = "https://${GITHUB_RO_TOKEN}:@github.com/Org/private_pkg/archive/v0.1.0.tar.gz"}

pipenv install passes fine, but then I try to do pipenv install --dev -e . with the following setup.py:

from setuptools import find_packages, setup

setup(
    name="foo",
    version="0.0.1",
    url="https://github.com/...",
    packages=find_packages(),
    install_requires=(
        "prometheus-client",
        "uberlogging",
        "private_pkg",
    )
)

it fails with Could not find a version that matches private_pkg, though private_pkg is listed in the
satisfied requirements below. pip install -e . works just fine in pipenv's venv and the resulting installation is functional. Bug?

$ pipenv install --dev -e .
Installing -e .…
Obtaining file:///home/haizaar/dev/healthshield/channeler
Requirement already satisfied, skipping upgrade: prometheus-client in /home/.../lib/python3.6/site-packages (from foo==0.0.1) (0.4.2)
Requirement already satisfied, skipping upgrade: uberlogging in /home/.../lib/python3.6/site-packages (from foo==0.0.1) (0.0.2)
Requirement already satisfied, skipping upgrade: private_pkg in /home/.../lib/python3.6/site-packages (from foo==0.0.1) (0.1.0)
Requirement already satisfied, skipping upgrade: python-json-logger in /home/.../lib/python3.6/site-packages (from uberlogging->foo==0.0.1) (0.1.9)
Requirement already satisfied, skipping upgrade: humanfriendly in /home/.../lib/python3.6/site-packages (from uberlogging->foo==0.0.1) (4.17)
Requirement already satisfied, skipping upgrade: structlog in /home/.../lib/python3.6/site-packages (from uberlogging->foo==0.0.1) (18.2.0)
Requirement already satisfied, skipping upgrade: coloredlogs in /home/.../lib/python3.6/site-packages (from uberlogging->foo==0.0.1) (10.0)
Requirement already satisfied, skipping upgrade: pyyaml in /home/.../lib/python3.6/site-packages (from private_pkg->foo==0.0.1) (3.13)
Requirement already satisfied, skipping upgrade: configmanager in /home/.../lib/python3.6/site-packages (from private_pkg->foo==0.0.1) (1.34.1)
Requirement already satisfied, skipping upgrade: setuptools in /home/.../lib/python3.6/site-packages (from python-json-logger->uberlogging->foo==0.0.1) (40.5.0)
Requirement already satisfied, skipping upgrade: six in /home/.../lib/python3.6/site-packages (from structlog->uberlogging->foo==0.0.1) (1.11.0)
Requirement already satisfied, skipping upgrade: configparser==3.5.0 in /home/.../lib/python3.6/site-packages (from configmanager->private_pkg->foo==0.0.1) (3.5.0)
Requirement already satisfied, skipping upgrade: future==0.16.0 in /home/.../lib/python3.6/site-packages (from configmanager->private_pkg->foo==0.0.1) (0.16.0)
Requirement already satisfied, skipping upgrade: hookery==1.4.0 in /home/.../lib/python3.6/site-packages (from configmanager->private_pkg->foo==0.0.1) (1.4.0)
Requirement already satisfied, skipping upgrade: funcsigs in /home/.../lib/python3.6/site-packages (from hookery==1.4.0->configmanager->private_pkg->foo==0.0.1) (1.0.2)
Installing collected packages: foo
  Found existing installation: foo 0.0.1
    Uninstalling foo-0.0.1:
      Successfully uninstalled foo-0.0.1
  Running setup.py develop for foo
Successfully installed foo

Adding foo to Pipfile's [dev-packages]…
Pipfile.lock (68f562) out of date, updating to (4f417c)…
Locking [dev-packages] dependencies…

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches private_pkg
No versions found
Was https://pypi.org/simple reachable?

$ pipenv --support

Pipenv version: '2018.10.13'

Pipenv location: '/home/.../.local/lib/python3.6/site-packages/pipenv'

Python location: '/usr/bin/python3.6'

Python installations found:

  • 3.7.0: /usr/bin/python3.7
  • 3.7.0: /usr/bin/python3.7m
  • 3.6.6: /home/.../bin/python3.6
  • 3.6.6: /usr/bin/python3.6
  • 3.6.6: /usr/bin/python3.6m
  • 3.5.2: /usr/bin/python3.5
  • 3.5.2: /usr/bin/python3.5m
  • 2.7.12: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.6',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.15.0-36-generic',
 'platform_system': 'Linux',
 'platform_version': '#39~16.04.1-Ubuntu SMP Tue Sep 25 08:59:23 UTC 2018',
 'python_full_version': '3.6.6',
 'python_version': '3.6',
 'sys_platform': 'linux'}

System environment variables:

  • LC_PAPER
  • XDG_VTNR
  • XDG_SESSION_ID
  • LC_ADDRESS
  • CLUTTER_IM_MODULE
  • LC_MONETARY
  • XDG_GREETER_DATA_DIR
  • VIRTUALENVWRAPPER_SCRIPT
  • VIRTUALENVWRAPPER_PROJECT_FILENAME
  • SESSION
  • GPG_AGENT_INFO
  • PIP_PYTHON_PATH
  • SHELL
  • VTE_VERSION
  • TERM
  • QT_LINUX_ACCESSIBILITY_ALWAYS_ON
  • DIRENV_DIR
  • LC_NUMERIC
  • WINDOWID
  • GNOME_KEYRING_CONTROL
  • UPSTART_SESSION
  • GTK_MODULES
  • USER
  • QT_ACCESSIBILITY
  • LC_TELEPHONE
  • LS_COLORS
  • UNITY_HAS_3D_SUPPORT
  • XDG_SESSION_PATH
  • XDG_SEAT_PATH
  • SSH_AUTH_SOCK
  • DEFAULTS_PATH
  • VIRTUAL_ENV
  • WORKON_HOME
  • DIRENV_WATCHES
  • UNITY_DEFAULT_PROFILE
  • XDG_CONFIG_DIRS
  • PIPENV_ACTIVE
  • PATH
  • DESKTOP_SESSION
  • QT_QPA_PLATFORMTHEME
  • QT_IM_MODULE
  • VIRTUALENVWRAPPER_HOOK_DIR
  • LC_IDENTIFICATION
  • JOB
  • PWD
  • XDG_SESSION_TYPE
  • XMODIFIERS
  • LANG
  • GNOME_KEYRING_PID
  • MANDATORY_PATH
  • GDM_LANG
  • LC_MEASUREMENT
  • NODE_PATH
  • IM_CONFIG_PHASE
  • COMPIZ_CONFIG_PROFILE
  • PS1
  • PYTHONDONTWRITEBYTECODE
  • PAPERSIZE
  • GDMSESSION
  • GTK2_MODULES
  • SESSIONTYPE
  • GITAWAREPROMPT
  • XDG_SEAT
  • HOME
  • SHLVL
  • LANGUAGE
  • _VIRTUALENVWRAPPER_API
  • GNOME_DESKTOP_SESSION_ID
  • UPSTART_INSTANCE
  • PIP_SHIMS_BASE_MODULE
  • LOGNAME
  • UPSTART_EVENTS
  • XDG_SESSION_DESKTOP
  • PYTHONPATH
  • GITHUB_RO_TOKEN
  • COMPIZ_BIN_PATH
  • QT4_IM_MODULE
  • XDG_DATA_DIRS
  • DBUS_SESSION_BUS_ADDRESS
  • LESSOPEN
  • UPSTART_JOB
  • INSTANCE
  • DISPLAY
  • XDG_RUNTIME_DIR
  • GTK_IM_MODULE
  • XDG_CURRENT_DESKTOP
  • DIRENV_DIFF
  • LC_TIME
  • LESSCLOSE
  • XAUTHORITY
  • LC_NAME
  • _

Pipenv–specific environment variables:

  • PIPENV_ACTIVE: 1

Debug–specific environment variables:

  • SHELL: /bin/bash
  • LANG: en_US.UTF-8

Contents of Pipfile ('/home/.../Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
prometheus-client = "*"
uberlogging = {file = "https://github.com/haizaar/uberlogging/archive/v0.0.2.tar.gz"}
private_pkg = {file = "https://${GITHUB_RO_TOKEN}:@github.com/Org/private_pkg/archive/v0.1.0.tar.gz"}

[dev-packages]
medium = {editable = true, path = "."}

[requires]
python_version = "3.6"

Contents of Pipfile.lock ('/home/.../Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "ae2ceeae45ac82d25d2563ac6b7d421babd5dda5437a4f0c35fddc78be4f417c"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.6"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "private_pkg: {
            "file": "https://${GITHUB_RO_TOKEN}:@github.com/Org/private_pkg/archive/v0.1.0.tar.gz"
        },
        "prometheus-client": {
            "hashes": [
                "sha256:046cb4fffe75e55ff0e6dfd18e2ea16e54d86cc330f369bebcc683475c8b68a9"
            ],
            "index": "pypi",
            "version": "==0.4.2"
        },
        "uberlogging": {
            "file": "https://github.com/haizaar/uberlogging/archive/v0.0.2.tar.gz"
        }
    },
    "develop": {}
}
@techalchemy techalchemy added Type: Duplicate This issue is a duplicate of an already-existing issue. Type: Behavior Change This issue describes a behavior change. Status: Requires PEEP This issue requires an accompanying enhancement proposal labels Nov 3, 2018
@techalchemy
Copy link
Member

This currently only works in the sources section of your pipfile. Also, I'm not really sure that you need the : where you have it before the @. This would be better left to an ssh config file or you can wait on #3111 which already contains a proposed PEEP on this topic.

If you have a need to expand variables here, you will need to write an enhancement proposal and submit it as a PR against the peeps directory according to PEEP-001

See also: #3048, #3049, #3137

@haizaar
Copy link
Author

haizaar commented Nov 4, 2018

This currently only works in the sources section of your pipfile.

I'm not sure what you mean: pipenv install works just fine for my direct urls - credentials are expanded properly.

P.S. You are right, : is not needed before @. It does not affect the issue btw :)

@haizaar haizaar changed the title pipenv install --dev -e . fails due to dependency resolution for private modules installed through URL pipenv install -e . fails due to dependency resolution for packages installed through direct URL Nov 4, 2018
@haizaar
Copy link
Author

haizaar commented Nov 4, 2018

Did more tests - the issue is not related to credentials.
I think the issue is related to dependency resolution for packages installed through direct URL.

Here is a simple reproduction:

cd $(mktemp -d)
pipenv install https://github.com/haizaar/uberlogging/archive/v0.0.2.tar.gz
cat > setup.py << EOF
from setuptools import find_packages, setup
setup(
    name="foo",
    version="0.0.1",
    packages=find_packages(),
    install_requires=(
        "uberlogging",
    )
)
EOF
pipenv install -e .

The last one fails with:

$ pipenv install -e .
Installing -e .…
Obtaining file:///tmp/tmp.d1vO9BlI1j
Requirement already satisfied, skipping upgrade: uberlogging in /home/.../lib/python3.6/site-packages (from foo==0.0.1) (0.0.2)
Requirement already satisfied, skipping upgrade: python-json-logger in /home/.../lib/python3.6/site-packages (from uberlogging->foo==0.0.1) (0.1.9)
Requirement already satisfied, skipping upgrade: coloredlogs in /home/.../lib/python3.6/site-packages (from uberlogging->foo==0.0.1) (10.0)
Requirement already satisfied, skipping upgrade: humanfriendly in /home/.../lib/python3.6/site-packages (from uberlogging->foo==0.0.1) (4.17)
Requirement already satisfied, skipping upgrade: structlog in /home/.../lib/python3.6/site-packages (from uberlogging->foo==0.0.1) (18.2.0)
Requirement already satisfied, skipping upgrade: setuptools in /home/.../lib/python3.6/site-packages (from python-json-logger->uberlogging->foo==0.0.1) (40.5.0)
Requirement already satisfied, skipping upgrade: six in /home/.../lib/python3.6/site-packages (from structlog->uberlogging->foo==0.0.1) (1.11.0)
Installing collected packages: foo
  Running setup.py develop for foo
Successfully installed foo

Adding foo to Pipfile's [packages]…
Pipfile.lock (5018c8) out of date, updating to (368469)…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches uberlogging
No versions found
Was https://pypi.org/simple reachable?

@haizaar
Copy link
Author

haizaar commented Nov 5, 2018

@techalchemy Sorry if I'm nudging, but usually you are quite responsive and there is silence on this one. Have you had a chance to look at my last comment where I show that it's actually not related to credentials and affects any package installed by direct URL? I.e. pipenv install -d + "package installed from direct URL" = NO_GO

@techalchemy
Copy link
Member

Pip needs to know where to look to resolve dependencies—usually with PIP_FIND_LINKS or some other variables. The resolution process can’t occur unless the resolver has instructions for finding packages generally, and by default it only checks PyPI which won’t have your package

@haizaar
Copy link
Author

haizaar commented Nov 5, 2018

Pip actually works, i.e.

pipenv install
pipenv run pip install -e .

works just fine. I guess it's because pip sees that all install_requires-ed packages are already installed. It only breaks with pipenv:

pipenv install
pipenv install -e .

If it works in pip it should work with pipenv as well, right? At least this is the end game vision, right?

@techalchemy
Copy link
Member

techalchemy commented Nov 5, 2018

Pip 18.1 is the first release to work this way. We vendor pip for dependency resolution and I only just updated it a week ago. Please test against master. This functionality has existed for like a month

@haizaar
Copy link
Author

haizaar commented Nov 5, 2018

Tried with the latest master. Still breaks with similar error.

Detailed error log

/ # pip install git+https://github.com/pypa/pipenv.git
Collecting git+https://github.com/pypa/pipenv.git
Cloning https://github.com/pypa/pipenv.git to /tmp/pip-req-build-w6rf67vx
Installing build dependencies ... done
Requirement already satisfied: pip>=9.0.1 in /usr/lib/python3.6/site-packages (from pipenv==2018.10.14.dev0) (18.1)
Collecting certifi (from pipenv==2018.10.14.dev0)
Downloading https://files.pythonhosted.org/packages/56/9d/1d02dd80bc4cd955f98980f28c5ee2200e1209292d5f9e9cc8d030d18655/certifi-2018.10.15-py2.py3-none-any.whl (146kB)
100% |████████████████████████████████| 153kB 4.8MB/s
Requirement already satisfied: setuptools>=36.2.1 in /usr/lib/python3.6/site-packages (from pipenv==2018.10.14.dev0) (39.0.1)
Collecting virtualenv-clone>=0.2.5 (from pipenv==2018.10.14.dev0)
Downloading https://files.pythonhosted.org/packages/16/9d/6419a4f0fe4350db7fdc01e9d22e949779b6f2d2650e4884aa8aededc5ae/virtualenv_clone-0.4.0-py2.py3-none-any.whl
Collecting virtualenv (from pipenv==2018.10.14.dev0)
Downloading https://files.pythonhosted.org/packages/7c/17/9b7b6cddfd255388b58c61e25b091047f6814183e1d63741c8df8dcd65a2/virtualenv-16.1.0-py2.py3-none-any.whl (1.9MB)
100% |████████████████████████████████| 1.9MB 5.2MB/s
Installing collected packages: certifi, virtualenv-clone, virtualenv, pipenv
Running setup.py install for pipenv ... done
Successfully installed certifi-2018.10.15 pipenv-2018.10.14.dev0 virtualenv-16.1.0 virtualenv-clone-0.4.0
/ #
/ #
/ # cd $(mktemp -d)
/tmp/tmp.BHLlIg # pipenv install https://github.com/haizaar/uberlogging/archive/v0.0.2.tar.gz
Warning: the environment variable LANG is not set!
We recommend setting this in ~/.profile (or equivalent) for proper expected behavior.
Creating a virtualenv for this project…
Pipfile: /tmp/tmp.BHLlIg/Pipfile
Using /usr/bin/python3.6 (3.6.6) to create virtualenv…
✔ Complete Running... done.
Already using interpreter /usr/bin/python3.6
Using base prefix '/usr'
New python executable in /root/.local/share/virtualenvs/tmp.BHLlIg-eVE_7PpZ/bin/python3.6
Also creating executable in /root/.local/share/virtualenvs/tmp.BHLlIg-eVE_7PpZ/bin/python
Installing setuptools, pip, wheel...
done.
Virtualenv location: /root/.local/share/virtualenvs/tmp.BHLlIg-eVE_7PpZ
Creating a Pipfile for this project…
Installing https://github.com/haizaar/uberlogging/archive/v0.0.2.tar.gz…
⠸ Installing...Collecting https://github.com/haizaar/uberlogging/archive/v0.0.2.tar.gz
Downloading https://github.com/haizaar/uberlogging/archive/v0.0.2.tar.gz
Collecting coloredlogs (from uberlogging==0.0.2)
Downloading https://files.pythonhosted.org/packages/08/0f/7877fc42fff0b9d70b6442df62d53b3868d3a6ad1b876bdb54335b30ff23/coloredlogs-10.0-py2.py3-none-any.whl (47kB)
Collecting structlog (from uberlogging==0.0.2)
Downloading https://files.pythonhosted.org/packages/f0/00/0fd0ca13fa19361bec0418e4c3b6b7509048cb1fb2fa8b7cd6b3dffe13d8/structlog-18.2.0-py2.py3-none-any.whl
Collecting humanfriendly (from uberlogging==0.0.2)
Downloading https://files.pythonhosted.org/packages/79/1e/13d96248e3fcaa7777b61fa889feab44865c85e524bbd667acfa0d8b66e3/humanfriendly-4.17-py2.py3-none-any.whl (72kB)
Collecting python-json-logger (from uberlogging==0.0.2)
Downloading https://files.pythonhosted.org/packages/3a/ed/2ba5a2dfce45fa67e0588dd6457b59ffdef7f1fdcd2c1152e085e3c9a726/python_json_logger-0.1.9-py2.py3-none-any.whl
Collecting six (from structlog->uberlogging==0.0.2)
Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Requirement already satisfied, skipping upgrade: setuptools in /root/.local/share/virtualenvs/tmp.BHLlIg-eVE_7PpZ/lib/python3.6/site-packages (from python-json-logger->uberlogging==0.0.2) (40.5.0)
Building wheels for collected packages: uberlogging
Running setup.py bdist_wheel for uberlogging: started
Running setup.py bdist_wheel for uberlogging: finished with status 'done'
Stored in directory: /tmp/pip-ephem-wheel-cache-qom9kpcq/wheels/cf/c0/c4/1c378d530b8e88febf13b0d4afd1c54cd4fe22e1064249d012
Successfully built uberlogging
Installing collected packages: humanfriendly, coloredlogs, six, structlog, python-json-logger, uberlogging
Successfully installed coloredlogs-10.0 humanfriendly-4.17 python-json-logger-0.1.9 six-1.11.0 structlog-18.2.0 uberlogging-0.0.2

Adding 8795c83 to Pipfile's [packages]…
✔ Installation Succeeded Installing...
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (368469)!
Installing dependencies from Pipfile.lock (368469)…
🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:05
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
/tmp/tmp.BHLlIg # cat > setup.py << EOF

from setuptools import find_packages, setup
setup(
name="foo",
version="0.0.1",
packages=find_packages(),
install_requires=(
"uberlogging",
)
)
EOF
/tmp/tmp.BHLlIg # pipenv install -e .
Installing -e .…
⠼ Installing...Obtaining file:///tmp/tmp.BHLlIg
Requirement already satisfied, skipping upgrade: uberlogging in /root/.local/share/virtualenvs/tmp.BHLlIg-eVE_7PpZ/lib/python3.6/site-packages (from foo==0.0.1) (0.0.2)
Requirement already satisfied, skipping upgrade: python-json-logger in /root/.local/share/virtualenvs/tmp.BHLlIg-eVE_7PpZ/lib/python3.6/site-packages (from uberlogging->foo==0.0.1) (0.1.9)
Requirement already satisfied, skipping upgrade: coloredlogs in /root/.local/share/virtualenvs/tmp.BHLlIg-eVE_7PpZ/lib/python3.6/site-packages (from uberlogging->foo==0.0.1) (10.0)
Requirement already satisfied, skipping upgrade: humanfriendly in /root/.local/share/virtualenvs/tmp.BHLlIg-eVE_7PpZ/lib/python3.6/site-packages (from uberlogging->foo==0.0.1) (4.17)
Requirement already satisfied, skipping upgrade: structlog in /root/.local/share/virtualenvs/tmp.BHLlIg-eVE_7PpZ/lib/python3.6/site-packages (from uberlogging->foo==0.0.1) (18.2.0)
Requirement already satisfied, skipping upgrade: setuptools in /root/.local/share/virtualenvs/tmp.BHLlIg-eVE_7PpZ/lib/python3.6/site-packages (from python-json-logger->uberlogging->foo==0.0.1) (40.5.0)
Requirement already satisfied, skipping upgrade: six in /root/.local/share/virtualenvs/tmp.BHLlIg-eVE_7PpZ/lib/python3.6/site-packages (from structlog->uberlogging->foo==0.0.1) (1.11.0)
Installing collected packages: foo
Running setup.py develop for foo
Successfully installed foo

Adding foo to Pipfile's [packages]…
✔ Installation Succeeded Installing...
Pipfile.lock (5018c8) out of date, updating to (368469)…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
✘ Locking Failed! Locking...
[pipenv.exceptions.ResolutionFailure]: File "/usr/lib/python3.6/site-packages/pipenv/resolver.py", line 69, in resolve
[pipenv.exceptions.ResolutionFailure]: req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]: File "/usr/lib/python3.6/site-packages/pipenv/utils.py", line 568, in resolve_deps
[pipenv.exceptions.ResolutionFailure]: req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]: File "/usr/lib/python3.6/site-packages/pipenv/utils.py", line 434, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]: resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]: File "/usr/lib/python3.6/site-packages/pipenv/utils.py", line 402, in resolve
[pipenv.exceptions.ResolutionFailure]: raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches uberlogging
[pipenv.exceptions.ResolutionFailure]: No versions found
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches uberlogging
No versions found
Was https://pypi.org/simple reachable?
[pipenv.exceptions.ResolutionFailure]: req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]: File "/usr/lib/python3.6/site-packages/pipenv/utils.py", line 568, in resolve_deps
[pipenv.exceptions.ResolutionFailure]: req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]: File "/usr/lib/python3.6/site-packages/pipenv/utils.py", line 434, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]: resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]: File "/usr/lib/python3.6/site-packages/pipenv/utils.py", line 402, in resolve
[pipenv.exceptions.ResolutionFailure]: raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches uberlogging
[pipenv.exceptions.ResolutionFailure]: No versions found
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches uberlogging
No versions found
Was https://pypi.org/simple reachable?
/tmp/tmp.BHLlIg #

$ pipenv --support

Pipenv version: '2018.10.14.dev0'

Pipenv location: '/usr/lib/python3.6/site-packages/pipenv'

Python location: '/usr/bin/python3.6'

Python installations found:

  • 3.6.6: /usr/bin/python3
  • 3.6.6: /usr/bin/python3.6m

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.6',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.15.0-36-generic',
 'platform_system': 'Linux',
 'platform_version': '#39~16.04.1-Ubuntu SMP Tue Sep 25 08:59:23 UTC 2018',
 'python_full_version': '3.6.6',
 'python_version': '3.6',
 'sys_platform': 'linux'}

System environment variables:

  • HOSTNAME
  • SHLVL
  • HOME
  • OLDPWD
  • TERM
  • PATH
  • PWD
  • PIP_DISABLE_PIP_VERSION_CHECK
  • PIP_SHIMS_BASE_MODULE
  • PYTHONDONTWRITEBYTECODE
  • PIP_PYTHON_PATH
  • PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

  • PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
  • PWD: /tmp/tmp.BHLlIg

Contents of Pipfile ('/tmp/tmp.BHLlIg/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
"8795c83" = {file = "https://github.com/haizaar/uberlogging/archive/v0.0.2.tar.gz"}
foo = {editable = true, path = "."}

[dev-packages]

[requires]
python_version = "3.6"

Contents of Pipfile.lock ('/tmp/tmp.BHLlIg/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "56e5ae563599827b1b2b1646113786b350c9aa72bfa4a3c16e7e7740b9368469"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.6"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "8795c83": {
            "file": "https://github.com/haizaar/uberlogging/archive/v0.0.2.tar.gz"
        }
    },
    "develop": {}
}

@haizaar
Copy link
Author

haizaar commented Nov 5, 2018

I guess I'm lucky to start with those URL install just after they released :)

@haizaar
Copy link
Author

haizaar commented Nov 6, 2018

Can we reopen it in a meanwhile?

@techalchemy
Copy link
Member

Yes sorry :) I haven't actually sorted out how to hand this over to pip during resolution, but I need to get this sorted out before I cut the release

@techalchemy techalchemy reopened this Nov 6, 2018
@haizaar
Copy link
Author

haizaar commented Nov 6, 2018

I guess labels are off as well.

@techalchemy techalchemy added Type: Bug 🐛 This issue is a bug. Category: Dependency Resolution Issue relates to dependency resolution. and removed Type: Behavior Change This issue describes a behavior change. Type: Duplicate This issue is a duplicate of an already-existing issue. Status: Requires PEEP This issue requires an accompanying enhancement proposal labels Nov 6, 2018
@techalchemy
Copy link
Member

In looking this over, I think it may actually be due to a bug in wheel -- can you try to pin wheel<32.2.0 inside your virtualenv?

@haizaar
Copy link
Author

haizaar commented Nov 15, 2018

I guess you meant wheel<0.32.2. Anyways, it does not seem to help:

$ ls -d /home/.../lib/python3.6/site-packages/wheel-*dist-info
/home/.../python3.6/site-packages/wheel-0.32.1.dist-info

Then trying to pipenv install -e . spits the same error as in my original post.

techalchemy added a commit that referenced this issue Nov 25, 2018
- Fix dependecy resolution for _all_ dependencies which are just
  direct URLs or files
- Bypass pip-tools for non-editable requirements which can be resolved
- Fixes #3148

Signed-off-by: Dan Ryan <dan@danryan.co>
@haizaar
Copy link
Author

haizaar commented Jan 23, 2019

Happy New Year everyone. Do you think you'll have have to work on it in the near future?

@techalchemy
Copy link
Member

Next item on my list due to dependency links being disabled as of today’s pip release. It’s holding back my other changes now so I’ll be looking at it tomorrow. Thanks for your patience and sorry for taking so long on this one.

techalchemy added a commit that referenced this issue Jan 24, 2019
- Fix dependecy resolution for _all_ dependencies which are just
  direct URLs or files
- Bypass pip-tools for non-editable requirements which can be resolved
- Fixes #3148

Signed-off-by: Dan Ryan <dan@danryan.co>
techalchemy added a commit that referenced this issue Feb 11, 2019
- Fixes #3148

Signed-off-by: Dan Ryan <dan@danryan.co>
@techalchemy techalchemy added this to the March Release milestone Mar 5, 2019
@techalchemy
Copy link
Member

This was incredibly challenging but I have it completely implemented and will be merging it tonight

@haizaar
Copy link
Author

haizaar commented Mar 5, 2019 via email

@haizaar
Copy link
Author

haizaar commented Mar 6, 2019

Yeeeee-haw!!! It works! Thank you very much!

fwojciak pushed a commit to fwojciak/pipenv that referenced this issue May 29, 2020
2020.5.28 (2020-05-28)
======================

Features & Improvements
-----------------------

-   `pipenv install` and `pipenv sync` will no longer attempt to install satisfied dependencies during installation. pypa#3057, pypa#3506
-   Added support for resolution of direct-url dependencies in `setup.py` files to respect `PEP-508` style URL dependencies. pypa#3148
-   Added full support for resolution of all dependency types including direct URLs, zip archives, tarballs, etc.
    -   Improved error handling and formatting.
    -   Introduced improved cross platform stream wrappers for better `stdout` and `stderr` consistency. pypa#3298
-   For consistency with other commands and the `--dev` option description, `pipenv lock --requirements --dev` now emits both default and development dependencies. The new `--dev-only` option requests the previous behaviour (e.g. to generate a `dev-requirements.txt` file). pypa#3316
-   Pipenv will now successfully recursively lock VCS sub-dependencies. pypa#3328
-   Added support for `--verbose` output to `pipenv run`. pypa#3348
-   Pipenv will now discover and resolve the intrinsic dependencies of **all** VCS dependencies, whether they are editable or not, to prevent resolution conflicts. pypa#3368
-   Added a new environment variable, `PIPENV_RESOLVE_VCS`, to toggle dependency resolution off for non-editable VCS, file, and URL based dependencies. pypa#3577
-   Added the ability for Windows users to enable emojis by setting `PIPENV_HIDE_EMOJIS=0`. pypa#3595
-   Allow overriding `PIPENV_INSTALL_TIMEOUT` environment variable (in seconds). pypa#3652
-   Allow overriding `PIP_EXISTS_ACTION` evironment variable (value is passed to pip install). Possible values here: <https://pip.pypa.io/en/stable/reference/pip/#exists-action-option> Useful when you need to `PIP\_EXISTS\_ACTION=i` (ignore existing packages) - great for CI environments, where you need really fast setup. pypa#3738
-   Pipenv will no longer forcibly override `PIP_NO_DEPS` on all vcs and file dependencies as resolution happens on these in a pre-lock step. pypa#3763
-   Improved verbose logging output during `pipenv lock` will now stream output to the console while maintaining a spinner. pypa#3810
-   Added support for automatic python installs via `asdf` and associated `PIPENV_DONT_USE_ASDF` environment variable. pypa#4018
-   Pyenv/asdf can now be used whether or not they are available on PATH. Setting `PYENV_ROOT`/`ASDF_DIR` in a `.env` file allows Pipenv to install an interpreter without any shell customizations, so long as pyenv/asdf is installed. pypa#4245
-   Added `--key` command line parameter for including personal PyUp.io API tokens when running `pipenv check`. pypa#4257

Behavior Changes
----------------

-   Make conservative checks of known exceptions when subprocess returns output, so user won\'t see the whole traceback - just the error. pypa#2553
-   Do not touch Pipfile early and rely on it so that one can do `pipenv sync` without a Pipfile. pypa#3386
-   Re-enable `--help` option for `pipenv run` command. pypa#3844
-   Make sure `pipenv lock -r --pypi-mirror {MIRROR_URL}` will respect the pypi-mirror in requirements output. pypa#4199

Bug Fixes
---------

-   Raise `PipenvUsageError` when \[\[source\]\] does not contain url field. pypa#2373
-   Fixed a bug which caused editable package resolution to sometimes fail with an unhelpful setuptools-related error message. pypa#2722
-   Fixed an issue which caused errors due to reliance on the system utilities `which` and `where` which may not always exist on some
    systems.
-   Fixed a bug which caused periodic failures in python discovery when executables named `python` were not present on the target `$PATH`. pypa#2783
-   Dependency resolution now writes hashes for local and remote files to the lockfile. pypa#3053
-   Fixed a bug which prevented `pipenv graph` from correctly showing all dependencies when running from within `pipenv shell`. pypa#3071
-   Fixed resolution of direct-url dependencies in `setup.py` files to respect `PEP-508` style URL dependencies. pypa#3148
-   Fixed a bug which caused failures in warning reporting when running pipenv inside a virtualenv under some circumstances.
-   Fixed a bug with package discovery when running `pipenv clean`. pypa#3298
-   Quote command arguments with carets (`^`) on Windows to work around unintended shell escapes. pypa#3307
-   Handle alternate names for UTF-8 encoding. pypa#3313
-   Abort pipenv before adding the non-exist package to Pipfile. pypa#3318
-   Don\'t normalize the package name user passes in. pypa#3324
-   Fix a bug where custom virtualenv can not be activated with pipenv shell pypa#3339
-   Fix a bug that `--site-packages` flag is not recognized. pypa#3351
-   Fix a bug where `pipenv --clear` is not working pypa#3353
-   Fix unhashable type error during `$ pipenv install --selective-upgrade` pypa#3384
-   Dependencies with direct `PEP508` compliant VCS URLs specified in their `install_requires` will now be successfully locked during the resolution process. pypa#3396
-   Fixed a keyerror which could occur when locking VCS dependencies in
    some cases. pypa#3404
-   Fixed a bug that `ValidationError` is thrown when some fields are missing in source section. pypa#3427
-   Updated the index names in lock file when source name in Pipfile is changed. pypa#3449
-   Fixed an issue which caused `pipenv install --help` to show duplicate entries for `--pre`. pypa#3479
-   Fix bug causing `[SSL: CERTIFICATE_VERIFY_FAILED]` when Pipfile `[[source]]` has `verify_ssl=false` and url with custom port. pypa#3502
-   Fix `sync --sequential` ignoring `pip install` errors and logs. pypa#3537
-   Fix the issue that lock file can\'t be created when `PIPENV_PIPFILE` is not under working directory. pypa#3584
-   Pipenv will no longer inadvertently set `editable=True` on all vcs dependencies. pypa#3647
-   The `--keep-outdated` argument to `pipenv install` and `pipenv lock` will now drop specifier constraints when encountering editable dependencies.
    -   In addition, `--keep-outdated` will retain specifiers that would otherwise be dropped from any entries that have not been updated. pypa#3656
-   Fixed a bug which sometimes caused pipenv to fail to respect the `--site-packages` flag when passed with `pipenv install`. pypa#3718
-   Normalize the package names to lowercase when comparing used and in-Pipfile packages. pypa#3745
-   `pipenv update --outdated` will now correctly handle comparisons between pre/post-releases and normal releases. pypa#3766
-   Fixed a `KeyError` which could occur when pinning outdated VCS dependencies via `pipenv lock --keep-outdated`. pypa#3768
-   Resolved an issue which caused resolution to fail when encountering poorly formatted `python_version` markers in `setup.py` and `setup.cfg` files. pypa#3786
-   Fix a bug that installation errors are displayed as a list. pypa#3794
-   Update `pythonfinder` to fix a problem that `python.exe` will be mistakenly chosen for virtualenv creation under WSL. pypa#3807
-   Fixed several bugs which could prevent editable VCS dependencies from being installed into target environments, even when reporting
    successful installation. pypa#3809
-   `pipenv check --system` should find the correct Python interpreter when `python` does not exist on the system. pypa#3819
-   Resolve the symlinks when the path is absolute. pypa#3842
-   Pass `--pre` and `--clear` options to `pipenv update --outdated`. pypa#3879
-   Fixed a bug which prevented resolution of direct URL dependencies which have PEP508 style direct url VCS sub-dependencies with
    subdirectories. pypa#3976
-   Honor `PIPENV_SPINNER` environment variable pypa#4045
-   Fixed an issue with `pipenv check` failing due to an invalid API key from `pyup.io`. pypa#4188
-   Fixed a bug which caused versions from VCS dependencies to be included in `Pipfile.lock` inadvertently. pypa#4217
-   Fixed a bug which caused pipenv to search non-existent virtual environments for `pip` when installing using `--system`. pypa#4220
-   `Requires-Python` values specifying constraint versions of python starting from `1.x` will now be parsed successfully. pypa#4226
-   Fix a bug of `pipenv update --outdated` that can\'t print output correctly. pypa#4229
-   Fixed a bug which caused pipenv to prefer source distributions over wheels from `PyPI` during the dependency resolution phase. Fixed an issue which prevented proper build isolation using `pep517` based builders during dependency resolution. pypa#4231
-   Don\'t fallback to system Python when no matching Python version is found. pypa#4232

Vendored Libraries
------------------

- Updated `pip_shims` to support `--outdated` with new pip versions. pypa#3766
- Update vendored dependencies and invocations
  - Update vendored and patched dependencies
  - Update patches on `piptools`, `pip`, `pip-shims`, `tomlkit`
  - Fix invocations of dependencies
  - Fix custom `InstallCommand` instantiation
  - Update `PackageFinder` usage
  - Fix `Bool` stringify attempts from `tomlkit`
  - Updated vendored dependencies:
    -   **attrs**: `18.2.0 => `19.1.0`
    -   **certifi**: `2018.10.15 => `2019.3.9`
    -   **cached\_property**: `1.4.3 => `1.5.1`
    -   **cerberus**: `1.2.0 => `1.3.1`
    -   **click**: `7.0.0 => `7.1.1`
    -   **click-completion**: `0.5.0 => `0.5.1`
    -   **colorama**: `0.3.9 => `0.4.3`
    -   **contextlib2**: `(new) => `0.6.0.post1`
    -   **distlib**: `0.2.8 => `0.2.9`
    -   **funcsigs**: `(new) => `1.0.2`
    -   **importlib\_metadata** `1.3.0 => `1.5.1`
    -   **importlib-resources**: `(new) => `1.4.0`
    -   **idna**: `2.7 => `2.9`
    -   **jinja2**: `2.10.0 => `2.11.1`
    -   **markupsafe**: `1.0 => `1.1.1`
    -   **more-itertools**: `(new) => `5.0.0`
    -   **orderedmultidict**: `(new) => `1.0`
    -   **packaging**: `18.0 => `19.0`
    -   **parse**: `1.9.0 => `1.15.0`
    -   **pathlib2**: `2.3.2 => `2.3.3`
    -   **pep517**: `(new) => `0.5.0`
    -   **pexpect**: `4.6.0 => `4.8.0`
    -   **pip-shims**: `0.2.0 => `0.5.1`
    -   **pipdeptree**: `0.13.0 => `0.13.2`
    -   **pyparsing**: `2.2.2 => `2.4.6`
    -   **python-dotenv**: `0.9.1 => `0.10.2`
    -   **pythonfinder**: `1.1.10 => `1.2.2`
    -   **pytoml**: `(new) => `0.1.20`
    -   **requests**: `2.20.1 => `2.23.0`
    -   **requirementslib**: `1.3.3 => `1.5.4`
    -   **scandir**: `1.9.0 => `1.10.0`
    -   **shellingham**: `1.2.7 => `1.3.2`
    -   **six**: `1.11.0 => `1.14.0`
    -   **tomlkit**: `0.5.2 => `0.5.11`
    -   **urllib3**: `1.24 => `1.25.8`
    -   **vistir**: `0.3.0 => `0.5.0`
    -   **yaspin**: `0.14.0 => `0.14.3`
    -   **zipp**: `0.6.0`
    - Removed vendored dependency **cursor**. pypa#4169

-   Add and update vendored dependencies to accommodate `safety` vendoring:
    -   **safety** `(none)` => `1.8.7`
    -   **dparse** `(none)` => `0.5.0`
    -   **pyyaml** `(none)` => `5.3.1`
    -   **urllib3** `1.25.8` => `1.25.9`
    -   **certifi** `2019.11.28` => `2020.4.5.1`
    -   **pyparsing** `2.4.6` => `2.4.7`
    -   **resolvelib** `0.2.2` => `0.3.0`
    -   **importlib-metadata** `1.5.1` => `1.6.0`
    -   **pip-shims** `0.5.1` => `0.5.2`
    -   **requirementslib** `1.5.5` => `1.5.6` pypa#4188

-   Updated vendored `pip` => `20.0.2` and `pip-tools` => `5.0.0`. pypa#4215
-   Updated vendored dependencies to latest versions for security and bug fixes:
    -   **requirementslib** `1.5.8` => `1.5.9`
    -   **vistir** `0.5.0` => `0.5.1`
    -   **jinja2** `2.11.1` => `2.11.2`
    -   **click** `7.1.1` => `7.1.2`
    -   **dateutil** `(none)` => `2.8.1`
    -   **backports.functools\_lru\_cache** `1.5.0` => `1.6.1`
    -   **enum34** `1.1.6` => `1.1.10`
    -   **toml** `0.10.0` => `0.10.1`
    -   **importlib\_resources** `1.4.0` => `1.5.0` pypa#4226
-   Changed attrs import path in vendored dependencies to always import from `pipenv.vendor`. pypa#4267

Improved Documentation
----------------------

-   Added documenation about variable expansion in `Pipfile` entries. pypa#2317
-   Consolidate all contributing docs in the rst file pypa#3120
-   Update the out-dated manual page. pypa#3246
-   Move CLI docs to its own page. pypa#3346
-   Replace (non-existant) video on docs index.rst with equivalent gif. pypa#3499
-   Clarify wording in Basic Usage example on using double quotes to escape shell redirection pypa#3522
-   Ensure docs show navigation on small-screen devices pypa#3527
-   Added a link to the TOML Spec under General Recommendations & Version Control to clarify how Pipfiles should be written. pypa#3629
-   Updated the documentation with the new `pytest` entrypoint. pypa#3759
-   Fix link to GIF in README.md demonstrating Pipenv\'s usage, and add descriptive alt text. pypa#3911
-   Added a line describing potential issues in fancy extension. pypa#3912
-   Documental description of how Pipfile works and association with Pipenv. pypa#3913
-   Clarify the proper value of `python_version` and `python_full_version`. pypa#3914
-   Write description for `--deploy` extension and few extensions differences. pypa#3915
-   More documentation for `.env` files pypa#4100
-   Updated documentation to point to working links. pypa#4137
-   Replace docs.pipenv.org with pipenv.pypa.io pypa#4167
-   Added functionality to check spelling in documentation and cleaned up existing typographical issues. pypa#4209
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Category: Dependency Resolution Issue relates to dependency resolution. Type: Bug 🐛 This issue is a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants