pipenv --three fails

[tjquillan]

Issue description

Running pipenv --three throws 'TypeError: 'NoneType' object is not iterable'

Expected result

Pipenv to be initialized with python three

Actual result

pipenv --three fails and throws an error. A log file will be included as the error is quite long.
log.txt

Steps to replicate

Run pipenv --three

---

$ pipenv --support

Pipenv version: '2018.11.14'

Pipenv location: '/usr/lib/python3.7/site-packages/pipenv'

Python location: '/usr/bin/python'

Python installations found:

Traceback (most recent call last):
File "/usr/bin/pipenv", line 11, in
load_entry_point('pipenv==2018.11.14', 'console_scripts', 'pipenv')()
File "/usr/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 764, in call
return self.main(*args, **kwargs)
File "/usr/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/usr/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1114, in invoke
return Command.invoke(self, ctx)
File "/usr/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/usr/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
return f(get_current_context(), args, **kwargs)
File "/usr/lib/python3.7/site-packages/pipenv/cli/command.py", line 142, in cli
get_pipenv_diagnostics()
File "/usr/lib/python3.7/site-packages/pipenv/help.py", line 33, in get_pipenv_diagnostics
python_paths = finder.find_all_python_versions()
File "/usr/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 122, in find_all_python_versions
python_version_dict = getattr(self.system_path, "python_version_dict")
File "/usr/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 54, in system_path
ignore_unsupported=self.ignore_unsupported,
File "/usr/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 451, in create
ignore_unsupported=ignore_unsupported,
File "<attrs generated init 75e45f144e3d5510d54dd5fca6730b98fa0220a9>", line 38, in init
File "/usr/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 116, in attrs_post_init
self._setup_pyenv()
File "/usr/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 196, in _setup_pyenv
version_glob_path="versions/", ignore_unsupported=self.ignore_unsupported
File "/usr/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 154, in create
sort_function=sort_function, version_glob_path=version_glob_path)
File "<attrs generated init 4cb48a43d6709d3a578ba9505d6f7e6efcc13cef>", line 17, in init
File "/usr/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 98, in get_versions
for p in self.get_version_order():
File "/usr/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 70, in get_version_order
version_order = [versions[v] for v in parse_pyenv_version_order()]
TypeError: 'NoneType' object is not iterable

[frostming]

Thanks for reporting, I believe it is fixed in master branch, can you have a try?

[SolarLiner]

Tried the master branch, still produces the same error.

$ pipenv --support

Pipenv version: '2018.11.15.dev0'

Pipenv location: '/home/solarliner/.local/lib/python3.6/site-packages/pipenv'

Python location: '/usr/bin/python3'

Python installations found:

Traceback (most recent call last):
  File "/home/solarliner/.local/bin/pipenv", line 11, in <module>
    load_entry_point('pipenv==2018.11.15.dev0', 'console_scripts', 'pipenv')()
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1114, in invoke
    return Command.invoke(self, ctx)
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/cli/command.py", line 142, in cli
    get_pipenv_diagnostics()
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/help.py", line 33, in get_pipenv_diagnostics
    python_paths = finder.find_all_python_versions()
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 122, in find_all_python_versions
    python_version_dict = getattr(self.system_path, "python_version_dict")
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 54, in system_path
    ignore_unsupported=self.ignore_unsupported,
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 451, in create
    ignore_unsupported=ignore_unsupported,
  File "<attrs generated init 75e45f144e3d5510d54dd5fca6730b98fa0220a9>", line 38, in __init__
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 116, in __attrs_post_init__
    self._setup_pyenv()
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 196, in _setup_pyenv
    version_glob_path="versions/*", ignore_unsupported=self.ignore_unsupported
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 154, in create
    sort_function=sort_function, version_glob_path=version_glob_path)
  File "<attrs generated init d1562b3dd8a63d3504f390eb8f2d396673d146ac>", line 17, in __init__
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 98, in get_versions
    for p in self.get_version_order():
  File "/home/solarliner/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 70, in get_version_order
    version_order = [versions[v] for v in parse_pyenv_version_order() if v in versions]
TypeError: 'NoneType' object is not iterable

Rolling back to 2018.10.13 fixed the issue.

[rgant]

At least on my system pyenv doesn't have the file ~/.pyenv/version. Both pipenv and pyenv installed using Homebrew.

[jxltom]

parse_pyenv_version_order should return [] instead of None when pyenv version order file does not exist.

[jxltom]

A workaround is to to create pyenv version file such as by pyenv global system until fix is released.

[rgant]

I just tried that @jxltom and got:

  File "/usr/local/Cellar/pipenv/2018.11.14/libexec/lib/python3.7/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 70, in <listcomp>
    version_order = [versions[v] for v in parse_pyenv_version_order()]
KeyError: 'system'

[rgant]

Using a version number works. pyenv global 3.7.1

[jxltom]

For using pyenv global system, you have to use pipenv master branch, sorry for the trouble.

Yeah, pyenv global 3.7.1 or an empty version file will work for latest released pipenv. 😄

[rgant]

After pipenv installed the version of python in my Pipfile I got this error:

Warning: The Python you just installed is not available on your PATH, apparently.

However running pyenv rehash and pipenv sync --dev again results in the creation of my virtualenv in the correct python version.

[jxltom]

@rgant Are your sure 3.7.1 is installed by pyenv indeed?

And can you show output of pipenv --three --verbose? So it will be easier to debug.

[rgant]

Sorry @jxltom I'm trying to get something done ASAP and don't really have time to assist with debugging tonight. All of my pythons are installed by pyenv. And I didn't make it clear that while 3.7.1 was a python already installed, my Pipfile specified a different version 3.6.4.

These are the commands I ran tonight after setting the global pyenv version.

$ pipenv sync --dev
Warning: Python 3.6.4 was not found on your system…
Would you like us to install CPython 3.6.4 with pyenv? [Y/n]: Y
Installing CPython 3.6.4 with pyenv (this may take a few minutes)…
✔ Success! 
python-build: use openssl from homebrew
python-build: use readline from homebrew
Downloading Python-3.6.4.tar.xz...
-> https://www.python.org/ftp/python/3.6.4/Python-3.6.4.tar.xz
Installing Python-3.6.4...
python-build: use readline from homebrew
Installed Python-3.6.4 to /Users/rgant/.pyenv/versions/3.6.4


Warning: The Python you just installed is not available on your PATH, apparently.
$ pyenv rehash
$ ls -la /Users/rgant/.pyenv/v*
-rw-r--r--  1 rgant  staff  6 Nov 18 21:45 /Users/rgant/.pyenv/version

/Users/rgant/.pyenv/versions:
total 0
drwxr-xr-x  4 rgant  staff  128 Nov 18 21:45 .
drwxr-xr-x  5 rgant  staff  160 Nov 18 21:43 ..
drwxr-xr-x  6 rgant  staff  192 Nov 18 21:47 3.6.4
drwxr-xr-x  6 rgant  staff  192 Nov  6 16:16 3.7.1
$ echo $PATH
./node_modules/.bin:/Users/rgant/.rbenv/shims:/Users/rgant/.pyenv/shims:/Users/rgant/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:.
$ pipenv sync --dev
Creating a virtualenv for this project…
Pipfile: ./Pipfile
Using /Users/rgant/.pyenv/versions/3.6.4/bin/python3 (3.6.4) to create virtualenv…
✔ Complete 
Using base prefix '/Users/rgant/.pyenv/versions/3.6.4'
New python executable in /Users/rgant/.local/share/virtualenvs/blah-TBr8V2Mx/bin/python3
Also creating executable in /Users/rgant/.local/share/virtualenvs/blah-TBr8V2Mx/bin/python
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /Users/rgant/.pyenv/versions/3.6.4/bin/python3
Virtualenv location: /Users/rgant/.local/share/virtualenvs/blah-TBr8V2Mx
Installing dependencies from Pipfile.lock (12f483)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 46/46 — 00:00:12
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
All dependencies are now up-to-date!

[jxltom]

@rgant Thanks, it is already very helpful

[gkapfham]

Hello @rgant and @iboyperson and the others discussing this issue. I've read through the comments on this issue and a host of other issues for pipenv and other Python tools and I'm still not sure how we are supposed to resolve this problem. For reference, if I run pipenv locally on my Ubuntu 16.04 development machine, everything works correctly. However, if I try to run pipenv on Travis CI then I cannot find a way to ensure that the builds pass correctly. Specifically, on Travis CI I always see this error message:

TypeError: 'NoneType' object is not iterable

Here are the relevant portions of the travis.yml configuration file that I am using:

# use the Python 3.6 language
language: python
python:
  - "3.6"

# ignore the virtualenv that Travis creates
# setup a secure login to CodeCov
env:
  global:
    - PIPENV_VENV_IN_PROJECT=1
    - PIPENV_IGNORE_VIRTUALENVS=1
    - LANG=en_US.UTF-8
    - LC_ALL=en_US.UTF-8
    - secure: hnWlE/s8Vo+OM+Vz2p60+/jmMoGwnIdpVg2OUnu269TJYapPG+qF6ks1jYpb/XfRucfndZSZXy4UZfJnzf082hoQmAUGHm3yrRTDkrsGGK1bxea3Rhwj1bn8IrjcZqOAQAtpCsnhFbzrdRTtng8+PCBcYyru+IG6jwOrOv2EMe/CBVLNvSoRdtBgcz4QKl2SRsLhEQh+pjgZiUEY+d7mNfyhGzCD8xS4abj8KfNfKDIZfKHX9E1mvBuZm9GQC17wvLyOt0j4gtEZeoGLGF63fuLvMqZhpZh00lJ6wmSD9vTok7pqgpErR1yrzZmVDrD61bz0PL/b8kHjOLLd0HcOw9q0D0KPX5BojOJxVKglc8GaAI4CksnMmjfLjieZRMag8K8qI8lKkIiD/AABAjqwHJ0Ywm7ru42alY163y4ieatXDMBQpoLsSn1P4gs0LDvGe6Kby+jeM8nL8y+5L5qGTODnqYDHes0rHVMIXQAr7HLKyru6/Bwzz8woYr8WjBmTd1UxLtFcfSYr8Rj+Yklldz4Wt1p4Wnpj9Hd5S2eVLXeteTBKA4y/tFSenZTke+NlVK8zmHa06yVHTbLf14ggTe4hZgUM/mLXRnVFy73LyaBPQkDQpvc6HELMxj94z5OZ6DpDrESDDxVnGq3H876qiJfi/rMFAkTbtgeMu4MRbow=

# install mdl for checking Markdown
before_install:
  - gem install mdl

# command to install dependencies
install:
  - pip install --upgrade pip
  - pip install --upgrade pipenv
  - pipenv install --dev --three

# send emails when there is a change or failure
notifications:
  email:
    on_success: change
    on_failure: always

# use a cache to improve performance
cache: pip

# set GATORGRADER_HOME environment variable
before_script:
  - export GATORGRADER_HOME="/home/travis/build/gkapfham/gatorgrader"

# perform testing:
# --> run the test suite with pytest
# --> lint the functions with flake8
# --> lint the test suites with flake8
# --> lint the functions with pylint
# --> lint the test suites with pylint
# --> lint the README documentation with mdl
script:
  - pipenv run pytest --cov-config pytest.cov --cov
  - pipenv run flake8 tests*
  - pipenv run flake8 gatorgrader*
  - pipenv run pylint tests
  - pipenv run pylint gator
  - mdl README.md

# report coverage information to CodeCov
after_success:
  - pipenv run codecov

I should note that I have also tried things like, for instance, using Python 3.6-dev and not including the setting of the LANG and LC_ALL environment variables (which was suggested in the pipenv issue tracker sometime in 2017, if I recall correctly). The above Travis CI configuration worked throughout all of the past summer and is only now not working when I was changing my Pipfile and Pipfile.lock because of the fact that GitHub detected a security vulnerability in the requests version that I was using.

Can anyone please advise me on how to update my .travis.yml file so that I can again use pipenv? I really appreciate any assistance that you can provide! Thanks in advance!

[maxim-belkin]

#3247 (comment)

an empty version file will work for latest released pipenv.

This is the only solution that worked for me.

[gkapfham]

Hi @maxim-belkin! I saw that point earlier, but, honestly, I do not know why there is a need to specify an empty version file when I am using Python on Travis CI. I see that pyenv uses this version file in its root. Is it the case that Travis CI uses pyenv to install Python versions? (I don't see that in the documentation). Is what you are suggesting a temporary workaround until a pipenv issue is fixed? Or, is this the new standard way to use pipenv? Any follow-on tips that you can provide would be greatly appreciated. As my schedule permits, I will try more alternatives this week and report back on what seemed to work correctly.

[maxim-belkin]

I think there is a bug in pipenv and it expects that file to exist. Specifying anything in that file did not work for me.

Compare the version of pipenv that you have on your computer and on Travis. You probably have a "slightly outdated" (not the very latest) version of pipenv on your local machine whereas Travis installs the latest one. I think if you do pip install -U [--user] pipenv locally, you'll start getting the same error :)

[gkapfham]

While I acknowledge that this issue was already closed, I thought that I should write to confirm that the issue has been resolved for me. After testing a significant number of different configurations, I can confirm that all of my builds on Travis CI and locally are now working correctly. Thanks for releasing a new version of pipenv that resolves the problem!

[frostming]

@gkapfham Thanks for your information, it helps a lot.