jedi#goto() uses python_host_prog not current active virtualenv

[martinpelikan]

Issue

I'm using jedi-vim for goto functionality alongside deoplete-jedi for completions. I have disabled jedi-vim's completions to accomplish this, as seen in my vimrc here. Autocompletion appears to work in my use case, but goto does not.

I recently used pyenv and pyenv-virtualenv as described here to set up a virtualenv specific to neovim, named neovim3. You can see I have this set in my g:python3_host_prog.

I also have a virtualenv set for each project I manipulate. For the sake of example let's say I have a Tornado app that I install in a virtualenv named myapp. Installing this via my setup.py will install all the requirements, such as Tornado.

When editing my code, pyenv has correctly activated the virtualenv named myapp when I switched to that directory. I load up vim, which uses the Python interpreter found in the neovim3 virtualenv that has the neovim Python client and other tools.

Now, when I try autocompletion with something like the following

from tornado import gen
gen.<tab>

I get completions for everything defined and imported in the scope of that gen module, from absolute_import through to YieldPoint. I pick one of these classes or methods, such as gen.sleep and want to jump to its definition by hitting <leader>d and I get this error: jedi-vim: Couldn't find any definitions for this.

At this point I'm confused, since I thought deoplete-jedi uses jedi for its completion list. As an experiment I install myapp in the neovim3 virtualenv, and suddenly goto starts working for all the Tornado packages/modules.

So it looks like jedi looks for definitions only in the scope of what's available in neovim3.

Since I've recently started trying to adopt best practices relating to virtualenvs, I would prefer not to install all of my projects into the neovim3 virtualenv, and only keep neovim/plugin requirements in there. What then is my best option? Am I doing something wrong in my configuration or is this simply not a supported case?

Steps to reproduce

As above, my vim config is available here

Output of “:verbose JediDebugInfo”

Jedi-vim debug information

Using Python version: 3

• sys.version: 3.6.1 (default, Jul 11 2017, 10:53:38), [GCC 7.1.1 20170621]
• site module: /home/mpelikan/.pyenv/versions/3.6.1/lib/python3.6/site.py
  Jedi path: /home/mpelikan/dotfiles/neovim/.config/nvim/plugged/jedi-vim/jedi/jedi/__init__.py
• version: 0.10.2
• sys_path:
  • /home/mpelikan/.pyenv/versions/3.4.2/envs/pvc_appliance/lib/python3.6/site-packages
  • /home/mpelikan/dotfiles/neovim/.config/nvim/plugged/jedi-vim
  • /home/mpelikan/.pyenv/versions/3.6.1/lib/python36.zip
  • /home/mpelikan/.pyenv/versions/3.6.1/lib/python3.6
  • /home/mpelikan/.pyenv/versions/3.6.1/lib/python3.6/lib-dynload
  • /home/mpelikan/.pyenv/versions/neovim3/lib/python3.6/site-packages
  • _vim_path_
• jedi-vim git version: 6411de0
• jedi git submodule status: 5427b02712828b2875d35b5ee1c8b5e58f820537 jedi (v0.10.2)

Settings

g:jedi#force_py_version = '3' (default: 'auto')
g:jedi#completions_enabled = 0 (default: 1)


  omnifunc=pythoncomplete#Complete
	Last set from /usr/share/nvim/runtime/ftplugin/python.vim
  completeopt=menuone,longest,preview
	Last set from ~/dotfiles/neovim/.config/nvim/plugged/jedi-vim/plugin/jedi.vim

:version

NVIM v0.2.0
Build type: Release
Compilation: /usr/bin/cc -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -Wconversion -DNVIM_MSGPACK_HAS_FLOAT32 -O2 -DNDEBUG -DDISABLE_LOG -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wvla -fstack-protector-strong -fdiagnostics-color=auto -DINCLUDE_GENERATED_DECLARATIONS -D_GNU_SOURCE -I/build/neovim/src/build/config -I/build/neovim/src/neovim-0.2.0/src -I/usr/include -I/usr/include -I/usr/include -I/usr/include -I/usr/include -I/usr/include -I/usr/include -I/build/neovim/src/build/src/nvim/auto -I/build/neovim/src/build/include
Compiled by builduser

Optional features included (+) or not (-): +acl   +iconv    +jemalloc +tui      
For differences from Vim, see :help vim-differences

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/usr/share/nvim"

:messages

"sequencersession.py" 710L, 28007C

:scriptnames

  1: ~/dotfiles/neovim/.config/nvim/init.vim
  2: ~/dotfiles/neovim/.config/nvim/config/builtins.vim
  3: ~/dotfiles/neovim/.config/nvim/config/plugins.vim
  4: ~/dotfiles/neovim/.config/nvim/autoload/plug.vim
  5: /usr/share/nvim/runtime/filetype.vim
  6: /usr/share/vim/vimfiles/ftdetect/augeas.vim
  7: ~/dotfiles/neovim/.config/nvim/plugged/vim-fish/ftdetect/fish.vim
  8: ~/dotfiles/neovim/.config/nvim/plugged/vim-markdown/ftdetect/markdown.vim
  9: /usr/share/nvim/runtime/ftplugin.vim
 10: /usr/share/nvim/runtime/indent.vim
 11: /usr/share/nvim/runtime/syntax/syntax.vim
 12: /usr/share/nvim/runtime/syntax/synload.vim
 13: /usr/share/nvim/runtime/syntax/syncolor.vim
 14: ~/dotfiles/neovim/.config/nvim/plugged/neovim-colors-solarized-truecolor-only/colors/solarized.vim
 15: ~/dotfiles/neovim/.config/nvim/config/mappings.vim
 16: ~/dotfiles/neovim/.config/nvim/config/types.vim
 17: /usr/share/vim/vimfiles/plugin/fzf.vim
 18: ~/dotfiles/neovim/.config/nvim/plugged/FastFold/plugin/fastfold.vim
 19: ~/dotfiles/neovim/.config/nvim/plugged/FoldText/plugin/FoldText.vim
 20: ~/dotfiles/neovim/.config/nvim/plugged/deoplete.nvim/plugin/deoplete.vim
 21: ~/dotfiles/neovim/.config/nvim/plugged/deoplete.nvim/autoload/deoplete.vim
 22: ~/dotfiles/neovim/.config/nvim/plugged/deoplete.nvim/autoload/deoplete/init.vim
 23: /usr/share/nvim/runtime/autoload/provider/python3.vim
 24: /usr/share/nvim/runtime/autoload/provider/pythonx.vim
 25: /usr/share/nvim/runtime/autoload/remote/host.vim
 26: /usr/share/nvim/runtime/plugin/rplugin.vim
 27: ~/.local/share/nvim/rplugin.vim
 28: /usr/share/nvim/runtime/autoload/remote/define.vim
 29: ~/dotfiles/neovim/.config/nvim/plugged/deoplete.nvim/autoload/deoplete/mapping.vim
 30: ~/dotfiles/neovim/.config/nvim/plugged/deoplete.nvim/autoload/deoplete/util.vim
 31: ~/dotfiles/neovim/.config/nvim/plugged/deoplete.nvim/autoload/deoplete/handler.vim
 32: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/plugin/airline.vim
 33: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline.vim
 34: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/init.vim
 35: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/parts.vim
 36: ~/dotfiles/neovim/.config/nvim/plugged/vim-sort-motion/plugin/sort_motion.vim
 37: ~/dotfiles/neovim/.config/nvim/plugged/jedi-vim/plugin/jedi.vim
 38: ~/dotfiles/neovim/.config/nvim/plugged/tabular/plugin/Tabular.vim
 39: ~/dotfiles/neovim/.config/nvim/plugged/vim-indexed-search/plugin/IndexedSearch.vim
 40: ~/dotfiles/neovim/.config/nvim/plugged/fzf.vim/plugin/fzf.vim
 41: ~/dotfiles/neovim/.config/nvim/plugged/vim-dirvish/plugin/dirvish.vim
 42: ~/dotfiles/neovim/.config/nvim/plugged/vim-sneak/plugin/sneak.vim
 43: ~/dotfiles/neovim/.config/nvim/plugged/tagbar/plugin/tagbar.vim
 44: ~/dotfiles/neovim/.config/nvim/plugged/vim-signify/plugin/signify.vim
 45: ~/dotfiles/neovim/.config/nvim/plugged/vim-indent-object/plugin/indent-object.vim
 46: ~/dotfiles/neovim/.config/nvim/plugged/vim-better-whitespace/plugin/better-whitespace.vim
 47: ~/dotfiles/neovim/.config/nvim/plugged/vim-devicons/plugin/webdevicons.vim
 48: ~/dotfiles/neovim/.config/nvim/plugged/vim-livedown/plugin/livedown.vim
 49: ~/dotfiles/neovim/.config/nvim/plugged/vim-mundo/plugin/mundo.vim
 50: ~/dotfiles/neovim/.config/nvim/plugged/SimpylFold/plugin/SimpylFold.vim
 51: ~/dotfiles/neovim/.config/nvim/plugged/vim-commentary/plugin/commentary.vim
 52: ~/dotfiles/neovim/.config/nvim/plugged/vim-eunuch/plugin/eunuch.vim
 53: ~/dotfiles/neovim/.config/nvim/plugged/vim-fugitive/plugin/fugitive.vim
 54: ~/dotfiles/neovim/.config/nvim/plugged/vim-sensible/plugin/sensible.vim
 55: ~/dotfiles/neovim/.config/nvim/plugged/vim-surround/plugin/surround.vim
 56: ~/dotfiles/neovim/.config/nvim/plugged/vim-unimpaired/plugin/unimpaired.vim
 57: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline-themes/plugin/airline-themes.vim
 58: ~/dotfiles/neovim/.config/nvim/plugged/ale/plugin/ale.vim
 59: ~/dotfiles/neovim/.config/nvim/plugged/ale/autoload/ale.vim
 60: ~/dotfiles/neovim/.config/nvim/plugged/targets.vim/plugin/targets.vim
 61: /usr/share/nvim/runtime/plugin/gui_shim.vim
 62: /usr/share/nvim/runtime/plugin/gzip.vim
 63: /usr/share/nvim/runtime/plugin/health.vim
 64: /usr/share/nvim/runtime/plugin/man.vim
 65: /usr/share/nvim/runtime/plugin/matchit.vim
 66: /usr/share/nvim/runtime/plugin/matchparen.vim
 67: /usr/share/nvim/runtime/plugin/netrwPlugin.vim
 68: /usr/share/nvim/runtime/plugin/rrhelper.vim
 69: /usr/share/nvim/runtime/plugin/shada.vim
 70: /usr/share/nvim/runtime/plugin/spellfile.vim
 71: /usr/share/nvim/runtime/plugin/tarPlugin.vim
 72: /usr/share/nvim/runtime/plugin/tohtml.vim
 73: /usr/share/nvim/runtime/plugin/tutor.vim
 74: /usr/share/nvim/runtime/plugin/zipPlugin.vim
 75: ~/dotfiles/neovim/.config/nvim/plugged/tabular/after/plugin/TabularMaps.vim
 76: ~/dotfiles/neovim/.config/nvim/plugged/tabular/autoload/tabular.vim
 77: ~/dotfiles/neovim/.config/nvim/plugged/ale/after/plugin/ale.vim
 78: ~/dotfiles/neovim/.config/nvim/plugged/jedi-vim/ftplugin/python/jedi.vim
 79: ~/dotfiles/neovim/.config/nvim/plugged/jedi-vim/autoload/jedi.vim
 80: ~/dotfiles/neovim/.config/nvim/plugged/vim-autopep8/ftplugin/python_autopep8.vim
 81: ~/dotfiles/neovim/.config/nvim/plugged/SimpylFold/ftplugin/python/SimpylFold.vim
 82: ~/dotfiles/neovim/.config/nvim/plugged/SimpylFold/autoload/SimpylFold.vim
 83: /usr/share/nvim/runtime/ftplugin/python.vim
 84: ~/dotfiles/neovim/.config/nvim/plugged/vim-textobj-python/after/ftplugin/python/textobj-python.vim
 85: ~/dotfiles/neovim/.config/nvim/plugged/vim-textobj-user/autoload/textobj/user.vim
 86: ~/dotfiles/neovim/.config/nvim/plugged/jedi-vim/after/ftplugin/python/jedi.vim
 87: ~/dotfiles/neovim/.config/nvim/plugged/vim-python-pep8-indent/indent/python.vim
 88: /usr/share/nvim/runtime/indent/python.vim
 89: /usr/share/nvim/runtime/syntax/python.vim
 90: ~/dotfiles/neovim/.config/nvim/plugged/jedi-vim/after/syntax/python.vim
 91: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions.vim
 92: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/quickfix.vim
 93: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/netrw.vim
 94: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/hunks.vim
 95: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/tagbar.vim
 96: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/branch.vim
 97: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/virtualenv.vim
 98: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/ale.vim
 99: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/whitespace.vim
100: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/po.vim
101: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/wordcount.vim
102: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/tabline.vim
103: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/tabline/autoshow.vim
104: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/tabline/tabs.vim
105: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/tabline/buffers.vim
106: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/tabline/ctrlspace.vim
107: ~/dotfiles/neovim/.config/nvim/plugged/vim-mundo/autoload/airline/extensions/mundo.vim
108: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/section.vim
109: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/highlighter.vim
110: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline-themes/autoload/airline/themes/solarized.vim
111: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/themes.vim
112: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/util.vim
113: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/builder.vim
114: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/default.vim
115: ~/dotfiles/neovim/.config/nvim/plugged/vim-signify/autoload/sy.vim
116: ~/dotfiles/neovim/.config/nvim/plugged/vim-signify/autoload/sy/util.vim
117: ~/dotfiles/neovim/.config/nvim/plugged/vim-signify/autoload/sy/repo.vim
118: ~/dotfiles/neovim/.config/nvim/plugged/ale/autoload/ale/util.vim
119: ~/dotfiles/neovim/.config/nvim/plugged/ale/autoload/ale/linter.vim
120: ~/dotfiles/neovim/.config/nvim/plugged/ale/ale_linters/python/flake8.vim
121: ~/dotfiles/neovim/.config/nvim/plugged/ale/ale_linters/python/mypy.vim
122: ~/dotfiles/neovim/.config/nvim/plugged/ale/ale_linters/python/pylint.vim
123: ~/dotfiles/neovim/.config/nvim/plugged/ale/autoload/ale/events.vim
124: ~/dotfiles/neovim/.config/nvim/plugged/ale/autoload/ale/cursor.vim
125: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/tabline/buflist.vim
126: ~/dotfiles/neovim/.config/nvim/plugged/vim-devicons/autoload/airline/extensions/tabline/formatters/webdevicons.vim
127: ~/dotfiles/neovim/.config/nvim/plugged/vim-airline/autoload/airline/extensions/tabline/formatters/default.vim
128: ~/dotfiles/neovim/.config/nvim/plugged/tagbar/autoload/tagbar.vim
129: ~/dotfiles/neovim/.config/nvim/plugged/ale/autoload/ale/statusline.vim
130: ~/dotfiles/neovim/.config/nvim/plugged/deoplete.nvim/autoload/deoplete/custom.vim
131: ~/dotfiles/neovim/.config/nvim/plugged/vim-signify/autoload/sy/sign.vim
132: ~/dotfiles/neovim/.config/nvim/plugged/ale/autoload/ale/engine.vim
133: ~/dotfiles/neovim/.config/nvim/plugged/ale/autoload/ale/python.vim
134: ~/dotfiles/neovim/.config/nvim/plugged/ale/autoload/ale/path.vim
135: ~/dotfiles/neovim/.config/nvim/plugged/ale/autoload/ale/sign.vim
136: ~/dotfiles/neovim/.config/nvim/plugged/ale/autoload/ale/list.vim
137: ~/dotfiles/neovim/.config/nvim/plugged/ale/autoload/ale/highlight.vim
138: /usr/share/nvim/runtime/autoload/provider/clipboard.vim

[blueyed]

What is the Python version in your neovim3 virtualenv? (likely different than the host (3.6))
Does it match (MAJOR.MINOR) the one in your project?

Jedi will use $VIRTUAL_ENV, but only if it matches..

[martinpelikan]

Thanks @blueyed , you're right, 3.6.1 in the neovim3 environment and 3.4.2 in the development environment.

Since I want to develop against the same version as what that code will be deployed with, would it make sense to move the neovim3 version to 3.4.2 (can you think of any limitations of doing this even outside of Jedi?)

I think in my desperate attempts I did see mention of pending work that will fix this on the Jedi side. Did I miss any documentation about this behaviour available in the README or wiki for Jedi? Is there some more general way to deal with this for the time being?

For example, have a neovim version neovim342, neovim361, etc. for every version of development venv, detect the current VENV's python version and set the appropriate neovim venv?

[blueyed]

I've made some PR for jedi, which was rejected to use a VIRTUAL_ENV always. You might want to use that patch.
Apart from that, you can install neovim also in the work/development environment, of course.

The best is to have the current Python (3.6) by default, and move your projects over there, too.. ;)

[martinpelikan]

@blueyed You're referring to davidhalter/jedi#829? Shame it wasn't merged and put behind some kind of feature flag or non-default option... Hopefully davidhalter/jedi#385 addresses this in the near future then. Thank you.

As much as I'd like to have everything on latest Python, that is unfortunately not an option :(. Other than your patch, I'm still tempted to see if it's possible to have a neovim venv for every project version, but not sure if this is viable/practical.

[blueyed]

Well, davidhalter/jedi#385 might take some more years maybe unfortunately.

davidhalter/jedi#829 could maybe be improved to have some whitelist and/or a way to enable/disable it easily.

I'm still tempted to see if it's possible to have a neovim venv for every project version

Why not install neovim just per project then?

[martinpelikan]

I just tried your branch with and had the same results (perhaps pyenv does something different than expected with paths? Perhaps it's my mistake).

Why not install neovim just per project then?

I don't have a very compelling answer. I only really need to deal with two Python versions, the latest for internal/personal projects and 3.4.2 for shippable ones. That would mean I could pay the upfront cost of creating two venvs and have vim determine which one to use at startup. I thought maybe I could do this via shelling out to python --version, and just doing some naive string manipulation. That is, if I had any idea of how to script in VimL and assuming my idea even makes sense to have this work.

Looks like neovim has no real dependencies or potential for conflict, so installing in every venv is probably the next best thing then.

[blueyed]

You can use glob() in your .vimrc and look at $VIRTUAL_ENV also.
Maybe https://github.com/blueyed/dotfiles/blob/12ff8d2572e5333ef4a11c0c56f630e8d0e4d13f/vimrc#L3923 is an inspiration (although I am using something else locally already) - I cannot keep track with updating my online configs.. :/

Looks like neovim has no real dependencies or potential for conflict, so installing in every venv is probably the next best thing then.

Yes, it is not that heavyweight.
You only have to do it for the non-default (i.e. old version) also.

Closing the issue then for now.