Startup error in multi-file document with relative paths

[mgunyho]

Description

I have a tex file in a subfolder that \inputs a file in another subfolder that is relative to the main document. Latexmk complies it fine, but when I open the file in vimtex I get the following error

Error detected while processing function vimtex#init[7]..<SNR>46_init_state[2]..vimtex#state#init_local[13]..vimtex#state#class#new[17]..vimtex#parser#preamble[1]..vimtex#parser#tex#parse_preamble[6]..<SNR>57_parse_preamble[16]..vimtex#parser#tex#input_parser[19]..<SNR>57_input_to_filename[26]..vimtex#kpsewhich#find[22]..vimtex#kpsewhich#run:
line   11:
E712: Argument of filter() must be a List or Dictionary
Error detected while processing function vimtex#init[7]..<SNR>46_init_state[2]..vimtex#state#init_local[13]..vimtex#state#class#new[17]..vimtex#parser#preamble[1]..vimtex#parser#tex#parse_preamble[6]..<SNR>57_parse_preamble[16]..vimtex#parser#tex#input_parser[19]..<SNR>57_input_to_filename[26]..vimtex#kpsewhich#find:
line   22:
E896: Argument of get() must be a List, Dictionary or Blob

The symptoms are similar to #2188, but seems like it's a different problem.

Steps to reproduce

I have the following file structure:

.
├── fig
│   └── some_figure.tex
├── main.tex
└── sections
    └── introduction.tex

with the following contents:

main.tex

\documentclass{minimal}
\begin{document}
    \input{sections/introduction}
\end{document}

sections/introduction.tex

\input{fig/some_figure}

fig/some_figure.tex

this is a figure

(In reality this is a .pdf_tex file generated by Inkscape, but the error is the same.)

minimal.vim

set nocompatible
let &runtimepath  = '~/.vim/plugged/vimtex,' . &runtimepath
let &runtimepath .= ',~/.vim/plugged/vimtex/after'
filetype plugin indent on
syntax enable

Then, running vim -u minimal.vim sections/introduction.tex produces the above error.

I also tried :VimtexClearCache kpsewhich and adding let g:vimtex_cache_root = '.' to the vimrc as discussed in #2188 but it didn't help.

Expected behavior

Vimtex should load the file normally

Actual behavior

I get the error message as above, and vimtex is disabled (\ll does'nt work, no syntax highlighting between \begin{} and \end{}, etc).

Do you use a latexmkrc file?

No

VimtexInfo

This is when I open main.tex (I'm running ubuntu 20.04.3, it's not there in VimtexInfo for some reason):

System info:
  OS:
  Vim version: VIM 8.1 (1-2269)
  Has clientserver: true
  Servername: undefined (vim started without --servername)

VimTeX project: main
  base: main.tex
  root: /home/marci/sandbox/vimtex-debug/test
  tex: /home/marci/sandbox/vimtex-debug/test/main.tex
  main parser: recursive search
  document class: minimal
  source files:
    main.tex
    sections/introduction.tex
    fig/some_figure.tex
  compiler: latexmk
    engine: -pdf
    options:
      -verbose
      -file-line-error
      -synctex=1
      -interaction=nonstopmode
    callback: 1
    continuous: 1
    executable: latexmk
  viewer: General
  qf method: LaTeX logfile

[lervag]

I have a tex file in a subfolder that \inputs a file in another subfolder that is relative to the main document. Latexmk complies it fine, but when I open the file in vimtex I get the following error

The symptoms are similar to #2188, but seems like it's a different problem.

Thanks for the good issue description! Yes, this seems to be different. But I can't reproduce this, unfortunately. Which version of kpsewhich are you using (I'm on v6.3.3)? Could you install neovim and see if you have the same issue there?

Aha... sorry:

System info:
  OS:

That's strange; what is the output of lsb_release -d in your terminal; or uname -sr? This may explain the issue, actually! The kpsewhich utility needs to run an external job, which I've recently refactored and improved. This functionality runs different methods depending on your os, but if we can't decide the os, then that's probably what is happening.

If you open Vim with VimTeX loaded, what is the output of :echo vimtex#util#get_os()?

[mgunyho]

kpsewhich is version 6.3.1.

$ lsb_release -d
Description:    Ubuntu 20.04.3 LTS
$ uname -sr
Linux 5.11.0-40-generic

echo vimtex#util#get_os() prints linux.

I can try with neovim later today.

[lervag]

What's the output of :echo vimtex#jobs#cached('lsb_release -d')? And :echo vimtex#jobs#cached('uname -sr')?

[mgunyho]

Both of those printed nothing.

I installed neovim, and with that the problem didn't appear. However, after that, the problem was also gone with regular vim! The problem appears again if I remove the files in ~/.cache/vimtex. So I guess something related to the cache is not properly initialized with plain vim.

Also, the vimtex#jobs#cached commands work in neovim, except if I have run them in regular vim first. So to summarize:

$ rm -f ~/.cache/vimtex/*
$ vim -u minimal.vim sections/introduction.tex
> error
$ nvim -u minimal.vim sections/introduction.tex
> no error
:echo vimtex#jobs#cached('lsb_release -d')
> ['Description:  Ubuntu 20.04.3 LTS']
$ vim -u minimal.vim sections/introduction.tex
> no error
:echo vimtex#jobs#cached('lsb_release -d')
> ['Description:  Ubuntu 20.04.3 LTS']

$ rm -f ~/.cache/vimtex/*
$ vim -u minimal.vim sections/introduction.tex
> error
:echo vimtex#jobs#cached('lsb_release -d')
> nothing
$ nvim -u minimal.vim sections/introduction.tex
> no error
:echo vimtex#jobs#cached('lsb_release -d')
> nothing
$ vim -u minimal.vim sections/introduction.tex
> no error
:echo vimtex#jobs#cached('lsb_release -d')
> nothing

Here's the contents of the cache after the commands:

$ rm -f ~/.cache/vimtex/*
$ vim -u minimal.vim sections/introduction.tex
$ tail -vn +1 ~/.cache/vimtex/*
==> /home/marci/.cache/vimtex/capture.json <==
{"uname":"","__validate":"cache_v0"}

$ vim -u minimal.vim sections/introduction.tex
:echo vimtex#jobs#cached('lsb_release -d')
> nothing
$ tail -vn +1 ~/.cache/vimtex/*
==> /home/marci/.cache/vimtex/capture.json <==
{"lsb_release -d":"","uname":"","__validate":"cache_v0"}

$ nvim -u minimal.vim sections/introduction.tex
$ tail -vn +1 ~/.cache/vimtex/*
==> /home/marci/.cache/vimtex/capture.json <==
{"lsb_release -d":"","uname":"","__validate":"cache_v0"}

==> /home/marci/.cache/vimtex/kpsewhich.json <==
{"fig/some_figure.tex": [["/home/marci/sandbox/vimtex-debug/test1/fig/some_figure.tex", "/home/marci/sandbox/vimtex-debug/test1"]], "__validate": "cache_v0"}

==> /home/marci/.cache/vimtex/nvim_servernames.log <==
/tmp/nvimkCi9ed/0


$ rm -f ~/.cache/vimtex/*
$ nvim -u minimal.vim sections/introduction.tex
:echo vimtex#jobs#cached('lsb_release -d')
> ['Description:  Ubuntu 20.04.3 LTS']
$ tail -vn +1 ~/.cache/vimtex/*
==> /home/marci/.cache/vimtex/capture.json <==
{"lsb_release -d": ["Description:\tUbuntu 20.04.3 LTS"], "uname": ["Linux"], "__validate": "cache_v0"}

==> /home/marci/.cache/vimtex/kpsewhich.json <==
{"fig/some_figure.tex": [["/home/marci/sandbox/vimtex-debug/test1/fig/some_figure.tex", "/home/marci/sandbox/vimtex-debug/test1"]], "__validate": "cache_v0"}

==> /home/marci/.cache/vimtex/nvim_servernames.log <==
/tmp/nvim05LS2u/0

[lervag]

Thanks for the thorough investigations! Let's focus on Vim; there seems to be a problem here that is hard to locate.

Could you do this (notice capture not cached):

• vim -u minimal.vim main.tex
• :echo vimtex#jobs#capture('uname')
• :echo vimtex#jobs#capture('lsb_release -d')

What's the output now?

[lervag]

It seems you are capable of digging in things yourself. The problematic code seems to be related to the VimTeX jobs api, here:

vimtex/autoload/vimtex/jobs.vim

Lines 46 to 70 in 24cc527

	function! vimtex#jobs#capture(cmd, ...) abort " {{{1
	" Run an external process and capture the command output.
	"
	" The optional argument is a dictionary of options. Each option is parsed in
	" the code below.
	"
	" Return: Command output as list of strings.
	let l:opts = a:0 > 0 ? a:1 : {}
	call vimtex#paths#pushd(get(l:opts, 'cwd', ''))
	let l:output = vimtex#jobs#{s:backend}#capture(a:cmd)
	call vimtex#paths#popd()
	return l:output
	endfunction
	" }}}1
	function! vimtex#jobs#cached(cmd) abort " {{{1
	" Cached version of vimtex#jobs#capture(...)
	let l:cache = vimtex#cache#open('capture')
	return l:cache.has(a:cmd)
	\ ? l:cache.get(a:cmd)
	\ : l:cache.set(a:cmd, vimtex#jobs#capture(a:cmd))
	endfunction

I would guess it is either the caching or the capturing that is the problem. That's the point of my above request. If the captures still yield empty output, then it seems there must be something wrong with this:

vimtex/autoload/vimtex/jobs/vim.vim

Lines 118 to 123 in 24cc527

	function! s:vim_unix_capture(cmd) abort " {{{1
	silent! let l:output = systemlist(a:cmd)
	return v:shell_error == 127 ? ['command not found'] : l:output
	endfunction
	" }}}1

So my next suggestion would be to look at the output of :echo systemlist('uname') and so on.

[mgunyho]

The capture command returns empty for both of those in vim, but they work in nvim. I'll investigate a bit further.

[mgunyho]

Aha. In vim, running :echo systemlist('uname') is giving me errors like Can't open file /tmp/xxxx/0. As suggested on stackoverflow here, this was because I have a weird shell (xonsh). Doing set shell=/bin/bash fixes the problem. But xonsh works fine with nvim. So I guess vim is to blame here.

So all in all, the problem is because of my shell configuration. Sorry about the hassle.

Thanks for the help, and thank you very much for maintaining vimtex!

[lervag]

Great, I suspected that shell could have been the cause, but you provided a minimal vimrc file that did not change the shell option. I forgot that the default value is $SHELL. Perhaps I should update the instructions here.

Or, rather; perhaps I should just reset the shell for VimTeX (user would not notice, but this issue would be resolved without you having to change the shell option).

[mgunyho]

That sounds reasonable, although you would have to be careful to make sure to use a shell that is installed. I don't know if it can be assumed that /bin/bash exists. But then I don't know if /bin/sh can run all the commands you want to run.

[lervag]

I believe my commit should be safe. We don't need bash, sh will suffice.

[mgunyho]

Cool, I can try it a bit later.

[mgunyho]

Yeah, seems like it's working! Can setting the original value of shell fail if systemlist() or system() return an error? Is that possible?

Anyway, thanks again!

[lervag]

Yeah, seems like it's working!

Happy to hear it!

Can setting the original value of shell fail if systemlist() or system() return an error? Is that possible?

No, I don't think so.