Custom syntax cmd gets overwritten on package detection

[lostgeek]

Description

Custom syntax command for concealment breaks after vimtex detects the packages containing those commands.

Steps to reproduce

minimal.vim

set nocompatible
set runtimepath^=~/.vim/bundle/vimtex
set runtimepath+=~/.vim/bundle/vimtex/after
filetype plugin indent on
syntax enable

let g:vimtex_syntax_custom_cmds = [
      \ {'name': 'ch', 'mathmode': 0, 'argspell': 0, 'argstyle':'bold', 'conceal':1},
      \ {'name': 'qty', 'mathmode': 0, 'argspell': 0, 'argstyle':'bold', 'arggreedy':1, 'conceal':1},
      \]

minimal.tex

\documentclass[12pt]{article}

% \usepackage{chemformula}
% \usepackage{siunitx}

\begin{document}
\section*{Example}

\begin{itemize}
    \item asdf
\end{itemize}

\ch{C2H2}

\qty{1}{\metre}

\end{document}

1. Open Vim with vim -u minimal.vim minimal.tex
2. Uncomment imports
3. Start compilation via <leader>ll
4. Observe custom conceal of \ch and \qty commands before and after compilation

Expected behavior

The custom conceal commands to work both before and after package detection.

Actual behavior

The custom conceal commands break when packages chemformula and siunitx are detected.

Do you use a latexmkrc file?

No

VimtexInfo

System info:
  OS: Arch Linux
  Vim version: NVIM v0.8.2
  Has clientserver: true
  Servername: /run/user/1000/nvim.53714.0

VimTeX project: example
  base: example.tex
  root: /home/lostgeek/temp
  tex: /home/lostgeek/temp/minimal.tex
  main parser: current file verified
  document class: article
  compiler: latexmk
    engine: -pdf
    options:
      -verbose
      -file-line-error
      -synctex=1
      -interaction=nonstopmode
    build_dir: build
    callback: 1
    continuous: 1
    executable: latexmk
  viewer: Zathura
    xwin id: 0
  qf method: LaTeX logfile


% After compilation:
  packages: amsbsy amsgen amsmath amsopn amstext array chemformula color epstopdf-base etoolbox expl3 graphics graphicx iftex ifthen infwarerr keyval l3keys2e ltxcmds nicefrac pdftexcmds pgf pgfcomp-version-0-65 pgfcomp-version-1-18 pgfcore pgffor pgfkeys pgfmath pgfrcs pgfsys siunitx textcomp tikz translations trig xcolor xfrac xparse xtemplate

[lervag]

The reason is that VimTeX has syntax addons for these packages:

• https://github.com/lervag/vimtex/blob/master/autoload/vimtex/syntax/p/chemformula.vim
• https://github.com/lervag/vimtex/blob/master/autoload/vimtex/syntax/p/siunitx.vim

These addons are loaded on demand after compilation. Since the addons define syntax for the specified commands, they will overload the user customization.

If my intuition is right, you should have the same issue even if the packages are not commented out when you load the file?

[lostgeek]

Yes, exactly. When the packages are not commented out on load, the same behavior happens.

In the meantime, I found out that I can disable loading those syntax addons via g:vimtex_syntax_packages, which solves this problem for me. But I assume that I'm losing some functionality that the syntax addons offered to me.'

I was wondering whether the custom syntax can be applied after / on top of the predefined syntax addons.

[lervag]

Yes, exactly. When the packages are not commented out on load, the same behavior happens.

Thanks for confirming.

In the meantime, I found out that I can disable loading those syntax addons via g:vimtex_syntax_packages, which solves this problem for me. But I assume that I'm losing some functionality that the syntax addons offered to me.'

Yes, you are correct on both accounts. The functionality you loose is (subjectively) better syntax highlighting of several commands from those packages, including \ch and \qty. However, it does not include any conceal effects.

I was wondering whether the custom syntax can be applied after / on top of the predefined syntax addons.

Well, perhaps, but only partially. The syntax extensions from latex packages are loaded in two different stages:

1. During buffer initialization of VimTeX, and
2. When a document was successfully compiled.

The purpose of 2 is to load syntax extensions after the .fls file is created, because then we are guaranteed to know which packages a document actually uses.

So, what we can do here is to move the custom syntax to the end of stage 1. That should work in most cases, but it may be overriden in stage 2.

One minor issue with this is that it will create an inconsistent behaviour. E.g., with your minimal example. Start with a fresh uncompiled project with the \usepackage lines commented. Then your customizations will apply. Now uncomment, compile, and observe that your customizations are overloaded.
Similarly, if you open the same project with the \usepackage lines uncommented, then your customizations will not become overloaded.

This inconsistency may be harder for people to recognize and debug.

Still, I'm tempted to go with this, because I think it will be closer to what users would expected. Do you agree?

[lostgeek]

I think my favoured solution would be that syntax customisations gets applied both after stage 1 and after stage 2. The solution you suggested creates an inconsistency that is even more likely to create user confusion, as it is unclear why the override works on one stage, but not the other. I had been confused myself, but once realising that package detection caused the issue, the behaviour made sense.

But that might come from my particular use case, where I generally always turn the continuous compilation of the file on while working. So I rarely am in that spot in between stage 1 and 2, where customisations apply.

Of course, I have no idea at all whether my suggestion is easy to implement.

[lervag]

The solution you suggested creates an inconsistency that is even more likely to create user confusion, as it is unclear why the override works on one stage, but not the other. I had been confused myself, but once realising that package detection caused the issue, the behaviour made sense.

Yes, I agree. Choosing between imperfects.

I think my favoured solution would be that syntax customisations gets applied both after stage 1 and after stage 2.

That should be possible, yes. It would add a minor cost, but I think it is negligible. Let's try it; please open an issue if you should find any issues afterwards.

[lostgeek]

Tried out the update and it seems to be working great. Appreciate the effort! ❤️

[lervag]

Glad to hear it!