Auto-generating syntax files and command comments

[kmk3]

Background

Currently, each list on firejail.vim is generated/updated by manually running a
set of commands that try to find all applicable items by running regexes and
then pasting the result.

I managed to create a few targets to automate this (WIP), but I noticed that
the regexes for profile.c miss some commands, as they parse the actual source
code and what they look for is very specific.

For example, the following code from firejail.vim:

" Commands grabbed from: src/firejail/profile.c
" Generate list with: { sed -En 's/.*strn?cmp\(ptr, "([^"]+) ".*/\1/p' src/firejail/profile.c; echo private-lib; } | grep -Ev '^(include|ignore|caps\.drop|caps\.keep|protocol|restrict-namespaces|seccomp|seccomp\.drop|seccomp\.keep|env|rmenv|net|ip)$' | sort -u | tr '\n' '|' # private-lib is special-cased in the code and doesn't match the regex; grep-ed patterns are handled later with 'syn match nextgroup=' directives (except for include which is special-cased as a fjCommandNoCond keyword)
syn match fjCommand /\v(apparmor|bind|blacklist|blacklist-nolog|cpu|defaultgw|dns|hostname|hosts-file|ip6|iprange|join-or-start|mac|mkdir|mkfile|mtu|name|netfilter|netfilter6|netmask|nice|noblacklist|noexec|nowhitelist|overlay-named|private|private-bin|private-cwd|private-etc|private-home|private-lib|private-opt|private-srv|read-only|read-write|rlimit-as|rlimit-cpu|rlimit-fsize|rlimit-nofile|rlimit-nproc|rlimit-sigpending|timeout|tmpfs|veth-name|whitelist|xephyr-screen) / skipwhite contained

It has the following comment:

private-lib is special-cased in the code and doesn't match the regex; grep-ed
patterns are handled later with 'syn match nextgroup=' directives (except for
include which is special-cased as a fjCommandNoCond keyword)

Having to create and maintain workarounds makes it harder to ensure that each
list is complete and that it does not have extraneous commands.

Proposal

So to make the regexes less brittle (and therefore easier to maintain), for
every command being parsed on profile.c I propose adding a comment above it
that names the command and specifies whether it has any arguments or not (which
is currently the main way that commands are grouped in firejail.vim).

For example:

No arguments:

	// @profile_command quiet
	else if (strncmp(ptr, "quiet", 5) == 0) {
		// [...]
	}

Single argument:

	// @profile_command mkdir dirname
	else if (strncmp(ptr, "mkdir ", 6) == 0) {
		// [...]
	}

This should make the regex matching much simpler and less error-prone.

For example:

# Commands with no arguments
sed -En 's|// @profile_command ([^ ]+)$|\1|p' src/firejail/profile.c

# Commands with 1 argument
sed -En 's|// @profile_command ([^ ]+) ([^ ]+)|\1|p' src/firejail/profile.c

More comment examples:

// @cli_option --mkdir=dirname // (for main.c; just for illustration)
// @profile_command mkdir dirname
// @profile_conditional ?HAS_APPIMAGE

Note: The items in the other source files (such as in errno.c) seem to follow a
simpler format, so their regexes can maybe be left as is.

Considerations

This would require ensuring that whenever a command is added/modified/removed,
that the same is done to its comment, in order to keep the syntax lists
up-to-date. And also running make (or something like make syntax) after
doing that.

Before:

• Add/modify/remove a command
• Remember to check firejail.vim
• Find and copy the correct list generation commands from firejail.vim
• Run the commands
• Paste the output back into firejail.vim, replacing the appropriate line
• Run git diff and check that the command was added/modified/removed in
  firejail.vim
• Commit

After:

• Add/modify/remove a command and its comment
• Run make (or make syntax)
• Run git diff and check that the command was added/modified/removed in the
  syntax files
• Commit

Note: The above could be checked also in CI.

Note2: If the auto-generation of syntax files works reliably, it should be much
easier to maintain multiple syntax files (such as the one from #5502).

Future Work

Maybe the command comments could be expanded later to also include a brief
description in order to also generate --help (in usage.c), but for now this is
just the simplest way that I could think of to generate the syntax files in a
consistent manner.

---

Relates to:

• Add vim syntax and ftdetect files #2679
• Add basic gtksourceview language-spec (file type detection/syntax highlighting for profiles) #5502
• build: make shell commands more portable in firejail.vim #5577
• build: auto-generate syntax files #5627

Cc: @glitsj16 @laomaiweng @netblue30 @reinerh @rusty-snake @smitsohu (as
contributors to firejail.vim)

[glitsj16]

FWIW I support this proposal. Generating the syntax files consistently would be a nice and welcome additional feature.

[kmk3]

PR opened for the auto-generation:

• build: auto-generate syntax files #5627