Add tell/ear interface for CLI access to vmod objects

[nigoroll]

Runtime modification of vmod object properties has been a long standing item on our wishlist. For example, #3652 is about a use case to change a custom director property, which is not covered by the director health state. Another simple example is to instruct a vmod object to emit log messages for tracing only when needed.

This commit implements a basic interface for CLI access to vmod objects:

VMOD objects now can have ears, and the CLI gets a command to tell messages to ears.

vmod ears

VMOD object classes gain a special method type $Ear, which is almost identical to $Method, except that only one ear is supported per class, and only the specific signature

$Ear STRANDS earname(STRANDS)

is supported.

The ear receives input via the single STRANDS argument and can either return NULL for error, or a response as a STRANDS.

For the error case, the ear can write details to ctx->msg.

cli tell command

The tell command takes an optional vcl name, object name and message to send. Individual message arguments are passed as constituents of the STRANDS argument to the ear.

demo

A new test case demos the functionality: The debug.obj class has gained an ear which just returns the instance name followed by the original message:

varnish> help tell
200
tell [<vcl>.]<object> <msg> ...
Tell <msg> to <object> from the given <vcl> or the active vcl

varnish> tell obj0 is there anybody out there?
200
obj0: is there anybody out there?

varnish> tell whoisit hello?
300
No object named whoisit found

varnish> tell obj0 fail
300
You asked me to fail

[nigoroll]

force push: moved from VRT to VPI

[nigoroll]

TODOs based on feedback from @dridi and @bsdphk :

• boringification-rename $Ear -> $Cli
• So the ear is going to be renamed to "cli method"
• change cli method return value to integer
• return string in vsb argument both for success and failure

[nigoroll]

I have implemented the TODOs and force pushed.
The commit message of 0f53a0b reflects the new naming:

• vmod_cli_f for the cli method type
• SYM_CLI_METHOD for the vcc symbol
• $Cli for the vcc file stanza

RST generation in vmodtool is still todo once we have agreement on the current PR.

[dridi]

I like this iteration much better already, and I have a few comments.

[dridi]

AN(n);

[dridi]

Should we expose (or even restrict to) a subset of the VCLI status constants to VMODs?

[dridi]

Maybe make $CLI unconditional like $INIT and $FINI to simplify the libvcc side.

[dridi]

It probably belongs to cache_vcl.c, just like sending events happens from there, alongside vcl_cli_tell().

[bsdphk]

I think this is fundamentally a good idea.

I think this PR is a competent prototype that shows it is possible.

I am also pretty certain this is not how the final solution will look.

For one thing, object instances are not the only things we want to tell things:

vcl.tell blavcl.vmod_im_debugging debug_level 7

People have also been wanting to frob probes and backends without reloading VCL's from day one.

So I suggest the next step is to write the proposed user documentation for this feature, including examples, because that is a great way to find weaknesses in the design.

[nigoroll]

The existing documentation is here and in the commit message. I did not write documentation for vmod authors yet because the interface should be settled first.

My understanding from pow wow is that we should have proposed documentation for how this feature would look like when it has evolved further. My suggestion to that effect would be this (reflecting the change of the command to vcl.tell):

vcl.tell [<vcl>.]<thing> <msg> ...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Tell <msg> to <thing> from the given <vcl> or the active vcl

  <thing> can be any named object known to <vcl> like probes, backends, vmod 
  objects or vmods.

  Both <vcl> and <thing> can contain wildcards. When wildcards are used, status 
  codes >= 300 are ignored and the otherwise highest status code from any invoked 
  method is returned.

  If no match is found for <vcl> or <thing>, an error with status 300 is returned.

  Not all things are required to implement the tell interface. If they do, it is entirely up 
  to the respective implementation to interpret <msg>, implement any actions and 
  generate a response. Likewise, documentation of the supported <msg> will be 
  found with the documentation of <thing>.

  By convention, implementations of tell interfaces should:

  * support the "ping" command and respond with "pong"
  * interpret the first element of <msg> as a command
  * prefix supported command names by their vmod name
  * generate an error message for commands which are not understood

Examples:
---------

varnish> vcl.tell obj0 is there anybody out there?
200
obj0: is there anybody out there?

varnish> vcl.tell whoisit hello?
300
No object named whoisit found

varnish> vcl.tell obj0 fail
300
You asked me to fail

varnish> vcl.tell dir_fallback_* fallback.reset
200
dir_fallback_one: reset succeeded
dir_fallback_two: reset succeeded
dir_fallback_three: already on primary backend

varnish> vcl.tell *.* ping
200
vclA.probe1: pong
vclA.backend1: pong
vclA.obj1: pong

[bsdphk]

The more I think about this, the more I feel tempted to extend the vcli all the way out there.

For instance, what about help vcl.tell ?

That needs to do more than just say "Say something to something"

help vcl.tell some_vcl should at the very least explain who is listening.

help vcl.tell some_vcl.fooobj should explain what that object is willing to hear

[bsdphk]

I've looked at vcli today, with an eye to the "help" question, and I think we can swing that by allowing help functions to emit their own help (marking these with a flag letter).

[dridi]

Should we put this on hold until the security model question from #3906 is resolved? This ticket should definitely serve as input, I will mention it there as well.

Since you touch on flags and the help command, we have a downstream change that cleans certain things up and streamlines others in this area that we should be able to submit soon-ish.