RFC: vtctld as a mysql server

[sougou]

Vitess has always had an implicit differentiation of a control plane from the data plane. VTTablet has the tabletserver component (data), and a tabletmanager (control). VTGate serves queries (data), and vtctld manages the cluster (control). VTGate predominantly talks to the tabletserver, while vtctld talks to the tabletmanager.

This differentiation is at odds with mysql's model where SQL can be used to perform control and data commands. An increasing number of features are pushing this architecture coercing vtgate to take on more control plane functionality.

On the other hand, vtctld has also started accepting SQL commands for cluster management activities. We've always had the raw commands like ExecuteFetchAsDba. But more recently, we've introduced commands like VReplicationExec and VExec that accept SQL statements.

What if we changed vtctld to become a mysql server like VTGate, except that it will process only control plane commands? This will allow the protocol to take over the above commands, and then accommodate future commands. For example, we can accept DDLs that can launch gh-ost workflows (idea from @shlomi-noach). We can also look at moving all the vschema ddls to vtctld thereby lightening the burden that vtgate has been accumulating.

This won't be straightforward in certain cases. For example, the VExec's workflow model has to be translated into a logical "schema". However, this can all be worked out if the idea is appealing.

[derekperkins]

I think this is a great idea. Beyond just standard vtctld meta, we could also store basic statistics there for easy export into Grafana or otherwise pushed into the new UI.

[shlomi-noach]

I support the idea. Vtctld will be able to serve meta queries, information-schema -like, about status of the system. Current use case I have is status of schema migrations.