VStream schema tracking does not work with online DDL

[5antelope]

Overview of the Issue

We enabled schema tracking on vttablet, the goal is we can track schema changes properly so that we can stitch row event with field events together. It works well with the raw ALTER TABLE sql statements. However, it does not work with schema migration with online ddl. We got cannot determine table columns for messages error after the online ddl finished, which indicates the schema tracker still holds the old schema and it does not match the one we get from TABLE_MAP_EVENT

Skim through the code, I think this is because in onlind ddl schema migration, we mark the "end" of it with a RENAME TABLE statement - it should be the signal for Vitess to reload schema for the table. However, Vitess parse RENAME TABLE to a RenameTable which does not have a table name, therefore we skip the reload call.

Reproduction Steps

If my theory is correct, it can be reproduced by a unit test:

diff --git a/go/vt/vttablet/tabletserver/schema/tracker_test.go b/go/vt/vttablet/tabletserver/schema/tracker_test.go
index 39d97dc46a..2b30ee47e5 100644
--- a/go/vt/vttablet/tabletserver/schema/tracker_test.go
+++ b/go/vt/vttablet/tabletserver/schema/tracker_test.go
@@ -160,6 +160,7 @@ func TestMustReloadSchemaOnDDL(t *testing.T) {
                {"create table x(i int);", db1, true},
                {"bad", db2, false},
                {"create table db2.x(i int);", db2, true},
+               {"rename table db2.x to db2.y;", db2, true},
                {"create table db1.x(i int);", db2, false},
                {"create table _vt.x(i int);", db1, false},
                {"DROP VIEW IF EXISTS `pseudo_gtid`.`_pseudo_gtid_hint__asc:55B364E3:0000000000056EE2:6DD57B85`", db2, false},

Or to reproduce it end to end:

• start a vstream
• apply an schema migration online ddl

Operating system and Environment details

Running on top of 4358c6a

Log Fragments

VStream error from shard {"error":"vttablet: rpc error: code = Unknown desc = stream (at source tablet) error @ 50504740-0aa6-11ec-ad11-0a0a6ed2d22b:1-53236:1000011-1000302:2000029-2000031: cannot determine table columns for messages: event has [8 17 15 8], schema as [name:\"page\" type:INT64 table:\"messages\" org_table:\"messages\" database:\"vt_shichaoks\" org_name:\"page\" column_length:20 charset:63 flags:53251 name:\"time_created_ns\" type:TIMESTAMP table:\"messages\" org_table:\"messages\" database:\"vt_shichaoks\" org_name:\"time_created_ns\" column_length:19 charset:63 flags:9344 name:\"message\" type:VARCHAR table:\"messages\" org_table:\"messages\" database:\"vt_shichaoks\" org_name:\"message\" column_length:40000 charset:45]","errorVerbose":"Code: UNKNOWN\nvttablet: rpc error: code = Unknown desc = stream (at source tablet) error @ 50504740-0aa6-11ec-ad11-0a0a6ed2d22b:1-53236:1000011-1000302:2000029-2000031: cannot determine table columns for messages: event has [8 17 15 8], schema as [name:\"page\" type:INT64 table:\"messages\" org_table:\"messages\" database:\"vt_shichaoks\" org_name:\"page\" column_length:20 charset:63 flags:53251 name:\"time_created_ns\" type:TIMESTAMP table:\"messages\" org_table:\"messages\" database:\"vt_shichaoks\" org_name:\"time_created_ns\" column_length:19 charset:63 flags:9344 name:\"message\" type:VARCHAR table:\"messages\" org_table:\"messages\" database:\"vt_shichaoks\" org_name:\"message\" column_length:40000 charset:45]\n","shard":"80-"}

[deepthi]

Fixed in #8963 and #9043