error with "show full columns" after keyspace split

[aberfeldy]

Overview of the Issue

Using the operator I created a cluster on my kubernetes with 3 replicas based on vitess/lite:v8.0.0 to host a wordpress multisite installation. Following the instructions for "MoveTables" I was able to move all the wp_*_posts table in a different unsharded keyspace (posts). Wordpress still functions and can show all the data, which was previous in the main keyspace (blog). But once I hit update on an edited post, I get following error:

WordPress database error vtgate: http://wordpress-zone1-vtgate-13455580-56cff4957d-m5278:15000/: target: blog.-.master, used tablet: zone1-1610175284 (10.42.9.250): vttablet: rpc error: code = NotFound desc = Table 'vt_blog.wp_4_posts' doesn't exist (errno 1146) (sqlstate 42S02) (CallerID: user): Sql: "show full columns from wp_4_posts", BindVars: {} for query SHOW FULL COLUMNS FROM `wp_4_posts` 

It seems, the show full columns doesn't care about the routing created during the tables movement.
I found a similar issue #6803 which should be fixed in #6825, thus I checked twice if I'm using the correct image. But it should include the fix for it.

Reproduction Steps

Steps to reproduce this issue, example:

1. Deploy the following cluster for the operator:

apiVersion: planetscale.com/v2
kind: VitessCluster
metadata:
  name: wordpress
spec:
  images:
    vtctld: vitess/lite:v8.0.0
    vtgate: vitess/lite:v8.0.0
    vttablet: vitess/lite:v8.0.0
    vtbackup: vitess/lite:v8.0.0
    mysqld:
      mysql56Compatible: vitess/lite:v8.0.0
    mysqldExporter: prom/mysqld-exporter:v0.11.0
  cells:
    - name: zone1
      gateway:
        authentication:
          static:
            secret:
              name: wordpress-cluster-config
              key: users.json
        replicas: 1
        resources:
          requests:
            cpu: 400m
            memory: 4Gi
          limits:
            memory: 6Gi
  vitessDashboard:
    cells:
      - zone1
    extraFlags:
      security_policy: read-only
    replicas: 1
    resources:
      limits:
        memory: 521Mi
      requests:
        cpu: 100m
        memory: 128Mi

  keyspaces:
    - name: blog
      turndownPolicy: Immediate
      partitionings:
        - equal:
            parts: 1
            shardTemplate:
              databaseInitScriptSecret:
                name: wordpress-cluster-config
                key: init_db.sql
              replication:
                enforceSemiSync: false
              tabletPools:
                - cell: zone1
                  type: replica
                  replicas: 3
                  vttablet:
                    extraFlags:
                      db_charset: utf8mb4
                    resources:
                      requests:
                        cpu: 100m
                        memory: 2Gi
                  mysqld:
                    resources:
                      requests:
                        cpu: 100m
                        memory: 4Gi
                  dataVolumeClaimTemplate:
                    accessModes: ["ReadWriteOnce"]
                    resources:
                      requests:
                        storage: 100Gi
    - name: posts
      turndownPolicy: Immediate
      partitionings:
        - equal:
            parts: 1
            shardTemplate:
              databaseInitScriptSecret:
                name: wordpress-cluster-config
                key: init_db.sql
              replication:
                enforceSemiSync: false
              tabletPools:
                - cell: zone1
                  type: replica
                  replicas: 3
                  vttablet:
                    extraFlags:
                      db_charset: utf8mb4
                    resources:
                      requests:
                        cpu: 100m
                        memory: 2Gi
                  mysqld:
                    resources:
                      requests:
                        cpu: 100m
                        memory: 4Gi
                  dataVolumeClaimTemplate:
                    accessModes: ["ReadWriteOnce"]
                    resources:
                      requests:
                        storage: 100Gi
  updateStrategy:
    type: Immediate

2. using this json for vtctlclient MoveTables -workflow=blog2posts blog posts

{
  "wp_posts": {},
  "wp_2_posts": {},
  "wp_3_posts": {},
  "wp_4_posts": {},
  "wp_5_posts": {},
  "wp_6_posts": {},
  "wp_7_posts": {},
  "wp_8_posts": {},
  "wp_9_posts": {},
  "wp_10_posts": {},
  "wp_11_posts": {},
  "wp_12_posts": {},
  "wp_13_posts": {},
  "wp_14_posts": {},
  "wp_15_posts": {},
  "wp_16_posts": {},
  "wp_17_posts": {},
  "wp_18_posts": {},
  "wp_19_posts": {},
  "wp_20_posts": {},
  "wp_21_posts": {}
}

3. Running all of this according to the docs of "MoveTables"

vtctlclient MoveTables -workflow=blog2posts blog posts "$(cat split.json)"

vtctlclient VDiff posts.blog2posts

vtctlclient SwitchReads -tablet_type=rdonly posts.blog2posts
vtctlclient SwitchReads -tablet_type=replica posts.blog2posts
vtctlclient SwitchWrites posts.blog2posts

vtctlclient DropSources posts.blog2posts

Binary version

Using the vitess/lite:v8.0.0 docker image:

/vt/bin/vtgate --version
Version: 7e09d0c20 (Git branch 'heads/v8.0.0') built on Tue Oct 27 14:35:28 UTC 2020 by vitess@00939aa4be4b using go1.13.15 linux/amd64

Operating system and Environment details

Kuberentes v1.19.4
Vitess Operator v2.1.0

Log Fragments

From the master vttablet:
E1215 13:57:29.449644 1 tabletserver.go:1295] Table 'vt_blog.wp_4_posts' doesn't exist (errno 1146) (sqlstate 42S02) (CallerID: user): Sql: "show full columns from wp_4_posts", BindVars: {}

[harshit-gangal]

This looks to be issue with plan cache. Now we only cache plan for dml and select queries after this change. Can you try now?

[aberfeldy]

Tried with latest, works like a charm. Thanks a lot