Trying to use multiple keyspaces and routingrules

[CyrilDevOps]

When filing a bug, please include the following headings if
possible. Any example text in this template can be deleted.

Overview of the Issue

I move a table from my main keyspace to a secondary keyspace, and added some routingrules to route the query to the secondary keyspace for that table.
I am able to do basic select count(*) from mytable on other keyspace, but a more complex query with innerjoin ... return an error.

Reproduction Steps

Steps to reproduce this issue, example:

1. Deploy the following vschema:

I have to basic keyspace without sharding : legacy2 and shards.
I create my application table in keyspace legacy2 with liquibase script.
Application JDBC URL is jdbc:mysql://vitess:3306/legacy2 (keyspace legacy2)
Application test pass.
I use the the MoveTables/SwitchRead/Write/DropSource to move table 'asset' from keyspace legacy2 to keyspace shards
I then apply this basic routing rules :

  "rules": [
    {
      "fromTable": "legacy2.asset@rdonly",
      "toTables": [
        "shards.asset"
      ]
    },
    {
      "fromTable": "asset",
      "toTables": [
        "shards.asset"
      ]
    },
    {
      "fromTable": "asset@replica",
      "toTables": [
        "shards.asset"
      ]
    },
    {
      "fromTable": "asset@rdonly",
      "toTables": [
        "shards.asset"
      ]
    },
    {
      "fromTable": "legacy2.asset",
      "toTables": [
        "shards.asset"
      ]
    },
    {
      "fromTable": "shards.asset@replica",
      "toTables": [
        "shards.asset"
      ]
    },
    {
      "fromTable": "shards.asset@rdonly",
      "toTables": [
        "shards.asset"
      ]
    },
    {
      "fromTable": "legacy2.asset@replica",
      "toTables": [
        "shards.asset"
      ]
    }
  ]
}```

I am able to do basic select count(*) from asset; on legacy2 keyspace and on shards keyspace.
But If I rerun the application test, it fail with sql error.
The query ```SELECT    A.asset_id, A.space_id, A.asset_type_id, A.asset_eid, A.asset_eid_hash,     A.created_by_service, A.created_by_user, A.created_date,     A.last_modified_by_service, A.last_modified_by_user, A.last_modified_date,     A.deleted, A.last_change_eid, A.last_change_eid_hash, A.tip_view, A.is_processed,     S.space_eid, CONCAT(T.type_name, '-', T.type_semver) AS type_name,     S.collection_eid, S.collection_eid_hash   FROM asset A     INNER JOIN space S ON A.space_id = S.space_id     INNER JOIN type T ON A.asset_type_id = T.type_id   WHERE     asset_eid_hash = ? AND asset_eid = ?          AND S.deleted IS NULL```
in legacy2 return the error  : symbol asset_eid_hash not found

There is no sharding anywhere, just trying to use multiple keyspace transparantly for the application.

vitess@a534ffc0ae2c:/vt/vtdataroot/tmp$ vtctlclient GetVSchema legacy2
{
}
vitess@a534ffc0ae2c:/vt/vtdataroot/tmp$ vtctlclient GetVSchema shards
{
  "tables": {
    "asset": {
    }
  }
}
 

#### Binary version
Example:
Version: 11.0.0-SNAPSHOT (Git revision 68aa306d8 branch 'master') built on Thu May  6 02:24:13 UTC 2021 by vitess@buildkitsandbox using go1.15.6 linux/amd64

I use the last git clone and the docker example. (make docker ...)

Only change I made is on the vttablet-up.sh script, I added : 
 -init_db_name_override $keyspace \
 -db_charset utf8mb4 \

and on scripts/mysqlctl-up.sh : 
  -mysqlctl_mycnf_template /vt/local/my.cf \

My my.cf is the same one from the config except I change those to line to utf8mb4 : 
character_set_server = utf8mb4
collation_server = utf8mb4_unicode_ci

Our application is using utf8mb4 and default vitess config create collate erro

#### Operating system and Environment details

OS, Architecture, and any other information you can provide
about the environment.

vitess@a534ffc0ae2c:/vt/local$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
vitess@a534ffc0ae2c:/vt/local$ uname -sr
Linux 5.10.25-linuxkit
vitess@a534ffc0ae2c:/vt/local$ uname -m
x86_64

#### Log Fragments

Include appropriate log fragments. If the log is longer than a few dozen lines, please
include the URL to the [gist](https://gist.github.com/) of the log instead of posting it in the issue.

[systay]

👋

Hi there! We have finally gotten around to fixing this. It's solved in the gen4 planner, which you can use by using the planner=gen4fallback flag on vtgate.

Do let us know if this is not an acceptable solution for you

Thank you for reporting the issue!