Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Online DDL: support 'SHOW VITESS_MIGRATIONS' queries #7638

Merged
merged 26 commits into from
Mar 10, 2021

Conversation

shlomi-noach
Copy link
Contributor

@shlomi-noach shlomi-noach commented Mar 8, 2021

Description

Supporting these queries in vtgate:

  • SHOW VITESS_MIGRATIONS - shows all migrations for keyspace
  • SHOW VITESS_MIGRATIONS LIKE '...' - shows vitess migrations filtered for UUID, or context, or status. For example:
    • SHOW VITESS_MIGRATIONS LIKE '6df3201f_7fda_11eb_aa27_f875a4d24e90' - shows a specific migration
    • SHOW VITESS_MIGRATIONS LIKE 'failed' - show migrations with failed status
  • SHOW VITESS_MIGRATIONS WHERE ... - add any custom filter, e.g.:
    • SHOW VITESS_MIGRATIONS WHERE started_timestamp > NOW() - INTERVAL 7 DAY
  • SHOW VITESS_MIGRATIONS FROM keyspace [LIKE ...|WHERE ...] - explicitly indicate keyspace

Related Issue(s)

Checklist

  • Should this PR be backported?
  • Tests were added or are not required
  • Documentation was added or is not required

Deployment Notes

Impacted Areas in Vitess

Components that this PR will affect:

  • Query Serving
  • VReplication
  • Cluster Management
  • Build/CI
  • VTAdmin

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Copy link
Contributor Author

Seems like even though we specify key.DestinationAllShards, query only runs on a single shard.

systay and others added 12 commits March 8, 2021 14:35
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
…ess into online-ddl-sql-interface

Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Copy link
Contributor Author

Thank you very much to @harshit-gangal and @systay who guided and led me in the right path, and for noticing my Online DDL endtoend tests had wrong/broken shard configuration!

@shlomi-noach shlomi-noach marked this pull request as ready for review March 9, 2021 08:03
@shlomi-noach
Copy link
Contributor Author

This is now ready to review. This PR only introduced SHOW VITESS_MIGRATIONS command(s), which is a more urgent requirement. ALTER VITESS_MIGRATIONS will be handled in a followup PR.

Documentation to follow.

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Copy link
Contributor Author

Added one last commit, refactored a couple functions in endtoend tests, where I originally did a lot of copy+paste. There's more cleanup work to be done, but not in this PR.

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Copy link
Member

@harshit-gangal harshit-gangal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest to refactor the file directory as go/test/endtoend/onlineddl/<ghost/vrepl/...> and place query_util and vtgate_util under go/test/endtoend/onlineddl/ those are the only tests that are using it.

@@ -2480,7 +2480,7 @@ func (node *Show) Format(buf *TrackedBuffer) {
func (node *ShowLegacy) Format(buf *TrackedBuffer) {
nodeType := strings.ToLower(node.Type)
if (nodeType == "tables" || nodeType == "columns" || nodeType == "fields" || nodeType == "index" || nodeType == "keys" || nodeType == "indexes" ||
nodeType == "databases" || nodeType == "schemas" || nodeType == "keyspaces" || nodeType == "vitess_keyspaces" || nodeType == "vitess_shards" || nodeType == "vitess_tablets") && node.ShowTablesOpt != nil {
nodeType == "databases" || nodeType == "schemas" || nodeType == "keyspaces" || nodeType == "vitess_keyspaces" || nodeType == "vitess_shards" || nodeType == "vitess_tablets" || nodeType == "vitess_migrations") && node.ShowTablesOpt != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not needed as show vitess migrations is not legacy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Copy link
Contributor Author

@harshit-gangal I have restructured the tests, all nested under go/test/endtoend/onlineddl

go/test/endtoend/onlineddl # base package with shared functionality
go/test/endtoend/onlineddl/vrepl
go/test/endtoend/onlineddl/vrepl_stress
go/test/endtoend/onlineddl/ghost
go/test/endtoend/onlineddl/revert

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants