Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vtctl ApplySchema -uuid_list #9325

Merged
merged 9 commits into from
Dec 9, 2021
Merged

Conversation

shlomi-noach
Copy link
Contributor

@shlomi-noach shlomi-noach commented Dec 5, 2021

Description

vtctlient ApplySchema now support a new optional -uuid_list flag. It is possible for the user to explicitly specify the UUID(s) for given migration(s). UUIDs must be in a specific format. If given, number of UUIDs must match the number of DDL statements. Example:

vtctlclient OnlineDDL ApplySchema -sql "drop table t1, drop table t2" -uuid_list "d08f0000_51c9_11ec_9cf2_0a43f95f28a3,d08f0001_51c9_11ec_9cf2_0a43f95f28a3" commerce

Format is comma delimited, zero or more UUIDs.

Vitess will assign each migration with given UUID in order of appearance.
It is the user's responsibility to ensure given UUIDs are globally unique. If the user submits a migration with an already existing UUID, that migration never gets scheduled nor executed.

The purpose of this change is to give users better control over submitted migrations. Moreover, it is a means for users to achiever idempotency: the user can choose to resubmit a migration with same UUID, intentionally ensuring it does not execute again.

-uuid_list is optional. If omitted or if empty then Vitess auto-generates UUIDs for the migrations.

Related Issue(s)

Checklist

  • Should this PR be backported?
  • Tests were added or are not required
  • Documentation was added or is not required

… their own migration UUID and at their own risk if the UUID they provide is not unique

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@mavenraven
Copy link
Contributor

@shlomi-noach looks great to me from a consumer perspective!

Copy link
Member

@deepthi deepthi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good. A few nits/suggestions on flag and variables names.

@@ -60,6 +60,17 @@ For example:
vtctlclient OnlineDDL commerce complete d08ffe6b_51c9_11ec_9cf2_0a43f95f28a3
```

### vtctl/vtctlclient ApplySchema -uuid
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intuitively uuid seems to indicate that it is a single uuid versus a list or a set. uuids feels awkward, but maybe uuid_list??

@@ -245,15 +245,16 @@ func TestSchemaChange(t *testing.T) {
assert.Equal(t, 2, len(shard.Vttablets))
}

explicitUUID := ""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nit, but maybe call this providedUUID? Or providedUUIDList if you accept the first suggestion :)

@@ -181,15 +181,18 @@ func onlineDDLStatementSanity(sql string, ddlStmt sqlparser.DDLStatement) error
}

// NewOnlineDDLs takes a single DDL statement, normalizes it (potentially break down into multiple statements), and generates one or more OnlineDDL instances, one for each normalized statement
func NewOnlineDDLs(keyspace string, sql string, ddlStmt sqlparser.DDLStatement, ddlStrategySetting *DDLStrategySetting, requestContext string) (onlineDDLs [](*OnlineDDL), err error) {
func NewOnlineDDLs(keyspace string, sql string, ddlStmt sqlparser.DDLStatement, ddlStrategySetting *DDLStrategySetting, requestContext string, explicitUUID string) (onlineDDLs [](*OnlineDDL), err error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here as above.

@@ -3232,6 +3232,7 @@ func commandApplySchema(ctx context.Context, wr *wrangler.Wrangler, subFlags *fl
sql := subFlags.String("sql", "", "A list of semicolon-delimited SQL commands")
sqlFile := subFlags.String("sql-file", "", "Identifies the file that contains the SQL commands")
ddlStrategy := subFlags.String("ddl_strategy", string(schema.DDLStrategyDirect), "Online DDL strategy, compatible with @@ddl_strategy session variable (examples: 'gh-ost', 'pt-osc', 'gh-ost --max-load=Threads_running=100'")
uuids := subFlags.String("uuid", "", "Optional: comma delimited explicit UUIDs for migration. If given, must match number of DDL changes")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intuitively uuid seems to indicate that it is a single uuid versus a list or a set. uuids feels awkward for a flag name, but maybe uuid_list??

@@ -251,21 +251,21 @@ func (t TupleExpr) Evaluate(env *ExpressionEnv) (EvalResult, error) {
}

// Evaluate implements the Expr interface
func (b *BindVariable) Evaluate(env *ExpressionEnv) (EvalResult, error) {
val, ok := env.BindVars[b.Key]
func (bv *BindVariable) Evaluate(env *ExpressionEnv) (EvalResult, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes to this file seem unrelated. Are they really required?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh this was due to the new linter throwing errors in that file. I'll see what I can do here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok this at least was already merged by another PR

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach shlomi-noach changed the title vtctl ApplySchema -uuid vtctl ApplySchema -uuid_list Dec 9, 2021
@shlomi-noach
Copy link
Contributor Author

flag -uuid renamed to -uuid_list

explicitUUID renamed to proidedUUID throughout the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Cluster management Type: Enhancement Logical improvement (somewhere between a bug and feature)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants