Vitess 10.0.0

This release complies with VEP-3 which removes the upgrade order requirement. Components can be upgraded in any order. It is recommended that the upgrade order should still be followed if possible, except to canary test the new version of VTGate before upgrading the rest of the components.

Known Issues

• Running binaries with --version or running select @@version from a MySQL client still shows 10.0.0-RC1

• Online DDL cannot be used if you are using the keyspace filtering feature of VTGate

• VReplication errors when a fixed-length binary column is used as the sharding key #8080

• A critical vulnerability CVE-2021-44228 in the Apache Log4j logging library was disclosed on Dec 9 2021.
  The project provided release 2.15.0 with a patch that mitigates the impact of this CVE. It was quickly found that the initial patch was insufficient, and additional CVEs
  CVE-2021-45046 and CVE-2021-44832 followed.
  These have been fixed in release 2.17.1. This release of Vitess, v10.0.0, uses a version of Log4j below 2.17.1, for this reason, we encourage you to use version v10.0.5 instead, to benefit from the vulnerability patches.

• An issue where the value of the -force flag is used instead of -keep_data flag's value in v2 vreplication workflows (#9174) is known to be present in this release. A workaround is available in the description of issue #9174.

Bugs Fixed

VTGate / MySQL compatibility

• Remove printing of ENFORCED word so that statements remain compatible with mysql 5.7 #7458
• Allow any ordering of generic options in column definitions #7459
• Corrects the comment handling in vitess #7581
• Fix regression - should be able to plan subquery on top of subquery #7682
• Nullable Timestamp Column Fix #7740
• VTGate: Fix the error messages in drop, create and alter database commands #7397
• VTGate: Fix information_schema query with system schema in table_schema filter #7430
• VTGate: Fix Set Statement in Tablet when executed with bindvars #7431
• VTGate: Fix for Query Serving when Toposerver is Down #7484
• VTGate: Add necessary bindvars when preparing queries #7493
• VTGate: Show anywhere plan fix to consider default keyspace #7531
• VTGate: Fix table parsing on VSchema generation #7511
• VTGate: Use the emulated MySQL version for special comments #7510
• VTGate: Reset Session for Reserved Connection when the connection id is not found #7539
• VTGate: Healthcheck: update healthy tablets correctly when a stream returns an error or timeout #7732
• VTGate: Fix for reserved connection usage with transaction #7646
• VTGate: Fix MySQL Workbench failure on login with select current_user() #7705
• VTGate: Constraint names and database names with spaces. #7745
• VTGate: Fix dual table query when system schema is selected database #7734

Other

• VTTablet: Correctly initialize statsTabletTypeCounts during startup #7390
• Backup/Restore: Respect -disable_active_reparents in backup/restore #7576
• Backup/Restore: check disable_active_reparents properly before waiting for position update #7703

Functionality Added or Changed

VTGate / MySQL compatibility / Query Serving

• VTGate: Gen4 Planner: AxB vs BxA #7274
• VTGAte: Gen4 fallback planning #7370
• VTGate: Support for CALL procedures #7287
• VTGate: Set default for @@workload to OLTP #7288
• VTGate: Change @@Version and @@version_comment #7337
• VTGate: Fix VitessAware system variables of type boolean return NULL when MySQL is not involved #7353
• VTGate: Add stats for RowsAffected similar to RowsReturned #7380
• VTGate: Added information_schema_stats_expiry to allowed list of set vars #7435
• VTGate: LFU Cache Implementation #7439
• VTGate: Describe table to route based on table name and qualifier #7445
• VTGate: Olap error message fix #7448
• VTGate: Temporary Table support in unsharded keyspace #7411
• VTGate: Publish table size on schema #7444
• VTGate: Support for caching_sha2_password plugin in mysql/client #6716
• VTGate: Moving Show plan from executor to planbuilder #7475
• VTGate: Adds another case to merge routes for information_schema queries #7504
• VTGate: Add innodb_read_rows as vttablet metric #7520
• VTGate: Adds support for Show variables #7547
• VTGate: gen4: fail unsupported queries #7409
• VTGate: Fix Metadata in SHOW queries #7540
• VTGate: Update AST helper generation #7558
• VTGate: Avoiding addition of redundant unary operators #7579
• VTGate: Optimise AST rewriting #7583
• VTGate: Add Show Status query to vtexplain and make asthelpergen/sizegen quiet #7590
• VTGate: Add support for SELECT ALL #7593
• VTGate: Empty statement error code change in sql parsing #7618
• VTGate: Socket system variable to return vitess mysql socket #7637
• VTGate: Make DROP/CREATE DATABASE pluggable #7381
• VTGate: Allow Select with lock to pass through in vttablet #7584
• VTGate: Fix ordering in SELECT INTO and printing of strings #7655
• VTGate: AST Equals code generator #7672
• VTGate: [tabletserver] More resilient wait for schema changes #7684
• VTGate: Fix flush statement planner #7695
• VTGate: Produce query warnings for using features not supported when sharded #7538
• VTGate: Support for ALTER VITESS_MIGRATION statements #7663
• VTGate: Solve I_S queries using CNF rewriting #7677
• VTGate: System schema queries #7685
• VTGate: Make the AST visitor faster #7701
• VTGate: COM_PREPARE - Single TCP response packet with all MySQL Packets #7713
• VTGate: Replace the database name in result fields only if needed #7714
• VTGate: Split ast_helper into individual gen files #7727
• VTGate: Adds support for ordering on character fields for sharded keyspace queries #7678
• VTGate: Show columns query on system schema #7729
• VTGate: Disallow foreign key constraint on ddl #7780
• VTGate: VTGate: support -enable_online_ddl flag #7694
• VTGate: Default to false for system settings to be changed per session at the database connection level #7921
• VTGate: vtctl: return error on invalid ddl_strategy #7924
• VTGate: [10.0] Squashed backport of #7903 #7927
• VTGate: [10.0] Fix bug with reserved connections to stale tablets #7935
• VTGate: [10.0] Fix for keyspaces_to_watch regression #7936
• VTGate: [10.0] Update healthy tablets correctly for primary down #7937
• VTGate: [10.0] Allow modification of tablet unhealthy_threshold via debugEnv #7938

Testing

• Fuzzing: Add vtctl fuzzer #7605
• Fuzzing: Add more fuzzers #7622
• Fuzzing: Add 3 fuzzers for mysql endpoints #7639
• Fuzzing: Add oss-fuzz build script #7591
• Fuzzing: Add requirement for minimum length of input data #7722
• Fuzzing: Add new mysql fuzzer #7660
• Fuzzing: Add [grpcvtgateconn] fuzzer #7689
• Fuzzing: Make mysql fuzzer more calls during each iteration #7766

Performance

• VTGate: [perf] zero-copy tokenizer #7619
• VTGate: [perf: sqlparser faster formatting #7710
• VTGate :[perf] Cache reserved bind variables in queries #7698
• VTGate: [perf] sqlparser yacc codegen #7669
• VTGate: Making fast AST rewriter faster #7726
• VTGate: Cached Size Implementation #7387
• VTGate: Plan remove mutexes #7468
• LFU Cache Bug Fixes #7479
• [cache] Handle all possible initialization cases #7556
• VTGate: [servenv] provide a global flag for profiling #7496
• VTGate: [vttablet] Benchmarks and two small optimizations #7560
• [pprof]: allow stopping profiling early with a signal #7594
• perf: RPC Serialization #7519
• perf: keyword lookups in the tokenizer #7606

Cluster Management

• [vtctld] Migrate topo management RPCs #7395
• [vtctldclient] Set SilenceErrors on the root command, so we don't double-log #7404
• [vtctldclient] Command line flags: dashes and underscores synonyms #7406
• Extract the maxReplPosSearch struct out to topotools #7420
• Add protoutil package, refactor ISP to use it #7421
• Add ErrorGroup to package concurrency, use in waitOnNMinusOneTablets #7429
• [vtctld / wrangler] Extract some reparent methods out to functions for shared use between wrangler and VtctldServer #7434
• [vtctld/wrangler] Extract EmergencyReparentShard logic to dedicated struct and add unit tests #7464
• Provide named function for squashing usage errors; start using it #7451
• [concurrency] Add guard against potentially blocking forever in ErrorGroup.Wait() when NumGoroutines is 0 #7463
• Add hook for statsd integration #7417
• [concurrency] Add guard against potentially blocking forever in ErrorGroup.Wait() when NumGoroutines is 0 #7463
• Resilient rebuild keyspace graph check, tablet controls not in RebuildKeyspaceGraph command #7442
• [reparentutil / ERS] confirm at least one replica succeeded to SetMaster, or fail #7486
• [reparentutil / wrangler] Extract PlannedReparentShard logic from wrangler to PlannedReparenter struct #7501
• Add backup/restore duration stats #7512
• Refresh replicas and rdonly after MigrateServedTypes except on skipRefreshState. #7327
• [eparentutil] ERS should not attempt to WaitForRelayLogsToApply on primary tablets that were not running replication #7523
• [orchestrator] prevent XSS attack via 'orchestrator-msg' params #7526
• [vtctld] Add remaining reparent commands to VtctldServer #7536
• [reparentutil] ERS should not attempt to WaitForRelayLogsToApply on primary tablets that were not running replication #7523
• Shutdown vttablet gracefully if tablet record disappears #7563
• ApplySchema: -skip_preflight #7587
• Table GC: disable binary logging on best effort basis #7588
• Addition of waitSig pprof argument to start recording on USR1 #7616
• Add combine TLS certs feature #7609
• Check error response before attempting to access InitSh...

Vitess 10.0.0-rc1

This release complies with VEP-3 which removes the upgrade order requirement. Components can be upgraded in any order. It is recommended that the upgrade order should still be followed if possible, except to canary test the new version of VTGate before upgrading the rest of the components.

The following PRs made changes to behaviors that clients might rely on. They should be reviewed carefully so that client code can be changed in concert with a Vitess release deployment.

Bugs Fixed

VTGate / MySQL compatibility

• Remove printing of ENFORCED word so that statements remain compatible with mysql 5.7 #7458
• Allow any ordering of generic options in column definitions #7459
• Corrects the comment handling in vitess #7581
• Fix regression - should be able to plan subquery on top of subquery #7682
• Nullable Timestamp Column Fix #7740
• VTGate: Fix the error messages in drop, create and alter database commands #7397
• VTGate: Fix information_schema query with system schema in table_schema filter #7430
• VTGate: Fix Set Statement in Tablet when executed with bindvars #7431
• VTGate: Fix for Query Serving when Toposerver is Down #7484
• VTGate: Add necessary bindvars when preparing queries #7493
• VTGate: Show anywhere plan fix to consider default keyspace #7531
• VTGate: Fix table parsing on VSchema generation #7511
• VTGate: Use the emulated MySQL version for special comments #7510
• VTGate: Reset Session for Reserved Connection when the connection id is not found #7539
• VTGate: Healthcheck: update healthy tablets correctly when a stream returns an error or timeout #7732
• VTGate: Fix for reserved connection usage with transaction #7646
• VTGate: Fix MySQL Workbench failure on login with select current_user() #7705
• VTGate: Constraint names and database names with spaces. #7745
• VTGate: Fix dual table query when system schema is selected database #7734

Other

• VTTablet: Correctly initialize statsTabletTypeCounts during startup #7390
• Backup/Restore: Respect -disable_active_reparents in backup/restore #7576
• Backup/Restore: check disable_active_reparents properly before waiting for position update #7703

Functionality Added or Changed

VTGate / MySQL compatibility / Query Serving

• VTGate: Gen4 Planner: AxB vs BxA #7274
• VTGAte: Gen4 fallback planning #7370
• VTGate: Support for CALL procedures #7287
• VTGate: Set default for @@workload to OLTP #7288
• VTGate: Change @@Version and @@version_comment #7337
• VTGate: Fix VitessAware system variables of type boolean return NULL when MySQL is not involved #7353
• VTGate: Add stats for RowsAffected similar to RowsReturned #7380
• VTGate: Added information_schema_stats_expiry to allowed list of set vars #7435
• VTGate: LFU Cache Implementation #7439
• VTGate: Describe table to route based on table name and qualifier #7445
• VTGate: Olap error message fix #7448
• VTGate: Temporary Table support in unsharded keyspace #7411
• VTGate: Publish table size on schema #7444
• VTGate: Support for caching_sha2_password plugin in mysql/client #6716
• VTGate: Moving Show plan from executor to planbuilder #7475
• VTGate: Adds another case to merge routes for information_schema queries #7504
• VTGate: Add innodb_read_rows as vttablet metric #7520
• VTGate: Adds support for Show variables #7547
• VTGate: gen4: fail unsupported queries #7409
• VTGate: Fix Metadata in SHOW queries #7540
• VTGate: Update AST helper generation #7558
• VTGate: Avoiding addition of redundant unary operators #7579
• VTGate: Optimise AST rewriting #7583
• VTGate: Add Show Status query to vtexplain and make asthelpergen/sizegen quiet #7590
• VTGate: Add support for SELECT ALL #7593
• VTGate: Empty statement error code change in sql parsing #7618
• VTGate: Socket system variable to return vitess mysql socket #7637
• VTGate: Make DROP/CREATE DATABASE pluggable #7381
• VTGate: Allow Select with lock to pass through in vttablet #7584
• VTGate: Fix ordering in SELECT INTO and printing of strings #7655
• VTGate: AST Equals code generator #7672
• VTGate: [tabletserver] More resilient wait for schema changes #7684
• VTGate: Fix flush statement planner #7695
• VTGate: Produce query warnings for using features not supported when sharded #7538
• VTGate: Support for ALTER VITESS_MIGRATION statements #7663
• VTGate: Solve I_S queries using CNF rewriting #7677
• VTGate: System schema queries #7685
• VTGate: Make the AST visitor faster #7701
• VTGate: COM_PREPARE - Single TCP response packet with all MySQL Packets #7713
• VTGate: Replace the database name in result fields only if needed #7714
• VTGate: Split ast_helper into individual gen files #7727
• VTGate: Adds support for ordering on character fields for sharded keyspace queries #7678
• VTGate: Show columns query on system schema #7729
• VTGate: Disallow foreign key constraint on ddl #7780
• VTGate: VTGate: support -enable_online_ddl flag #7694

Testing

• Fuzzing: Add vtctl fuzzer #7605
• Fuzzing: Add more fuzzers #7622
• Fuzzing: Add 3 fuzzers for mysql endpoints #7639
• Fuzzing: Add oss-fuzz build script #7591
• Fuzzing: Add requirement for minimum length of input data #7722
• Fuzzing: Add new mysql fuzzer #7660
• Fuzzing: Add [grpcvtgateconn] fuzzer #7689
• Fuzzing: Make mysql fuzzer more calls during each iteration #7766

Performance

• VTGate: [perf] zero-copy tokenizer #7619
• VTGate: [perf: sqlparser faster formatting #7710
• VTGate :[perf] Cache reserved bind variables in queries #7698
• VTGate: [perf] sqlparser yacc codegen #7669
• VTGate: Making fast AST rewriter faster #7726
• VTGate: Cached Size Implementation #7387
• VTGate: Plan remove mutexes #7468
• LFU Cache Bug Fixes #7479
• [cache] Handle all possible initialization cases #7556
• VTGate: [servenv] provide a global flag for profiling #7496
• VTGate: [vttablet] Benchmarks and two small optimizations #7560
• [pprof]: allow stopping profiling early with a signal #7594
• perf: RPC Serialization #7519
• perf: keyword lookups in the tokenizer #7606

Cluster Management

• [vtctld] Migrate topo management RPCs #7395
• [vtctldclient] Set SilenceErrors on the root command, so we don't double-log #7404
• [vtctldclient] Command line flags: dashes and underscores synonyms #7406
• Extract the maxReplPosSearch struct out to topotools #7420
• Add protoutil package, refactor ISP to use it #7421
• Add ErrorGroup to package concurrency, use in waitOnNMinusOneTablets #7429
• [vtctld / wrangler] Extract some reparent methods out to functions for shared use between wrangler and VtctldServer #7434
• [vtctld/wrangler] Extract EmergencyReparentShard logic to dedicated struct and add unit tests #7464
• Provide named function for squashing usage errors; start using it #7451
• [concurrency] Add guard against potentially blocking forever in ErrorGroup.Wait() when NumGoroutines is 0 #7463
• Add hook for statsd integration #7417
• [concurrency] Add guard against potentially blocking forever in ErrorGroup.Wait() when NumGoroutines is 0 #7463
• Resilient rebuild keyspace graph check, tablet controls not in RebuildKeyspaceGraph command #7442
• [reparentutil / ERS] confirm at least one replica succeeded to SetMaster, or fail #7486
• [reparentutil / wrangler] Extract PlannedReparentShard logic from wrangler to PlannedReparenter struct #7501
• Add backup/restore duration stats #7512
• Refresh replicas and rdonly after MigrateServedTypes except on skipRefreshState. #7327
• [eparentutil] ERS should not attempt to WaitForRelayLogsToApply on primary tablets that were not running replication #7523
• [orchestrator] prevent XSS attack via 'orchestrator-msg' params #7526
• [vtctld] Add remaining reparent commands to VtctldServer #7536
• [reparentutil] ERS should not attempt to WaitForRelayLogsToApply on primary tablets that were not running replication #7523
• Shutdown vttablet gracefully if tablet record disappears #7563
• ApplySchema: -skip_preflight #7587
• Table GC: disable binary logging on best effort basis #7588
• Addition of waitSig pprof argument to start recording on USR1 #7616
• Add combine TLS certs feature #7609
• Check error response before attempting to access InitShardPrimary response #7658
• [vtctld] Migrate GetSrvKeyspace as GetSrvKeyspaces in VtctldServer #7680
• [vtctld] Migrate ShardReplicationPositions #7690
• [reparentutil | ERS] Bind status variable to each goroutine in WaitForRelayLogsToApply #7692
• [servenv] Fix var shadowing caused by short variable declaration #7702
• [vtctl|vtctldserver] List/Get Tablets timeouts #7715
• vtctl ApplySchema supports '-request_context' flag #7777

VReplication

• VReplication: vstreamer to throttle on source endpoint #7324
• VReplication: Throttle on target tablet #7364
• VReplication: Throttler: fix to client usage in vreplication and table GC #7422
• VReplication: MoveTables/Reshard add flags to start workflows in a stopped state and to stop workflow once copy is completed #7449
• VReplication: Support for caching_sha2_password plugin in mysql/client #6716
• VReplication: Validate SrvKeyspace during Reshard/SwitchReads #7481
• VReplication: [MoveTables] Refresh SrvVSchema (for Routing Rules) and source tablets (for Blacklisted Tables) on completion #7505
• VReplication : Data migration from another Vitess cluster #7546
• VReplication : [materialize] Add cells and tablet_types parameters #7562
• VReplication: JSON Columns: fix bug where vreplication of update statements were failing #7640
• VReplication: Make the frequency at which heartbeats update the _vt.vreplication table configurable #7659
• VReplication: Error out if binlog compression is turned on #7670
• VReplication: Tablet throttler: support for custom query & threshold #7541
• VStream API: allow aligning streams from different shards to minimize skews across the streams #7626

OnlineDDL

• Onlin...

Vitess 9.0.0

This release complies with VEP-3 which removes the upgrade order requirement. Components can be upgraded in any order. It is recommended that the upgrade order should still be followed if possible, except to canary test the new version of VTGate before upgrading the rest of the components.

Incompatible Changes

The following PRs made changes to behaviors that clients might rely on. They should be reviewed carefully so that client code can be changed in concert with a Vitess release deployment.

• The update to golang 1.15 (#7204) might break systems that use TLS certificates with a common name. A fix is documented here (golang/go#40748 (comment))

Vitess 9.0 is not compatible with the previous release of the Vitess Kubernetes Operator (2.2.0). A new version of the Operator (2.3.0) is available that is compatible.

Known Issue(s)

• VReplication errors when a fixed-length binary column is used as the sharding key #8080

Bugs Fixed

VTGate / MySQL compatibility

• Set Global #6957
• Set udv allow more expressions #6964
• Bug which caused the connection to not close in case of error writing an error packet #6977
• Bug fix #7048 for SelectNone in StreamExecute route engine #7050

Other

• Binary PK: fix bug where padding of binary columns was being done incorrectly #6963
• Pad non-fractional part of DECIMAL type #6967
• Bug fix regression in /healthz #7090
• Fix metadata related operation hangs when zk down #7228
• Fix accidentally-broken legacy vtctl output format #7285
• Healthcheck: use isIncluded correctly to fix replica/rdonly routing bug #6904

Functionality Added or Changed

VTGate / MySQL compatibility

• VTGate: Allow INSERT with all defaults #6969
• VTGate: Allow YEAR column type with length specified #6975
• VTGate: Retry Execute when reserved connection is lost #6983
• VTGate: Use ephemeral buffer when reading rows #6990
• VTGate: Allow time_zone in reserved connection #6998
• VTGate: Improved support for UNION #7007
• VTGate: Add DDL parser support for FULLTEXT indexes. #7001
• VTGate: Extend comparability of EvalResult to support hash codes #7016
• VTGate: Use proto equal method #7017
• VTGate: Fix COM_STMT_EXECUTE packet decode #7020
• VTGate: Adds Planning and Parsing Support for Create Index of MySQL 5.7 #7024
• VTGate: UNION DISTINCT support on vtgate #7029
• VTGate: Improve database ddl plan #7034
• VTGate: Support for hex & shard in vindex query #7044
• VTGate: Use distinct primitive to solve more queries #7047
• VTGate: Route using vindex for composite IN clause #7049
• VTGate: Optimise struct field layout #7052
• VTGate: Refactoring of plan building #7054
• VTGate: Rewrite joins written with the USING construct #6660
• VTGate: Add option to GetSchema to only send the row count and data length over the wire
  #6985
• VTGate: Adds Planning and Parsing Support for Create Database of MySQL 5.7 #7068
• VTGate: Make sure to check all GROUP BY columns #7080
• VTGate: Separate sub-query and derived table into different structs #7081
• VTGate: Adds Planning and Parsing Support for Alter Database of MySQL 5.7 #7086
• VTGate: Convert usages of DDL struct to DDLStatement interface #7096
• VTGate: Adds Planning and Parsing Support for Drop Database of MySQL 5.7 #7098
• VTGate: Restore SHOW SCHEMAS support; fixes #7100 #7102
• VTGate: Refactor Code to create a separate struct for CREATE TABLE #7116
• VTGate: Allows for vttestserver and vtcombo to respond to VtGateExecute. #7121
• VTGate: Support for lock and unlock tables #7139
• VTGate: Merge SelectDBA routes when possible #7140
• VTGate: Adds support for all the rails queries using information_schema #7143
• VTGate: Add support for unary expression in order by column #7163
• VTGate: Skip query rewriting for dual table #7164
• VTGate: Refactor Code to create a separate struct for ALTER VSCHEMA #7173
• VTGate: Refactor Show plans #7185
• VTGate: Show privilege support #7194
• VTGate: Planning and Parsing Support for Drop Table, Drop View and Alter View #7178
• VTGate: Cache only dml and select plans #7196
• VTGate: Planning and Parsing Support for Alter Table #7199
• VTGate: Add FindAllShardsInKeyspace to vtctldserver #7201
• VTGate: Initial implementation of vtctld service #7128
• VTGate: improve-log: FAILED_PRECONDITION #7215
• VTGate: Default to false for system settings to be changed per session at the database connection level #7299
• VTGate: Route INFORMATION_SCHEMA queries #6932
• VTGate: Adds Planning and Parsing Support for Create Index of MySQL 5.7 #7024
• VTGate: Log sql which exceeds max memory rows #7055
• VTGate: Enable Client Session Tracking feature in mysql protocol #6783
• VTGate: Show columns from table_name targeted like select queries #6825
• VTGate: This PR adds logic to simplify subquery expressions that are simple to
• VTGate: Adding MySQL Check Constraints #6865
• VTGate: Manage read your own writes system settings #6871
• VTGate: Allow table_schema comparisons #6887
• VTGate: Additional options support for SELECT INTO and LOAD DATA #6872
• VTGate: Fixes vtgate which throws an error in case of empty statements #6947
• VTGate: [Forward Port] #6940 - Fix error handling in olap mode #6949
• VTGate: Adds Planning and Parsing Support for Create View of MySQL 5.7 #7060
• VTGate: fix error: cannot run Select on table "dual" #7118
• VTGate: Allow system table to be set as default database #7150
• VTGate: Move auto_increment from reserved to non reserved keyword #7162
• VTGate: Add only expr of aliasedExpr to weightstring function #7165
• VTGate: [9.0] don't try to compare varchars in vtgate #7271
• VTGate: Load Data From S3 #6823
• VTGate: Unnest simple subqueries #6831
• VTGate: Adding MySQL Check Constraints #6869
• VTExplain: Add sequence table support for vtexplain #7186
• VSchema: Support back-quoted names #7073
• Healthcheck: healthy list should be recomputed when a tablet is removed #7176
• Healthcheck: Hellcatlk wants to merge 1 commit into master from master #6953

Set Statement Support

Set statement support has been added in Vitess. There are some system variables which are disabled by default and can be enabled using flag -enable_system_settings on VTGate. These system variables are set on the mysql server. Because they change the mysql session, using them leads to the Vitess connection no longer using the connection pool and forcing dedicated connections.

VReplication

• VReplication: refactored and enhanced support for JSON columns #6829
• VReplication: Don't update tx timestamp on heartbeat #6930
• VReplication E2E Tests: Refactored tests for readability and attempting to fix flakiness #6991
• VRepl/Tablet Picker: improve observability of selected tablet #6999
• VReplication: Handle comment statement type in vstreamer #7092
• VReplication e2e: Fine tuned test to reduce flakiness and added more logging to debug future flakiness #7138
• VReplication: Make relay log size & rows configurable. #6992
• VReplication: New workflows cli UX. Allow reads/writes to be switched independently #7071
• VReplication: DropSources: change table rename logic #7230
• VReplication: MoveTables: delete routing rules and update vschema on Complete and Abort #7234
• VReplication: V2 Workflow Start: wait for streams to start and report errors if any while starting a workflow #7248
• VReplication: Ignore temp tables created by onlineddl #7159
• VReplication: Set time zone to UTC while streaming rows #6845
• VReplication: Materialization and character sets: Add test to verify/demo a workaround for charset issues while using string functions in filters #6847
• VReplication: Tool to diagnose vreplication issues in production #6892
• VReplication: Allow multiple blacklists for master #6816
• VStreamer Field Event: add allowed values for set/enum #6981
• VDiff: lock keyspace while snapshoting, restart target in case of errors #7012
• VDiff: make enums comparable #6880
• VDiff: add ability to limit number of rows to compare #6890
• VDiff/Tablet Picker: fix issue where vdiff sleeps occasionally for tablet picker retry interval #6944
• [vtctld]: fix error state in Workflow Show #6970
• [vtctld] Workflow command: minor fixes #7008
• MoveTables: validate that source tables exist, move all tables #7018
• SwitchWrites bug: reverse replication workflows can have wrong start positions #7169

VTTablet

• VTTablet: fast and reliable state transitions #7011
• VTTablet: don't shutdown on too many connections #7039
• VTTablet: debug/env page to change variables in real-time #7189
• VTTablet: Adds better errors when there are timeouts in resource pools #7002
• VTTablet: Return to re-using server IDs for binlog connections #6941
• VTTablet: Correctly initialize the TabletType stats #6989
• Backup: Use pargzip instead of pgzip for compression. #7037
• Backup: Add s3 server-side encryption and decryption with customer provided key #7088

OnlineDDL

• Online DDL: follow ups in multiple trajectories #6901
• Online DDL: cancel running migrations executed by another tablet #7006
• OnlineDDL: Adding ddl_strategy session variable #7042
• Online DDL: ddl_strategy session variable and vtctl command line argument #7045
• Online DDL: Removing online ddl query hint from ALTER TABLE #7069
• Online DDL: vtgate -ddl-strategy flag renamed to -ddl_strategy #7074
  Automatically retry migration that was interrupted during master failover
  Automatically terminate migrations run by a failed tablet
• Online DDL:request_context/migration_context #7082
• Online DDL: Support CREATE, DROP statements in ApplySchema and online DDL #7083
• Online DDL: ddl_type column #7097
• OnlineDDL: "cancel-all" command to cancel all pending migrations in keyspace #7099
• OnlineDDL: Support...

Vitess 9.0.0-rc1

This release complies with VEP-3 which removes the upgrade order requirement. Components can be upgraded in any order. It is recommended that the upgrade order should still be followed if possible, except to canary test the new version of VTGate before upgrading the rest of the components.

Incompatible Changes

The following PRs made changes to behaviors that clients might rely on. They should be reviewed carefully so that client code can be changed in concert with a Vitess release deployment.

• The update to golang 1.15 might break systems that use TLS certificates with a common name. A fix is documented here (golang/go#40748 (comment))

Bugs Fixed

• Set Global #6957
• Binary PK: fix bug where padding of binary columns was being done incorrectly #6963
• Set udv allow more expressions #6964
• Pad non-fractional part of DECIMAL type #6967
• Bug which caused the connection to not close in case of error writing an error packet #6976
• Bug which caused the connection to not close in case of error writing an error packet #6977
• Bug fix #7048 for SelectNone in StreamExecute route engine #7050
• Bug fix regression in /healthz #7090
• Fix metadata related operation hang when zk down #7228

Functionality Added or Changed

• Vtctld: fix error state in Workflow Show #6970
• Adds better error when there are timeouts in resource pools #7002
• Backup: Use pargzip instead of pgzip for compression. #7037
• Vschema: Support back-quoted names #7073
• IsInternalOperationTableName: see if a table is used internally by vitess #7104
• Log sql which exceeds max memory rows #7055
• Add sequence table support for vtexplain #7186
• Vtctld: vtctldclient generator #7238

VTGate / MySQL compatibility

• VTGate: Allow INSERT with all defaults #6969
• VTGate: Allow YEAR column type with length specified #6975
• VTGate: Retry Execute when reserved connection is lost #6983
• VTGate: Use ephemeral buffer when reading rows #6990
• VTGate: Allow time_zone in reserved connection #6998
• VTGate: Improved support for UNION #7007
• VTGate: Add DDL parser support for FULLTEXT indexes. #7001
• VTGate: Extend comparability of EvalResult to support hash codes #7016
• VTGate: Use proto equal method #7017
• VTGate: Fix COM_STMT_EXECUTE packet decode #7020
• VTGate: Adds Planning and Parsing Support for Create Index of MySQL 5.7 #7024
• VTGate: UNION DISTINCT support on vtgate #7029
• VTGate: Improve database ddl plan #7034
• VTGate: Support for hex & shard in vindex query #7044
• VTGate: Use distinct primitive to solve more queries #7047
• VTGate: Route using vindex for composite IN clause #7049
• VTGate: Optimise struct field layout #7052
• VTGate: Refactoring of plan building #7054
• VTGate: Rewrite joins written with the USING construct #6660
• VTGate: Add option to GetSchema to only send the row count and data length over the wire #6985
• VTGate: Adds Planning and Parsing Support for Create Database of MySQL 5.7 #7068
• VTGate: Make sure to check all GROUP BY columns #7080
• VTGate: Separate sub-query and derived table into different structs #7081
• VTGate: Adds Planning and Parsing Support for Alter Database of MySQL 5.7 #7086
• VTGate: Convert usages of DDL struct to DDLStatement interface #7096
• VTGate: Adds Planning and Parsing Support for Drop Database of MySQL 5.7 #7098
• VTGate: Restore SHOW SCHEMAS support; fixes #7100 #7102
• VTGate: Refactor Code to create a separate struct for CREATE TABLE #7116
• VTGate: Allows for vttestserver and vtcombo to respond to VtGateExecute. #7121
• VTGate: Support for lock and unlock tables #7139
• VTGate: Merge SelectDBA routes when possible #7140
• VTGate: Adds support for all the rails queries using information_schema #7143
• VTGate: Add support for unary expression in order by column #7163
• VTGate: Skip query rewriting for dual table #7164
• VTGate: Refactor Code to create a separate struct for ALTER VSCHEMA #7173
• VTGate: Refactor Show plans #7185
• VTGate: Show privilege support #7194
• VTGate: Planning and Parsing Support for Drop Table, Drop View and Alter View #7178
• VTGate: Cache only dml and select plans #7196
• VTGate: Planning and Parsing Support for Alter Table #7199
• VTGate: Add FindAllShardsInKeyspace to vtctldserver #7201
• VTGate: improve-log: FAILED_PRECONDITION #7215

VReplication

• VReplication: refactored and enhanced support for JSON columns #6829
• VReplication: Don't update tx timestamp on heartbeat #6930
• VReplication E2E Tests: Refactored tests for readability and attempting to fix flakiness #6991
• VRepl/Tablet Picker: improve observability of selected tablet #6999
• VReplication: Handle comment statement type in vstreamer #7092
• VReplication e2e: Fine tuned test to reduce flakiness and added more logging to debug future flakiness #7138
• VReplication: Make relay log size & rows configurable. #6992
• VReplication: New workflows cli UX. Allow reads/writes to be switched independently #7071
• VReplication: DropSources: change table rename logic #7230
• VReplication: MoveTables: delete routing rules and update vschema on Complete and Abort #7234
• VReplication: V2 Workflow Start: wait for streams to start and report errors if any while starting a workflow #7248

VTtablet

• VTtablet: ignore errors if extracted gh-ost binary is identical to installed binary #6928
• VTtablet: fast and reliable state transitions #7011
• VTtablet: don't shutdown on too many connections #7039
• VTtablet: debug/env page to change variables in real-time #7189

VStreamer

• VStreamer Field Event: add allowed values for set/enum #6981

VDiff

• VDiff: lock keyspace while snapshoting, restart target in case of errors #7012

OnlineDDL

• Online DDL: follow ups in multiple trajectories #6901
• Online DDL: cancel running migrations executed by another tablet #7006
• OnlineDDL: Adding ddl_strategy session variable #7042
• Online DDL: ddl_strategy session variable and vtctl command line argument #7045
• Online DDL: Removing online ddl query hint from ALTER TABLE #7069
• Online DDL: vtgate -ddl-strategy flag renamed to -ddl_strategy #7074
  Automatically retry migration that was interrupted during master failover
  Automatically terminate migrations run by a failed tablet
• Online DDL:request_context/migration_context #7082
• Online DDL: Support CREATE, DROP statements in ApplySchema and online DDL #7083
• Online DDL: ddl_type column #7097
• OnlineDDL: "cancel-all" command to cancel all pending migrations in keyspace #7099
• OnlineDDL: Support vtctl OnlineDDL <keyspace> show <context> #7145
• OnlineDDL: Normalizing Online-DDL queries #7153
• Online DDL: ddl_strategy=direct #7172
• Online DDL: Executor database pool size increase #7206
• Online DDL: DROP TABLE translated to RENAME TABLE statement #7221
• Online DDL: Adding @@session_uuid to vtgate; used as 'context' #7263

VTadmin

• VTadmin: Initial vtadmin-api, clusters, and service discovery #7187
• VTadmin: The tiniest possible first implementation of vtadmin-web #7218
• VTadmin: Add cluster protos to discovery and vtsql package constructors #7224
• VTadmin: Add static file service discovery implementation #7229
• VTadmin: Query vtadmin-api from vtadmin-web with fetch + react-query #7239

Other

• Route INFORMATION_SCHEMA queries #6932
• Return to re-using server IDs for binlog connections #6941
• Fix comment typo #6974
• Correctly initialize the TabletType stats #6989
• Clean up plan building test cases #7057
• Add s3 server-side encryption and decryption with customer provided key #7088
• Fix all occurrences of fmt.Sprint(x) where x is int #7244
• local docker: copy examples/common into /vt/common to match MoveTables user guide #7252
• Fix incorrect comments #7257

Examples / Tutorials

Documentation

• Vitess Slack Guidelines v1.0 #6961
• Update docker-compose examples to take advantage of improvements in Vitess #7009
• Adds Planning and Parsing Support for Create Index of MySQL 5.7 #7024
• Do vschema_customer_sharded.json before create_customer_sharded.sql #7210
• Fix comment for IDPool #7212
• Added readme for the demo example #7226

Build Environment Changes

• Fix unit test error #6953, #6993
• Fixing the 5.6 builds of vitess/lite #6960
• Set udv allow more expressions #6964
• Pin mariadb to use mariadb-server-10.2 #6966
• Replace vitess:base with vitess:lite images for docker-compose services #7004
• Fix flakey TestParallelRunnerApprovalFirstRunningSecondRunning test #7014
• Allow custom image tags in compose #7043
• Support statsd for vitess #7072
• Add vtctl to make install-local #7125
• Updating Java unit tests for JDK9+ compatibility #7144
• Add Go Version to Bootstrap Image #7182
• Update Vitess v8.0 images #7174
• Fix broken package ref in UBI docker build #7183
• Convert CentOS extra packages installation to yum instead of downloading #7188
• Make docker_local: fix missing mysql_server package #7213
• Add unit test case to improve test coverage for go/sqltypes/result.go #7227
• Use context from Go's standard library #7235
• Update Golang to 1.15 #7204
• Add linter configuration #7247

Functionality Neutral Changes

• Healthcheck: add unit test for multi-cell replica configurations #6978
• Healthcheck: healthy list should be recomputed when a tablet is removed #7176
• Vtctld Workflow command: minor fixes #7008
• Adds timeout to checking for tablets. #7106
• Remove deprecated vtctl commands, flags and vttablet rpcs #7115
• Fixes comment to mention the existence of reference tables. #7122
• Updated pull request template to add more clarity #7193
• Redact password #7198
• action_repository: no need for http.Request #7124
• Table lifecycle: skip time hint for unspecified states #7151

Vitess 8.0.0

This release complies with VEP-3 which removes the upgrade order requirement. Components can be upgraded in any order. It is recommended that the upgrade order should still be followed if possible, except to canary test the new version of VTGate before upgrading the rest of the components.

Incompatible Changes

The following PRs made changes to behavior that clients might be relying on. They should be reviewed carefully so that client code can be changed in concert with a Vitess release deployment.

• Change error code from 1203 to 1153 for three specific error conditions. #6630
  The three errors being changed:
  When a GRPC message coming back to the vtgate exceeds the configured size limit (configured via grpc_max_message_size)
  If a write is attempting to make a change with a payload that is larger than the configured allowable size
  If a scatter query is attempting to collect and process too many rows in memory within the vtgate to construct a response
• Zero auto-increment mode: This enables inserting a 0 value (and not just NULL) into an auto-increment column to generate sequence values. This matches the default behavior of MySQL. #6749
• Turn off schema tracker by default #6712

Known Issue(s)

• VReplication errors when a fixed-length binary column is used as the sharding key #8080

Bugs Fixed

VTGate / MySQL compatibility

• Fix where clause in information schema with correct database name #6599
• Fix DDL execution on reserved connection without in active transaction #6514
• Fix allow passed table_schema in information_schema queries to vttablet if the keyspace is not found #6653
• Fix information schema for non-existent schema #6667
• Fix reserved connection in autocommit mode on DML #6748
• Fix dbname that needs escaping that we broke in fce4cfd #6794
• MySQL CLI, mysql --ssl does not work in MySQL 8.0 #5556
• OperationalError: (_mysql_exceptions.OperationalError) (2013, 'Lost connection to MySQL server during query') #6208
• Fix support for show tables where tables_in_db to use keyspace name, not underlying db name #6446
• Common empty result set query on MySQL returns a row in Vitess #6663
• Operator precedence issue #6734
• Table names out-of-order between replica and primary #6738
• Close Idle reserved connections and Rollback Idle transactions #6552
• Minor Vindex fixes: #6526
• VTgate does not report autocommit system variable and SERVER_STATUS_IN_TRANS flag correctly #5825
• MariaDB JDBC Connector fails to connect to vtgate with default options #5851
• VTgate panics if there are no vttablets and vtctlds #6117
• Remove shard session held in vtgate for reserved connection on connection failure #6522
• Session variable does not work as expected in 7.0 #6559
• Retry should not happen when in dedicated connection #6523
• Correctly report AUTOCOMMIT status in network packets #6551
• Close Idle reserved connections and Rollback Idle transactions #6552
• Reset Session for Reserved Connection Query Failure #6673
• Errors in multi-statement queries not fatal to statement sequence #6747 #6808
• Lock wait timeout on DELETE from table using lookup index #6698
• Application couldn't connect to external db giving error tabletserver.go:1522] Code: INVALID_ARGUMENT syntax error at position 33 near ':vtg2' (CallerID: gnocchi) : Sql: "select convert(:vtg1, CHAR(:vtg2)) as anon_1 from dual", #6466
• The client got the wrong database name #6753
• DBAPIError exception wrapped from (pymysql.err.InternalError) (1105, u'vtgate: http://gnocchi-gnocchizone-vtgate:15000/: reserved connections are not supported on old gen gateway') #6577
• Provide support for “SELECT INTO OUTFILE S3” #6811
• Fix error around breaking of multi statements #6824
• Support SHOW TABLE STATUS FROM db type queries in vtgate #6354
• SHOW FULL FIELDS doesn't follow routing rules #6803
• VDiff/Tablet Picker: fix picker retry logic to handle canceled context #6954
• Healthcheck: Use isIncluded correctly to fix replica/rdonly routing bug #6922

Other

• VReplication tablet_picker should keep trying to find a tablet until context expires #6546
• VReplication improve reverse workflow: update cells/tablet_types #6556
• VDiff: fix panic for tables with a unicode_loose_md5 vindex #6640
• Vtctld UI: Add mysqld port display back to vtctld web UI #6598 #6597
• Vtctld: Fix workflow CopyState fetching (and thus printing). #6657
• Fix unsaved event regression #6710
• Change tracker to use allprivs instead of dba #6720
• Fixes long wait filter keyspace #6721
• s3 ListBackups doesn’t work from root directory #6732
• Vtctld: ExecuteFetchAsDba and ExecuteFetchAsAllPrivs do not escape db names. #6545
• Can't switch on restore_from_backup on existing shard with no tables #4896
• Refactor conn.go #6818

Functionality Added or Changed

VTGate / MySQL compatibility

• Add support for DELETE IGNORE #6722
• Only take on simple dual queries in the vtgate #6666
• Support mycli access to vtgate #4365
• Add support for unicode_loose_xxhash #6457
• Provide virtual information_schema #5394
• information_schema.tables_schema comparison with dynamic value no longer works #6827
• Mysqldump: Allow setting multiple variables in one statement #5401
• Block lock function usage other than dual table #6470
• Lock Session Support #6517
• Make sure lookup vindexes are queryable inside transaction #6499
• Add unicode_loose_xxhash Vindex type #6549
• Support vindex update or delete when destination is provided by client in dml query #6554
• Make sure to handle EXPLAIN on tablets #6579
• Rewrite SHOW TABLES #6615
• Make offset work in OLAP mode #6655
• Support sql_calc_found_rows with limit in sharded keyspace #6680
• Lock shard session heartbeat #6683
• Mysql -ssl tag is deprecated in the later versions of mysql #6682
• Use statement support in olap mode #6692
• Delete row using consistent lookup vindex in where clause #6700
• Add vtexplain support for non-evenly sharded keyspaces #6705
• Make sure dual queries set found_rows correctly #6718
• Add if not exists support to database ddl in parsing #6724
• Allow Update of same vindex using same vindex lookup on consistent lookup vindex #6736
• Update Vindex only on changes #6737
• Add support for SHOW (DATABASES|VITESS_SHARDS|VITESS_TABLETS) LIKE #6750
• 'CASCADE' support #6743
• Mask database name #6791
• Make sure to backtick schema names properly #6550
• Transaction Limiter: fix quota leak if Begin() errors or if the tx killer runs #6731
• Allow Load Data From S3 #6823

OLAP Functioanlity

• Allow switching between OLAP and OLTP #6691
• Use statement support in olap mode #6692
• Fix error handling in olap mode #6940

Set Statement Support

Set statement support is added in Vitess. There are some system variables which are disabled by default and can be enabled using flag -enable_system_settings on VTGate.These system variables are set on the backing MySQL instance, and will force the connection to be dedicated instead of part of the connection pool.

• Disabled passthrough system variables by default. #6859
• Allow switching workload between OLAP and OLTP #4086 #6691
• Support for multiple session set statement syntax #6508 #6495 #6494
• Allow enumeration in set system variables #6493
• Handle boolean settings better #6501
• Set statements refactored to use plan building #6487
• Add more system settings #6486
• Evaluate system variables #6708

VReplication

• VReplication: _vt.vreplication source column VARBINARY->BLOB, allowing more tables to participate in a workflow #6421
• VReplication and vstreamer performance and error metrics #6519, #6132
• VReplication: Find collations for char primary keys and cast comparison during catchup #6568
• Varchar primary keys with collations can cause vreplication to miss rows #6622
• VReplication: Timestamp w/ Heartbeat #6635
• VReplication: Improve few error/log messages #6669
• VReplication: handle SET statements in the binlog stream #6772
• VReplication: e2e Tests added to CI #6730
• VDiff logs: additional logging around vdiff/vstreamer for observability #6684
• VStream: experimental POC of mysql protocol #6670
• Materialize: Support rollups using a literal value as a column #6733
• Materialization: Ignore non-participating tables that are renamed/deleted #6778
• Materialize: add ability for maintaining accurate table-level aggregates #6767
• Schema Tracker: Turn off schema tracker by default #6712 (breaking change)
• Workflow: Add List-All Command #6533
• Workflow: List -> Show, and Expand Metadata #6544
• Refactor: Cleanup Noisy Workflow Logging #6744
• Schema tracker: Change tracker to use the allprivs user instead of dba #6720
• vtctl: Adding VExec and Workflow commands #6410
• vtctl Workflow cmd: don't expect all shards to have workflow streams #6576
• Reshard: Sort table definitions before comparing them #6765 #6738
• Expose a shard range calculator as a vtctl command GenerateShardRanges #6751 #6752
• VStream: Field event now has all column attributes #6525
• VStream events do not populate field "Flags" on FIELD events #5914
• RFC: VReplication based SplitClone #4604
• Experimental: Mysql Protocol support for VStream #6675
• VReplication: Add metrics around vstreamer and vreplication metrics #6787
• Fix issues with missing tables in vstreams #6789
• Add validations and logs to fix common issues faced during MoveTables/Reshard #6814
• Allow multiple blacklists for master #6816

VTtablet

• VTtablet throttling #6668
  https://vitess.io/docs/reference/features/tablet-throttler/
• TabletManager: publish displayState #6648
• TabletManager: call setReadOnly inside ChangeTabletType #6762
• TabletManager: change how SetReadOnly is called to avoid errors from externally managed tablets #6786
• TabletManager: ca...

Vitess 7.0.3 Patch Update

This update fixes several regressions that were deemed significant enough to be backported to the release branch.

Bugs Fixed

• Vtgate : Should receive healthcheck updates from all tablets in cells_to_watch #6857
• Vtgate : Fix error around breaking of multistatements #6824
• Vtgate : Fix bug in REPLICA/RDONLY routing #6904
• Vttablet : checkNoDB should not require tables to be present #6788

Known Issues

This release includes MySQL 5.7.32 and Percona Xtrabackup 2.4.20 which have a known incompatibility. Percona is working on a resolution.

The issue will manifest if a user using 5.7.32:
* Creates a new compressed table.
* Runs any ALTER TABLE  that uses the algorithm copy (table rebuild) on a compressed table.

Build Changes

• Build : Download zookeeper 3.4.14 from archive site #6868

Vitess 8.0.0-rc1

This release complies with VEP-3 which removes the upgrade order requirement. Components can be upgraded in any order. It is recommended that the upgrade order should still be followed if possible, except to canary test the new version of VTGate before upgrading the rest of the components.

Incompatible Changes

The following PRs made changes to behavior that clients might be relying on. They should be reviewed carefully so that client code can be changed in concert with a Vitess release deployment.

• Change error code from 1203 to 1153 for three specific error conditions. #6630
  The three errors being changed:
  When a GRPC message coming back to the vtgate exceeds the configured size limit (configured via grpc_max_message_size)
  If a write is attempting to make a change with a payload that is larger than the configured allowable size
  If a scatter query is attempting to collect and process too many rows in memory within the vtgate to construct a response
• Zero auto-increment mode: This enables inserting a 0 value (and not just NULL) into an auto-increment column to generate sequence values. This matches the default behavior of MySQL. #6749
• Turn off schema tracker by default #6712

Bugs Fixed

VTGate / MySQL compatibility

• Fix where clause in information schema with correct database name #6599
• Fix DDL execution on reserved connection without in active transaction #6514
• Fix allow passed table_schema in information_schema queries to vttablet if the keyspace is not found #6653
• Fix information schema for non-existent schema #6667
• Fix reserved connection in autocommit mode on DML #6748
• Fix dbname that needs escaping that we broke in fce4cfd #6794
• MySQL CLI, mysql --ssl does not work in MySQL 8.0 #5556
• OperationalError: (_mysql_exceptions.OperationalError) (2013, 'Lost connection to MySQL server during query') #6208
• Fix support for show tables where tables_in_db to use keyspace name, not underlying db name #6446
• Common empty result set query on MySQL returns a row in Vitess #6663
• Operator precedence issue #6734
• Table names out-of-order between replica and primary #6738
• Close Idle reserved connections and Rollback Idle transactions #6552
• Minor Vindex fixes: #6526
• VTgate does not report autocommit system variable and SERVER_STATUS_IN_TRANS flag correctly #5825
• MariaDB JDBC Connector fails to connect to vtgate with default options #5851
• VTgate panics if there are no vttablets and vtctlds #6117
• Remove shard session held in vtgate for reserved connection on connection failure #6522
• Session variable does not work as expected in 7.0 #6559
• Retry should not happen when in dedicated connection #6523
• Correctly report AUTOCOMMIT status in network packets #6551
• Close Idle reserved connections and Rollback Idle transactions #6552
• Reset Session for Reserved Connection Query Failure #6673
• Errors in multi-statement queries not fatal to statement sequence #6747 #6808
• Lock wait timeout on DELETE from table using lookup index #6698
• Application couldn't connect to external db giving error tabletserver.go:1522] Code: INVALID_ARGUMENT syntax error at position 33 near ':vtg2' (CallerID: gnocchi) : Sql: "select convert(:vtg1, CHAR(:vtg2)) as anon_1 from dual", #6466
• The client got the wrong database name #6753
• DBAPIError exception wrapped from (pymysql.err.InternalError) (1105, u'vtgate: http://gnocchi-gnocchizone-vtgate:15000/: reserved connections are not supported on old gen gateway') #6577
• Provide support for “SELECT INTO OUTFILE S3” #6811

Other

• VReplication tablet_picker should keep trying to find a tablet until context expires #6546
• VReplication improve reverse workflow: update cells/tablet_types #6556
• VDiff: fix panic for tables with a unicode_loose_md5 vindex #6640
• Vtctld UI: Add mysqld port display back to vtctld web UI #6598 #6597
• Vtctld: Fix workflow CopyState fetching (and thus printing). #6657
• Fix unsaved event regression #6710
• Change tracker to use allprivs instead of dba #6720
• Fixes long wait filter keyspace #6721
• s3 ListBackups doesn’t work from root directory #6732
• Vtctld: ExecuteFetchAsDba and ExecuteFetchAsAllPrivs do not escape db names. #6545
• Can't switch on restore_from_backup on existing shard with no tables #4896
• Refactor conn.go #6818

Functionality Added or Changed

VTGate / MySQL compatibility

• New functionality to set workload = 'olap'; can not repeat, but set workload = 'oltp' can; #4086

• Add support for DELETE IGNORE #6722

• Only take on simple dual queries in the vtgate #6666

• Support mycli access to vtgate #4365

• Add support for unicode_loose_xxhash #6457

• Provide virtual information_schema #5394

• Mysqldump: Allow setting multiple variables in one statement #5401

• SET planning #6487

• Allow enumeration in set system variables #6493

• Allow set to have scope for each setting #6494

• HandleSet in executor to handle multiple settings in single set statement #6495

• Passthrough system settings flag: enabled by default #6586

• Handle boolean settings better #6501

• Add more system settings #6486

• Block lock function usage other than dual table #6470

• Lock Session Support #6517

• Make sure lookup vindexes are queryable inside transaction #6499

• Add unicode_loose_xxhash Vindex type #6549

• Support vindex update or delete when destination is provided by client in dml query #6554

• Make sure to handle EXPLAIN on tablets #6579

• Rewrite SHOW TABLES #6615

• Make offset work in OLAP mode #6655

• Support sql_calc_found_rows with limit in sharded keyspace #6680

• Lock shard session heartbeat #6683

• Mysql -ssl tag is deprecated in the later versions of mysql #6682

• Allow switching between OLAP and OLTP #6691

• Use statement support in olap mode #6692

• Delete row using consistent lookup vindex in where clause #6700

• Add vtexplain support for non-evenly sharded keyspaces #6705

• Evaluate system variables #6708

• Make sure dual queries set found_rows correctly #6718

• Add if not exists support to database ddl in parsing #6724

• Allow Update of same vindex using same vindex lookup on consistent lookup vindex #6736

• Update Vindex only on changes #6737

• Add support for SHOW (DATABASES|VITESS_SHARDS|VITESS_TABLETS) LIKE #6750

• Support multiple session set statement syntax #6508

• 'CASCADE' support #6743

• Mask database name #6791

• Make sure to backtick schema names properly #6550

• Transaction Limiter: fix quota leak if Begin() errors or if the tx killer runs #6731

OLAP Functioanlity

• Allow switching between OLAP and OLTP #6691
• Use statement support in olap mode #6692

System Variables

• Allow enumeration in set system variables #6493
• Allow set to have scope for each setting #6494
• HandleSet in executor to handle multiple settings in single set statement #6495

VReplication

• VReplication: _vt.vreplication source column VARBINARY->BLOB, allowing more tables to participate in a workflow #6421
• VReplication and vstreamer performance and error metrics #6519, #6132
• VReplication: Find collations for char primary keys and cast comparison during catchup #6568
• Varchar primary keys with collations can cause vreplication to miss rows #6622
• VReplication: Timestamp w/ Heartbeat #6635
• VReplication: Improve few error/log messages #6669
• VReplication: handle SET statements in the binlog stream #6772
• VReplication: e2e Tests added to CI #6730
• VDiff logs: additional logging around vdiff/vstreamer for observability #6684
• VStream: experimental POC of mysql protocol #6670
• Materialize: Support rollups using a literal value as a column #6733
• Materialization: Ignore non-participating tables that are renamed/deleted #6778
• Materialize: add ability for maintaining accurate table-level aggregates #6767
• Schema Tracker: Turn off schema tracker by default #6712 (breaking change)
• Workflow: Add List-All Command #6533
• Workflow: List -> Show, and Expand Metadata #6544
• Refactor: Cleanup Noisy Workflow Logging #6744
• Schema tracker: Change tracker to use the allprivs user instead of dba #6720
• vtctl: Adding VExec and Workflow commands #6410
• vtctl Workflow cmd: don't expect all shards to have workflow streams #6576
• Reshard: Sort table definitions before comparing them #6765 #6738
• Expose a shard range calculator as a vtctl command GenerateShardRanges #6751 #6752
• VStream: Field event now has all column attributes #6525
• VStream events do not populate field "Flags" on FIELD events #5914
• RFC: VReplication based SplitClone #4604
• Experimental: Mysql Protocol support for VStream #6675
• VReplication: Add metrics around vstreamer and vreplication metrics #6787
• Fix issues with missing tables in vstreams #6789
• Add validations and logs to fix common issues faced during MoveTables/Reshard #6814

VTtablet

• VTtablet throttling #6668
• TabletManager: publish displayState #6648
• TabletManager: call setReadOnly inside ChangeTabletType #6762
• TabletManager: change how SetReadOnly is called to avoid errors from externally managed tablets #6786
• TabletManager: call SetReadOnly inside ChangeTabletType #6804
• vttablet: Open and Close healthStreamer #6515
• vttablet: create database if not present #6490
• Restore: do not change tablet type to RESTORE if not actually performing a restore #6679
• Restore: checkNoDB should not require tables to be present #6695
• Heartbeat: Additional metrics/stats (lag histogram) #6528 #6634
• VTtablet throttler feature flag: -enable-lag-throttler #6815

VTorc - Orchestrator

• Orchestrator: initial import of Orchestrator #6582
• Orchestrator: vitess mode #6613
• Orchestrator: more changes #6714
• Orchestrator: use contexts with timeout for remote operations #6780
• Orchestrator: fixed orchestrator govet errors #6781
• Orchestrator: Add clusters_to_watch flag to. Defaults t...

Vitess 7.0.2 Patch Update

This update fixes several regressions that were deemed significant enough to be backported to the release branch.

Bugs Fixed

• vtgate : Operator precedence must take associativity into consideration #6764
• vtgate: Fix reserved connection in autocommit mode on DML #6748
• VDiff: fix panic for tables with a unicode_loose_md5 vindex #6745
• vttablet: Turn off schema tracker by default #6746
• restore: do not change tablet type to RESTORE if not actually performing a restore #6687
• vttablet : Tablet manager init should publish displayState #6686
• vttablet: Tablet manager add more logging to checkMastership #6685
• vttablet : Fix replmanager deadlock #6634

Vitess 7.0.1 patch update

This update fixes several regressions that were deemed significant enough to be backported to the release branch.
In addition, the experimental feature to set system variables is now behind a flag that disables it by default.

Configuration Changes

• Vtgate: added flag enable_system_settings to enable passthrough system settings, defaults to false #6562

Bugs Fixed

• Vreplication: Materialize should only get schema from source tablets if target is missing tables #6623
• Vtctld UI: mysql port is not displayed in tablet list #6597 #6600
• Vtgate: transaction killer was closing reserved connection after the specified time. Only idle connections should be closed. Idle transactions should be rolled back. #6583
• Vtcompose: docker-compose example was broken by tabletmanager changes #6587
• Vreplication: tablet_picker should respect canceled context. Also, keep trying to find a tablet until context expires. #6567
• Vtgate: vindex update/delete by destination #6541 #6572
• Vtgate: some usages of EXPLAIN were broken #6574 #6581
• Vtgate: handle multiple settings in single SET statement #6504
• Vtgate: cleanup shard session for reserved connection on failure #6563
• Vtgate: if executing DDL on reserved connection, start transaction if needed #6555
• Vtgate: make sure lookup vindexes use same transaction if one is open #6505
• Healthcheck: add diagnostic logging, do not add tablet if it is shutting down #6512

Vitess 6.0.20-20200818 Patch Update

This update fixes two regressions that were deemed significant enough to be backported to the release branch.

Bugs Fixed

• Vtgate: delete by destination broken for tables with owned vindexes #6541 #6554
• Vtgate: set statements with multiple settings should not error out #6566