From 885b362a8e7c4edbedb3f02f27ab09f062f77e98 Mon Sep 17 00:00:00 2001 From: Matt Lord Date: Tue, 6 Jun 2023 10:16:23 -0400 Subject: [PATCH 1/2] Copy v17.0.0-rc changelog to main Signed-off-by: Matt Lord --- changelog/17.0/17.0.0/changelog.md | 526 +++++++++++++++++++++++++ changelog/17.0/17.0.0/release_notes.md | 460 +++++++++++++++++++++ changelog/17.0/17.0.0/summary.md | 15 + 3 files changed, 1001 insertions(+) create mode 100644 changelog/17.0/17.0.0/changelog.md create mode 100644 changelog/17.0/17.0.0/release_notes.md diff --git a/changelog/17.0/17.0.0/changelog.md b/changelog/17.0/17.0.0/changelog.md new file mode 100644 index 00000000000..aad345de2c7 --- /dev/null +++ b/changelog/17.0/17.0.0/changelog.md @@ -0,0 +1,526 @@ +# Changelog of Vitess v17.0.0-rc1 + +### Announcement +#### Governance + * maintainers: add dbussink and rework some sections [#12329](https://github.com/vitessio/vitess/pull/12329) +### Bug fixes +#### Backup and Restore + * go/vt/mysqlctl/backupstats: increment *_bytes metric by actual number of bytes [#12501](https://github.com/vitessio/vitess/pull/12501) + * Avoid crash during BuiltinBackup FileUpload method [#12703](https://github.com/vitessio/vitess/pull/12703) + * do not demote a new primary after backup completion [#12856](https://github.com/vitessio/vitess/pull/12856) + * go/vt/mysqlctl: fix dup open/close backup stats [#12947](https://github.com/vitessio/vitess/pull/12947) + * Incremental backup: fix calculation of binlog files to use [#13066](https://github.com/vitessio/vitess/pull/13066) + * restore: Run PITR through mysqlctl [#13123](https://github.com/vitessio/vitess/pull/13123) +#### Build/CI + * Prevent bootstrap of ZK from clearing previous dist binaries [#12236](https://github.com/vitessio/vitess/pull/12236) + * Fix dubious ownership of git directory in `vitess/base` Docker build [#12530](https://github.com/vitessio/vitess/pull/12530) + * Small fixes to the auto-upgrade golang tool [#12838](https://github.com/vitessio/vitess/pull/12838) + * Add timeout to `golangci-lint` and bump its version [#12852](https://github.com/vitessio/vitess/pull/12852) + * codegen: ensure goimports also simplifies [#12883](https://github.com/vitessio/vitess/pull/12883) +#### CLI + * Purge logs without panicking [#12187](https://github.com/vitessio/vitess/pull/12187) + * Fix `vtctldclient`'s Root command to return an error on unknown command [#12481](https://github.com/vitessio/vitess/pull/12481) + * vtctldclient: Format GetKeyspace output using cli.MarshalJSON [#12495](https://github.com/vitessio/vitess/pull/12495) + * VReplication: Pass on --keep_routing_rules flag value for Cancel action [#13171](https://github.com/vitessio/vitess/pull/13171) +#### Cluster management + * Fix initialization code to also stop replication to prevent crash [#12534](https://github.com/vitessio/vitess/pull/12534) + * Update topo {Get,Create}Keyspace to prevent invalid keyspace names [#12732](https://github.com/vitessio/vitess/pull/12732) + * Flaky TestRestart: give mysqlctld process time to shutdown [#12799](https://github.com/vitessio/vitess/pull/12799) + * Fix `TestGatewayBufferingWhileReparenting` flakiness [#12817](https://github.com/vitessio/vitess/pull/12817) + * [topo] Disallow the slash character in shard names [#12843](https://github.com/vitessio/vitess/pull/12843) + * TabletThrottler: `vtctlclient UpdateThrottlerConfig` use VtctldServer RPC [#12936](https://github.com/vitessio/vitess/pull/12936) + * Keep retrying mysql port if it is 0 in Vttablets [#12986](https://github.com/vitessio/vitess/pull/12986) + * Fix: convertBoolToSemiSyncAction method to account for all semi sync actions [#13075](https://github.com/vitessio/vitess/pull/13075) +#### Evalengine + * evalengine: fix bugs with decimal rounding [#12360](https://github.com/vitessio/vitess/pull/12360) + * json: Fix JSON SQL serialization and ensure to run tests [#12861](https://github.com/vitessio/vitess/pull/12861) + * evalengine: unset Type for NewColumnWithCollation [#12886](https://github.com/vitessio/vitess/pull/12886) + * datetime: Fix weekday calculation for DST changes [#12931](https://github.com/vitessio/vitess/pull/12931) + * evalengine: Fix wrong coercion into float [#12998](https://github.com/vitessio/vitess/pull/12998) + * evalengine: TypeOf for Columns should only use value type when we have a value [#13148](https://github.com/vitessio/vitess/pull/13148) +#### Examples + * Examples: Add lib functions to wait for shard states [#12239](https://github.com/vitessio/vitess/pull/12239) + * Fixes for local docker environment [#12382](https://github.com/vitessio/vitess/pull/12382) + * Examples: Correct VTAdmin Discovery File Path And Add Check [#12415](https://github.com/vitessio/vitess/pull/12415) + * Fix Vtop example and release script [#12457](https://github.com/vitessio/vitess/pull/12457) + * Open vtadmin-api port in docker-local example to enable vtadmin access [#12467](https://github.com/vitessio/vitess/pull/12467) + * Use new init_db.sql file in all places and update v17.0.0 summary [#12716](https://github.com/vitessio/vitess/pull/12716) + * Use $hostname in vtadmin script as all other scripts do (#13231) [#13236](https://github.com/vitessio/vitess/pull/13236) +#### General + * Fix shorthand flag for recursive delete in zk tool [#12460](https://github.com/vitessio/vitess/pull/12460) + * Forward port of #12436: Sidecar schema init: use COPY algorithm while altering sidecardb tables [#12464](https://github.com/vitessio/vitess/pull/12464) + * Fixing backup tests flakiness [#12655](https://github.com/vitessio/vitess/pull/12655) + * Fix backup flaky tests [#12834](https://github.com/vitessio/vitess/pull/12834) + * Fix auto-upgrade-golang tooling [#13048](https://github.com/vitessio/vitess/pull/13048) + * Flakes: Skip flaky viperutil test TestPersistConfig for now to unblock CI [#13186](https://github.com/vitessio/vitess/pull/13186) + * Temporarily disable panic in mysqld when socket files are directly used [#13198](https://github.com/vitessio/vitess/pull/13198) +#### Observability + * Reset the current lag when closing the replication lag reader. [#12683](https://github.com/vitessio/vitess/pull/12683) +#### Online DDL + * Allow zero (in) date when setting up internal _vt schema [#12262](https://github.com/vitessio/vitess/pull/12262) + * Online DDL: improve retry of vreplication errors with `vitess` `ALTER TABLE` migrations [#12323](https://github.com/vitessio/vitess/pull/12323) + * OnlineDDL: mitigate scenario where a migration sees recurring cut-over timeouts [#12451](https://github.com/vitessio/vitess/pull/12451) + * Throttler: Store Config in Global Keyspace Topo Record [#12520](https://github.com/vitessio/vitess/pull/12520) + * Online DDL: enforce ALGORITHM=COPY on shadow table [#12521](https://github.com/vitessio/vitess/pull/12521) + * Online DDL: `ready_to_complete` race fix [#12612](https://github.com/vitessio/vitess/pull/12612) + * Mysqld.GetSchema: tolerate tables being dropped while inspecting schema [#12641](https://github.com/vitessio/vitess/pull/12641) + * Flaky test fix: OnlineDDL: convert time->unix->TIMESTAMP in golang space [#12826](https://github.com/vitessio/vitess/pull/12826) + * OnlineDDL: reject partial key coverage in PRIMARY KEY for vitess migrations [#12921](https://github.com/vitessio/vitess/pull/12921) +#### Operator + * Use latest k8s for operator example to work [#13085](https://github.com/vitessio/vitess/pull/13085) + * Upgrade mysqld memory limits to 1024Mi [#13122](https://github.com/vitessio/vitess/pull/13122) +#### Query Serving + * fix dual table handling [#12204](https://github.com/vitessio/vitess/pull/12204) + * grpcvtgateconn: add Dial and deprecate DialWithOpts [#12210](https://github.com/vitessio/vitess/pull/12210) + * Subquery bugfix [#12254](https://github.com/vitessio/vitess/pull/12254) + * fix: transaction_isolation to be applied at session level [#12281](https://github.com/vitessio/vitess/pull/12281) + * fix dual table handling [#12292](https://github.com/vitessio/vitess/pull/12292) + * fix: added null safe operator precendence rule [#12297](https://github.com/vitessio/vitess/pull/12297) + * Fix aggregation on outer joins [#12298](https://github.com/vitessio/vitess/pull/12298) + * (re)Formalize SQLError in VReplication, add underlying wrap/unwrap functionality [#12327](https://github.com/vitessio/vitess/pull/12327) + * sqlparser/schemadiff: normalize CAST type case [#12372](https://github.com/vitessio/vitess/pull/12372) + * Fix bug in vtexplain around JOINs [#12376](https://github.com/vitessio/vitess/pull/12376) + * Fix for USING when column names not lower cased [#12378](https://github.com/vitessio/vitess/pull/12378) + * Fix additional casting bugs in the sqlparser [#12393](https://github.com/vitessio/vitess/pull/12393) + * Fix scalar aggregation engine primitive for column truncation [#12468](https://github.com/vitessio/vitess/pull/12468) + * [release-16.0] BugFix: Unsharded query using a derived table and a dual table [#12484](https://github.com/vitessio/vitess/pull/12484) + * `ApplyVSchemaDDL`: escape Sequence names when writing the VSchema [#12519](https://github.com/vitessio/vitess/pull/12519) + * collations: fix sorting in UCA900 collations [#12555](https://github.com/vitessio/vitess/pull/12555) + * VSchema DDL: Add grammar to accept qualified table names in Vindex option values [#12577](https://github.com/vitessio/vitess/pull/12577) + * [gen4 planner] Make sure to not push down expressions when not possible [#12607](https://github.com/vitessio/vitess/pull/12607) + * Fix `panic` when executing a prepare statement with over `65,528` parameters [#12614](https://github.com/vitessio/vitess/pull/12614) + * Always add columns in the `Derived` operator [#12634](https://github.com/vitessio/vitess/pull/12634) + * planner: fix predicate simplifier [#12650](https://github.com/vitessio/vitess/pull/12650) + * [planner bugfix] add expressions to HAVING [#12668](https://github.com/vitessio/vitess/pull/12668) + * Use a left join to make sure that tables with tablespace=innodb_system are included in the schema [#12672](https://github.com/vitessio/vitess/pull/12672) + * [planner fix] make unknown column an error only for sharded queries [#12704](https://github.com/vitessio/vitess/pull/12704) + * Fix handling of table information for missing column [#12705](https://github.com/vitessio/vitess/pull/12705) + * Fix the returned schema object for broken views [#12713](https://github.com/vitessio/vitess/pull/12713) + * Fix `vtgate_schema_tracker` flaky tests [#12780](https://github.com/vitessio/vitess/pull/12780) + * [planbuilder bugfix] do not push aggregations into derived tables [#12810](https://github.com/vitessio/vitess/pull/12810) + * Fix: reset transaction session when no reserved connection [#12878](https://github.com/vitessio/vitess/pull/12878) + * fix: union distinct between unsharded route and sharded join [#12968](https://github.com/vitessio/vitess/pull/12968) + * bugfix: fix nil pointer in vtgate on topo connection error [#13010](https://github.com/vitessio/vitess/pull/13010) + * Cleanup rate limiters [#13016](https://github.com/vitessio/vitess/pull/13016) + * restrict flush statement on non primary tablets [#13042](https://github.com/vitessio/vitess/pull/13042) + * Fix the resilientQuery to give correct results during initialization [#13080](https://github.com/vitessio/vitess/pull/13080) + * Fix flakiness in `TestGatewayBufferingWhileReparenting` [#13106](https://github.com/vitessio/vitess/pull/13106) + * sqlparser: Treat view names as case sensitive [#13107](https://github.com/vitessio/vitess/pull/13107) + * Add a timeout to prevent unbounded waits that cause the primary tablet's health check stuck [#13111](https://github.com/vitessio/vitess/pull/13111) + * VTGate StreamExecute rpc to return session as response [#13131](https://github.com/vitessio/vitess/pull/13131) + * Remove the `200` indentation limit in the SQL parser [#13158](https://github.com/vitessio/vitess/pull/13158) + * vindexes: make lookup_unicodeloosemd5_hash non-planable [#13200](https://github.com/vitessio/vitess/pull/13200) +#### TabletManager + * TableFilter: fix excluded RE listing [#12318](https://github.com/vitessio/vitess/pull/12318) + * TableGC: fix PURGE race condition [#12505](https://github.com/vitessio/vitess/pull/12505) + * Fix transaction throttler ignoring the initial rate [#12618](https://github.com/vitessio/vitess/pull/12618) + * Skip recalculating the rate in MaxReplicationLagModule when it can't be done [#12620](https://github.com/vitessio/vitess/pull/12620) +#### VReplication + * VReplication Last Error: retry error if it happens after timeout [#12114](https://github.com/vitessio/vitess/pull/12114) + * VReplication: workflows with multiple streams fail to start on multi-primary setups [#12228](https://github.com/vitessio/vitess/pull/12228) + * VReplication: ignore GC tables in schema analysis [#12320](https://github.com/vitessio/vitess/pull/12320) + * VSCopy: Ensure that vgtid event includes a complete TableLastPK proto message [#12623](https://github.com/vitessio/vitess/pull/12623) + * VDiff: use PK column collation for proper ordering [#12845](https://github.com/vitessio/vitess/pull/12845) + * Fix bug in `SwitchTraffic` that wasn't respecting `--dry_run` for readonly and replica tablets during a resharding event [#12992](https://github.com/vitessio/vitess/pull/12992) + * Fix vreplication_log usage in VReplicationExec [#13038](https://github.com/vitessio/vitess/pull/13038) + * ApplySchema vttablet RPC: allow special characters in table and column names [#13054](https://github.com/vitessio/vitess/pull/13054) + * Purge old schema versions from memory in historian [#13056](https://github.com/vitessio/vitess/pull/13056) + * Flaky vplayer tests: temporarily disable noblob variant [#13100](https://github.com/vitessio/vitess/pull/13100) + * VReplication: Do not delete sharded target vschema table entries on Cancel [#13146](https://github.com/vitessio/vitess/pull/13146) + * VReplication: Handle reference tables in sharded->sharded MoveTables [#13161](https://github.com/vitessio/vitess/pull/13161) +#### VTAdmin + * [vtadmin] racy vtexplain [#12635](https://github.com/vitessio/vitess/pull/12635) + * fix: vtadmin vttablet url protocol [#12836](https://github.com/vitessio/vitess/pull/12836) + * update vite base to respect absolute path [#13110](https://github.com/vitessio/vitess/pull/13110) +#### VTorc + * Fix Vtorc recovery time during `DeadPrimary` [#12870](https://github.com/vitessio/vitess/pull/12870) + * Add new field to health status [#12942](https://github.com/vitessio/vitess/pull/12942) +### CI/Build +#### Build/CI + * Makefile: use buildx when GOOS/GOARCH are specified [#12081](https://github.com/vitessio/vitess/pull/12081) + * onlineddl_vrepl suite: fix auto_increment flakyness [#12246](https://github.com/vitessio/vitess/pull/12246) + * Use a working MariaDB mirror for CI [#12253](https://github.com/vitessio/vitess/pull/12253) + * Download mariadb from vitess-resources [#12271](https://github.com/vitessio/vitess/pull/12271) + * Bump the vitess-resources release increment from v2 to v4 [#12272](https://github.com/vitessio/vitess/pull/12272) + * sqlparser: Tools cleanup [#12407](https://github.com/vitessio/vitess/pull/12407) + * Make upgrade downgrade job names unique [#12485](https://github.com/vitessio/vitess/pull/12485) + * Update fossa check for Node deprecation [#12871](https://github.com/vitessio/vitess/pull/12871) + * Flakes: Wait for throttler to open in throttler custom config test [#12980](https://github.com/vitessio/vitess/pull/12980) + * Add minimal token permissions (continued) [#12996](https://github.com/vitessio/vitess/pull/12996) + * workflows: Update to latest actions/setup-go [#13023](https://github.com/vitessio/vitess/pull/13023) + * CI: add a check on NeedsIssue label [#13044](https://github.com/vitessio/vitess/pull/13044) +#### Examples + * Examples, Flakes: Wait for Shard's VReplication Engine to Open [#12560](https://github.com/vitessio/vitess/pull/12560) +#### General + * Bump reported version of main to 17.0.0-SNAPSHOT [#12234](https://github.com/vitessio/vitess/pull/12234) + * Upgrade the Golang version to `go1.20.2` [#12706](https://github.com/vitessio/vitess/pull/12706) + * Do not fail build on incorrect Go version [#12809](https://github.com/vitessio/vitess/pull/12809) + * Upgrade the Golang version to `go1.20.3` [#12833](https://github.com/vitessio/vitess/pull/12833) + * Remove misleading message in `build.env` [#12841](https://github.com/vitessio/vitess/pull/12841) + * Upgrade the Golang version to `go1.20.4` [#13052](https://github.com/vitessio/vitess/pull/13052) + * Code owners update [#13070](https://github.com/vitessio/vitess/pull/13070) +#### Online DDL + * Fixing `onlineddl_vrepl` flakiness, and adding more tests [#12325](https://github.com/vitessio/vitess/pull/12325) + * CI: extend timeouts in onlineddl_vrepl due to slow CI runners [#12583](https://github.com/vitessio/vitess/pull/12583) + * CI: increase overall test timeouts for all OnlineDDL tests [#12584](https://github.com/vitessio/vitess/pull/12584) +#### Query Serving + * Fix `SHOW VSCHEMA TABLES` tests using v17 vtgate that expected `dual` [#12381](https://github.com/vitessio/vitess/pull/12381) + * go/vt/vttablet/tabletserver: temporarily skip flaky consolidation test [#12604](https://github.com/vitessio/vitess/pull/12604) +#### VTAdmin + * Remove `nvm use` from vtadmin-up.sh script [#12788](https://github.com/vitessio/vitess/pull/12788) +### Dependabot +#### Build/CI + * Bump golang.org/x/net from 0.5.0 to 0.7.0 [#12390](https://github.com/vitessio/vitess/pull/12390) + * Update additional Go dependencies [#12401](https://github.com/vitessio/vitess/pull/12401) + * Fix another mixin dep [#12403](https://github.com/vitessio/vitess/pull/12403) +#### Observability + * Bump vitess.io/vitess from 0.15.2 to 0.16.1 in /vitess-mixin [#12875](https://github.com/vitessio/vitess/pull/12875) + * build(deps): bump vitess.io/vitess from 0.16.1 to 0.16.2 in /vitess-mixin [#13076](https://github.com/vitessio/vitess/pull/13076) +#### VTAdmin + * Bump dns-packet from 5.3.1 to 5.4.0 in /web/vtadmin [#12545](https://github.com/vitessio/vitess/pull/12545) + * Bump webpack from 5.65.0 to 5.76.1 in /web/vtadmin [#12632](https://github.com/vitessio/vitess/pull/12632) +### Documentation +#### Cluster management + * fix help text for backup_storage_number_blocks [#12258](https://github.com/vitessio/vitess/pull/12258) +#### Documentation + * release notes and misc cleanup around tabletenv flags [#12953](https://github.com/vitessio/vitess/pull/12953) + * Add Vinted to the list of adopters [#13134](https://github.com/vitessio/vitess/pull/13134) +#### Query Serving + * Copy remaining query serving design docs [#12170](https://github.com/vitessio/vitess/pull/12170) + * Add release notes summary for views [#12422](https://github.com/vitessio/vitess/pull/12422) + * SQLError: adding some code comments [#12613](https://github.com/vitessio/vitess/pull/12613) + * update v16 release notes about VTGate Advertised MySQL Version [#12957](https://github.com/vitessio/vitess/pull/12957) +### Enhancement +#### ACL + * remove tablet server's ACL check on the /healthz HTTP api route [#12897](https://github.com/vitessio/vitess/pull/12897) +#### Backup and Restore + * vtbackup, mysqlctl: detailed backup and restore metrics [#11979](https://github.com/vitessio/vitess/pull/11979) + * go/vt/mysqlctl: add configurable read buffer to builtin backups [#12073](https://github.com/vitessio/vitess/pull/12073) + * go/{vt,flags}: register builtin backup flags with vtbackup [#12558](https://github.com/vitessio/vitess/pull/12558) + * go/vt/mysqlctl: optionally store decompressor cmd in manifest [#12633](https://github.com/vitessio/vitess/pull/12633) + * Fix restore from backup execution path to use context from caller [#12828](https://github.com/vitessio/vitess/pull/12828) + * Incremental backup and point in time recovery for XtraBackup [#13156](https://github.com/vitessio/vitess/pull/13156) +#### Build/CI + * Revert default MySQL 80 version to `8.0.30` [#12252](https://github.com/vitessio/vitess/pull/12252) + * Block merge if the `Do Not Merge` label is set [#12489](https://github.com/vitessio/vitess/pull/12489) + * Auto upgrade the Golang version [#12585](https://github.com/vitessio/vitess/pull/12585) + * Add read-only token permissions to GitHub Action workflows [#12718](https://github.com/vitessio/vitess/pull/12718) + * Remove unnecessary code bits in workflows [#12756](https://github.com/vitessio/vitess/pull/12756) + * tools/build_version_flags: allow configurable `build time` [#12876](https://github.com/vitessio/vitess/pull/12876) +#### CLI + * [flags] extract tabletenv seconds type [#12920](https://github.com/vitessio/vitess/pull/12920) +#### Cluster management + * Introducing new error counter in VTTablet [#12266](https://github.com/vitessio/vitess/pull/12266) + * Auto refresh tables on vtgate Status Page [#12335](https://github.com/vitessio/vitess/pull/12335) + * Add configuration option for minimum TLS version [#12424](https://github.com/vitessio/vitess/pull/12424) + * Add tablet type filtering to vtctldclient GetTablets [#12915](https://github.com/vitessio/vitess/pull/12915) + * topo: Move to allow list for invalid names [#12917](https://github.com/vitessio/vitess/pull/12917) + * Add vstream metrics to vtgate [#13098](https://github.com/vitessio/vitess/pull/13098) +#### Evalengine + * evalengine: virtual machine [#12369](https://github.com/vitessio/vitess/pull/12369) + * Add 128 bit hashing for the evalengine [#12428](https://github.com/vitessio/vitess/pull/12428) + * evalengine: add more coercion paths to Hash128 [#12463](https://github.com/vitessio/vitess/pull/12463) + * evalengine: Implement integer division and modulo [#12656](https://github.com/vitessio/vitess/pull/12656) + * [ast & semantics] Make more type information available [#12710](https://github.com/vitessio/vitess/pull/12710) + * evalengine: New integration API [#12724](https://github.com/vitessio/vitess/pull/12724) + * vtgate: use collation when hashing query plans [#12728](https://github.com/vitessio/vitess/pull/12728) + * Implement logic to handle more JSON conversion [#12733](https://github.com/vitessio/vitess/pull/12733) + * evalengine: Add handling for current timestamp functions [#12742](https://github.com/vitessio/vitess/pull/12742) + * evalengine: Add support for more arithmetic functions [#12778](https://github.com/vitessio/vitess/pull/12778) + * evalengine: Fix handling of datetime and numeric comparisons [#12789](https://github.com/vitessio/vitess/pull/12789) + * evalengine: Implement ROUND() and TRUNCATE() [#12797](https://github.com/vitessio/vitess/pull/12797) + * evalengine: Implement CRC32 & CONV [#12804](https://github.com/vitessio/vitess/pull/12804) + * evalengine: Improve the timezone parsing logic [#12827](https://github.com/vitessio/vitess/pull/12827) + * evalengine: Implement initial crypto functions [#12835](https://github.com/vitessio/vitess/pull/12835) + * datetime: Add week number formatting [#12884](https://github.com/vitessio/vitess/pull/12884) + * evalengine: Add CONVERT_TZ function [#12892](https://github.com/vitessio/vitess/pull/12892) + * evalengine: Improve handling of datetime / time with precision and add more functions [#12907](https://github.com/vitessio/vitess/pull/12907) + * evalengine: Implement additional week modes [#12916](https://github.com/vitessio/vitess/pull/12916) + * evalengine: Add support for `UNIX_TIMESTAMP & `FROM_UNIXTIME` [#12928](https://github.com/vitessio/vitess/pull/12928) + * evalengine: Implement `MAKEDATE` and `MAKETIME`. [#12938](https://github.com/vitessio/vitess/pull/12938) + * evalengine: Implement LEFT, RIGHT, LPAD & RPAD [#13013](https://github.com/vitessio/vitess/pull/13013) + * evalengine: Implement TRIM, RTRIM and LTRIM [#13024](https://github.com/vitessio/vitess/pull/13024) + * evalengine: Implement UNHEX [#13028](https://github.com/vitessio/vitess/pull/13028) + * evalengine: Implement ORD [#13029](https://github.com/vitessio/vitess/pull/13029) + * evalengine: Add support for `STRCMP` [#13063](https://github.com/vitessio/vitess/pull/13063) + * evalengine: Add support for handling `IF` [#13067](https://github.com/vitessio/vitess/pull/13067) + * evalengine: `INTERVAL` support [#13073](https://github.com/vitessio/vitess/pull/13073) + * evalengine: Implement IP related functions [#13082](https://github.com/vitessio/vitess/pull/13082) + * evalengine: Implement `CONCAT` and `CONCAT_WS` [#13091](https://github.com/vitessio/vitess/pull/13091) + * evalengine: Add UUID functions [#13097](https://github.com/vitessio/vitess/pull/13097) +#### Examples + * examples schema: demi-idempotency via CREATE TABLE IF NOT EXISTS [#12453](https://github.com/vitessio/vitess/pull/12453) +#### General + * Upgrade to `go1.20.1` [#12399](https://github.com/vitessio/vitess/pull/12399) + * adding log statment to vtbackup [#12569](https://github.com/vitessio/vitess/pull/12569) + * Automatically add milestone to new Pull Request [#12759](https://github.com/vitessio/vitess/pull/12759) +#### Observability + * Add `Uptime` metric [#12712](https://github.com/vitessio/vitess/pull/12712) + * go/{stats,vt}: publish VReplicationStreamState to prometheus backend [#12772](https://github.com/vitessio/vitess/pull/12772) + * [vtadmin] log response errors [#12844](https://github.com/vitessio/vitess/pull/12844) +#### Online DDL + * schemadiff/OnlineDDL: ExtractConstraintOriginalName understands auto-generated names [#12275](https://github.com/vitessio/vitess/pull/12275) + * schemadiff: diffing two schemas generates a rich SchemaDiff object [#12551](https://github.com/vitessio/vitess/pull/12551) + * Online DDL: remove artifact entry upon GC [#12592](https://github.com/vitessio/vitess/pull/12592) + * Online DDL: configurable cut-over threshold [#12594](https://github.com/vitessio/vitess/pull/12594) + * gh-ost migrations: improved error log message [#12882](https://github.com/vitessio/vitess/pull/12882) + * OnlineDDL/vitess: only KILL 'RENAME' statement if not known to be successful [#12989](https://github.com/vitessio/vitess/pull/12989) + * Online DDL: better reporting of error message when RENAME fails [#13143](https://github.com/vitessio/vitess/pull/13143) +#### Query Serving + * VSCopy: Enable to copy from all shards in either a specified keyspace or all keyspaces [#11909](https://github.com/vitessio/vitess/pull/11909) + * vtgate: metrics for prepared stmts [#12141](https://github.com/vitessio/vitess/pull/12141) + * Add flag to select tx throttler tablet type [#12174](https://github.com/vitessio/vitess/pull/12174) + * sqlparser: Add parsing support to spatial POINT expression [#12198](https://github.com/vitessio/vitess/pull/12198) + * evalengine: new evaluation framework [#12247](https://github.com/vitessio/vitess/pull/12247) + * evalengine: it's time for JSON! [#12274](https://github.com/vitessio/vitess/pull/12274) + * Add parsing support for linestring constructor [#12299](https://github.com/vitessio/vitess/pull/12299) + * Add database name to _vt.views table (#12368) [#12374](https://github.com/vitessio/vitess/pull/12374) + * Schema RPC to fetch table/view definition [#12375](https://github.com/vitessio/vitess/pull/12375) + * sqlparser: add parsing support for polygon spatial datatype [#12377](https://github.com/vitessio/vitess/pull/12377) + * Emit per workload labels for existing per table vttablet metrics [#12394](https://github.com/vitessio/vitess/pull/12394) + * go/vt/vtgate: add json and use_fallback opts to numeric_static_map vdx [#12414](https://github.com/vitessio/vitess/pull/12414) + * Add basic metrics to `vttablet` transaction throttler [#12418](https://github.com/vitessio/vitess/pull/12418) + * Change `GetSchema` RPC to return `CreateView` instead of `SelectStmt` [#12421](https://github.com/vitessio/vitess/pull/12421) + * schemadiff: AlterTableAlgorithmStrategy [#12442](https://github.com/vitessio/vitess/pull/12442) + * GetSchema rpc to streaming api [#12447](https://github.com/vitessio/vitess/pull/12447) + * streamlog: make generic [#12494](https://github.com/vitessio/vitess/pull/12494) + * Horizon planning on operators [#12506](https://github.com/vitessio/vitess/pull/12506) + * Refactor and cleanup treatment of keyspace IDs and KeyRange [#12524](https://github.com/vitessio/vitess/pull/12524) + * collations: upgrade to MySQL 8.0.32 [#12557](https://github.com/vitessio/vitess/pull/12557) + * schemadiff: validate views' referenced columns via semantics [#12565](https://github.com/vitessio/vitess/pull/12565) + * sqlparser: Add parsing support to geom Collection types [#12608](https://github.com/vitessio/vitess/pull/12608) + * Parser: support PURGE BINARY LOGS statement [#12615](https://github.com/vitessio/vitess/pull/12615) + * sqlparser: Add parsing support for wkt funcs [#12651](https://github.com/vitessio/vitess/pull/12651) + * Add priority support to transaction throttler [#12662](https://github.com/vitessio/vitess/pull/12662) + * `schemadiff`: multi-error in schema normalization [#12675](https://github.com/vitessio/vitess/pull/12675) + * Add parsing support to wkb spatial funcs [#12739](https://github.com/vitessio/vitess/pull/12739) + * Log the length of the actual rows when the query exceeds the warning threshold [#12755](https://github.com/vitessio/vitess/pull/12755) + * [gen4 planner] Better handling of projections in operators [#12790](https://github.com/vitessio/vitess/pull/12790) + * Add parsing support to format functions [#12829](https://github.com/vitessio/vitess/pull/12829) + * Add parsing suppport to geom property functions [#12855](https://github.com/vitessio/vitess/pull/12855) + * planner fix: scoping rules for JOIN ON expression inside a subquery [#12881](https://github.com/vitessio/vitess/pull/12881) + * schemadiff: formalize SchemaDiff as the only schema diffing mechanism [#12885](https://github.com/vitessio/vitess/pull/12885) + * Trigger the loadTablets function when no primary is assigned to the shard [#12893](https://github.com/vitessio/vitess/pull/12893) + * Add vtgate and vttablet flag to truncate errors sent to client [#12899](https://github.com/vitessio/vitess/pull/12899) + * Add parsing support to point property functions [#12904](https://github.com/vitessio/vitess/pull/12904) + * schemadiff: rich `ImpossibleApplyDiffOrderError` [#12918](https://github.com/vitessio/vitess/pull/12918) + * Gen4: move insert planner to gen4 [#12934](https://github.com/vitessio/vitess/pull/12934) + * Optionally truncate queries that are written to logs [#12944](https://github.com/vitessio/vitess/pull/12944) + * Throttled transactions return MySQL error code 1041 ER_OUT_OF_RESOURCES [#12949](https://github.com/vitessio/vitess/pull/12949) + * Gen4 Operator Refactoring: ORDER BY & Debugging Enhancements [#12954](https://github.com/vitessio/vitess/pull/12954) + * `schemadiff`: better naming; formalizing `SchemaDiff` use. [#12955](https://github.com/vitessio/vitess/pull/12955) + * gen4 planner: Derived table handling on operators [#12978](https://github.com/vitessio/vitess/pull/12978) + * [Gen4] Aggregation and Grouping on Operators [#12994](https://github.com/vitessio/vitess/pull/12994) + * Add parsing support to linestring property functions [#12995](https://github.com/vitessio/vitess/pull/12995) + * planner: allow reference tables to specify global sources [#13030](https://github.com/vitessio/vitess/pull/13030) + * Add parsing support to := operator [#13032](https://github.com/vitessio/vitess/pull/13032) + * TxThrottler support for transactions outside BEGIN/COMMIT [#13040](https://github.com/vitessio/vitess/pull/13040) + * Add parsing support to geomCollection property functions and geohash/geojson functions [#13072](https://github.com/vitessio/vitess/pull/13072) + * Schema Tracking Refactor: Merge schema-tracking in health-streamer into schema.Engine [#13121](https://github.com/vitessio/vitess/pull/13121) + * enable settings pool by default [#13127](https://github.com/vitessio/vitess/pull/13127) + * Augmenting the `GetSchema` RPC to also work for `Table` and `All` type of input [#13197](https://github.com/vitessio/vitess/pull/13197) +#### TabletManager + * ApplySchema: deprecate '--skip_preflight' flag [#10716](https://github.com/vitessio/vitess/pull/10716) + * Update tabletpicker to support cell pref and tablet order options [#12282](https://github.com/vitessio/vitess/pull/12282) + * SidecarDB Init: don't fail on schema init errors [#12328](https://github.com/vitessio/vitess/pull/12328) + * Tablet throttler: adding log entries [#12966](https://github.com/vitessio/vitess/pull/12966) + * Tablet throttler: throttler-config-via-topo defaults 'true', deprecation message for old flags [#13130](https://github.com/vitessio/vitess/pull/13130) + * Tablet throttler: be explicit about client app name, exempt some apps from checks and heartbeat renewals [#13195](https://github.com/vitessio/vitess/pull/13195) +#### VReplication + * VReplication: Use MariaDB Compat JSON Functions (#12420) [#12434](https://github.com/vitessio/vitess/pull/12434) + * Add Workflow Update Client Command [#12622](https://github.com/vitessio/vitess/pull/12622) + * VDiff: Add --update-table-stats flag to VDiff2 [#12868](https://github.com/vitessio/vitess/pull/12868) + * VReplication: Support MySQL Binary Log Transaction Compression [#12950](https://github.com/vitessio/vitess/pull/12950) + * VReplication: More intelligently manage vschema table entries on unsharded targets [#13220](https://github.com/vitessio/vitess/pull/13220) +#### VTorc + * vtorc: have cooldown only apply to the same kind of analysis [#13103](https://github.com/vitessio/vitess/pull/13103) +#### web UI + * update query-string dep due to reported vulnerability [#12673](https://github.com/vitessio/vitess/pull/12673) + * [VTAdmin] Migrate to Vite [#12831](https://github.com/vitessio/vitess/pull/12831) +### Feature Request +#### Cluster management + * Initialize Tablet with super_read_only mode [#12206](https://github.com/vitessio/vitess/pull/12206) +#### Online DDL + * SchemaDiff: normalize boolean columns to tinyint(1), and, add TableQualifier hint to force adding a db qualifier to the alter diff. [#12133](https://github.com/vitessio/vitess/pull/12133) +#### Query Serving + * Support Custom SidecarDB Names on VTTablets [#12240](https://github.com/vitessio/vitess/pull/12240) + * Create Views allowed for same keyspace [#12409](https://github.com/vitessio/vitess/pull/12409) + * Add Prepare, Execute and Deallocate Statement Support related to prepared statement [#12752](https://github.com/vitessio/vitess/pull/12752) + * feat: keep track of the full table name of expanded columns [#12976](https://github.com/vitessio/vitess/pull/12976) + * gen4 planner: push aggregation under filtering [#13169](https://github.com/vitessio/vitess/pull/13169) +#### VReplication + * Workflow Show: display rows copied [#12231](https://github.com/vitessio/vitess/pull/12231) + * VReplication Workflows: support noblob binlog_row_image format for MoveTables and Reshard [#12905](https://github.com/vitessio/vitess/pull/12905) +#### VTAdmin + * [VTAdmin] Add GetSrvKeyspaces and GetSrvKeyspace [#12702](https://github.com/vitessio/vitess/pull/12702) + * Add panic recovery handler to vtadmin http middleware [#12864](https://github.com/vitessio/vitess/pull/12864) +### Internal Cleanup +#### Build/CI + * Stop launchable integration for unit tests [#12386](https://github.com/vitessio/vitess/pull/12386) + * Run launchable only on PRs against `main` [#12694](https://github.com/vitessio/vitess/pull/12694) +#### CLI + * Cleanup TODOs in vtorc flag parsing code from v15 [#12787](https://github.com/vitessio/vitess/pull/12787) + * [vtctld] Delete legacy dual purpose duration/int flag parsing [#12860](https://github.com/vitessio/vitess/pull/12860) +#### Cluster management + * delete deprecated vtctld healthcheck flags and associated realtime stats code [#12373](https://github.com/vitessio/vitess/pull/12373) + * [misc] Delete `automation*` protos [#12449](https://github.com/vitessio/vitess/pull/12449) + * Deprecate `durability_policy` flag in vtctld [#12930](https://github.com/vitessio/vitess/pull/12930) +#### Evalengine + * evalengine: use 128 bit hashing internally [#12452](https://github.com/vitessio/vitess/pull/12452) + * evalengine: Refactorings & fixes [#12554](https://github.com/vitessio/vitess/pull/12554) + * evalengine: More cleanup [#12573](https://github.com/vitessio/vitess/pull/12573) + * Move the JSON parser from evalengine [#12757](https://github.com/vitessio/vitess/pull/12757) + * datetime: Unify parse and print APIs [#12815](https://github.com/vitessio/vitess/pull/12815) + * evalengine/compiler: Unify compilation [#12913](https://github.com/vitessio/vitess/pull/12913) + * mysql/evalengine: Unify float parsing [#12979](https://github.com/vitessio/vitess/pull/12979) +#### Examples + * Remove `are-you-alive` example [#12432](https://github.com/vitessio/vitess/pull/12432) +#### General + * Fix CodeQL identified issues [#12199](https://github.com/vitessio/vitess/pull/12199) + * Fix typecasting issue for workflow types [#12217](https://github.com/vitessio/vitess/pull/12217) + * Fix additional integer type handling [#12237](https://github.com/vitessio/vitess/pull/12237) + * Move to independent sets package [#12251](https://github.com/vitessio/vitess/pull/12251) + * Copy release notes from release-16 to `main` [#12287](https://github.com/vitessio/vitess/pull/12287) + * Fix additional typecasting alerts [#12309](https://github.com/vitessio/vitess/pull/12309) + * Switch to using new Go 1.19 CRL parser [#12315](https://github.com/vitessio/vitess/pull/12315) + * Internal refactor: LastError as a public struct [#12321](https://github.com/vitessio/vitess/pull/12321) + * Move to Go 1.19 atomics [#12391](https://github.com/vitessio/vitess/pull/12391) + * Remove a bunch of debug logging [#12404](https://github.com/vitessio/vitess/pull/12404) + * Update the pull request template to include a checkpoint for verifying modified/created tests aren't flaky [#12443](https://github.com/vitessio/vitess/pull/12443) + * Add steps on how to merge during code-freeze [#12444](https://github.com/vitessio/vitess/pull/12444) + * Add a known issue into the release notes for xtrabackup and DDLs (#12536) [#12537](https://github.com/vitessio/vitess/pull/12537) + * Remove unused JQuery includes [#12552](https://github.com/vitessio/vitess/pull/12552) + * Minor Cleanup of SidecarDB Package [#12652](https://github.com/vitessio/vitess/pull/12652) + * [misc] Cleanup/fix/delete TODO comments [#12758](https://github.com/vitessio/vitess/pull/12758) + * sqlutils: Cleanup dead code [#12929](https://github.com/vitessio/vitess/pull/12929) + * topo: Remove unused templating code [#12939](https://github.com/vitessio/vitess/pull/12939) + * template: Move all HTML templates to safehtml [#12940](https://github.com/vitessio/vitess/pull/12940) + * style(grpc_codec): use switch-case [#12956](https://github.com/vitessio/vitess/pull/12956) + * fix some code comments throughout the codebase [#12964](https://github.com/vitessio/vitess/pull/12964) + * servenv: Move away from using default HTTP muxer [#12987](https://github.com/vitessio/vitess/pull/12987) + * tools: Cleanup more unused code [#12991](https://github.com/vitessio/vitess/pull/12991) + * coverage: Remove sonar coverage setup [#13000](https://github.com/vitessio/vitess/pull/13000) + * Update GOVERNANCE.md [#13047](https://github.com/vitessio/vitess/pull/13047) +#### Governance + * Added [Twitter] & [Etsy] to ADOPTERS.md [#12508](https://github.com/vitessio/vitess/pull/12508) +#### Observability + * flags: deprecate vtctld_addr and delete related code [#12580](https://github.com/vitessio/vitess/pull/12580) +#### Online DDL + * Deprecating VExec, part 2 - *post v16* [#12074](https://github.com/vitessio/vitess/pull/12074) + * Online DDL: remove legacy "stowaway table" logic [#12288](https://github.com/vitessio/vitess/pull/12288) + * [onlineddl] Remove some old backwards-compatibilities in online ddl code [#12427](https://github.com/vitessio/vitess/pull/12427) + * Query executor: preparing for `SHOW VITESS_MIGRATIONS` via `ShowBasic` [#12688](https://github.com/vitessio/vitess/pull/12688) +#### Query Serving + * [vtgate planner] Routing & Merging refactor [#12197](https://github.com/vitessio/vitess/pull/12197) + * Use the correct uint32 type for the GTID file position [#12229](https://github.com/vitessio/vitess/pull/12229) + * Fix typecasting alerts inside vtgate engine [#12238](https://github.com/vitessio/vitess/pull/12238) + * Change SQL error codes to use explicit type [#12249](https://github.com/vitessio/vitess/pull/12249) + * fix linter warnings [#12286](https://github.com/vitessio/vitess/pull/12286) + * [healthcheck] Remove deprecated fields from TabletHealth json marshalling [#12429](https://github.com/vitessio/vitess/pull/12429) + * Semantics refactor: formalize errors [#12572](https://github.com/vitessio/vitess/pull/12572) + * Refactor: go/vt/vtgate/engine/opcode to reduce `semantics` package dependencies [#12663](https://github.com/vitessio/vitess/pull/12663) + * evalengine: misc. cleanups [#12684](https://github.com/vitessio/vitess/pull/12684) + * [gen4 planner] Move more horizon planning to operators [#12750](https://github.com/vitessio/vitess/pull/12750) + * Move vreplication to vitess json parser [#12761](https://github.com/vitessio/vitess/pull/12761) + * simplify views ddl execution path [#12874](https://github.com/vitessio/vitess/pull/12874) + * refactor: clean up and simplify the semantics package [#12894](https://github.com/vitessio/vitess/pull/12894) + * Cleanup panics in `txthrottler`, reorder for readability [#12901](https://github.com/vitessio/vitess/pull/12901) + * txthrottler: further code cleanup [#12902](https://github.com/vitessio/vitess/pull/12902) + * feat: re-introduce expanded columns info in semantic state [#12908](https://github.com/vitessio/vitess/pull/12908) + * Add limit planning to operators [#12927](https://github.com/vitessio/vitess/pull/12927) + * [vttablet] cleanup unused code in tabletserver [#12943](https://github.com/vitessio/vitess/pull/12943) + * goyacc: don't import "unsafe" if not used [#13008](https://github.com/vitessio/vitess/pull/13008) + * fix misleading primary not serving message [#13022](https://github.com/vitessio/vitess/pull/13022) + * Remove enable-query-plan-field-caching flag that was deprecated in v15 [#13141](https://github.com/vitessio/vitess/pull/13141) + * sqlparser: Improve interval parsing [#13165](https://github.com/vitessio/vitess/pull/13165) + * refactor QueryProjection [#13174](https://github.com/vitessio/vitess/pull/13174) +#### TabletManager + * Table GC: remove spammy log entry [#12625](https://github.com/vitessio/vitess/pull/12625) + * mysqlctl: Remove unused schema version [#12999](https://github.com/vitessio/vitess/pull/12999) + * vttablet: Cleanup unused db version string [#13102](https://github.com/vitessio/vitess/pull/13102) + * mysqlctl: Remove custom callbacks [#13119](https://github.com/vitessio/vitess/pull/13119) + * mysqlctl: Remove usage of MYSQL_FLAVOR [#13135](https://github.com/vitessio/vitess/pull/13135) +#### VReplication + * Use consistent type for vreplication id [#12218](https://github.com/vitessio/vitess/pull/12218) + * Use consistent type port and replication state [#12248](https://github.com/vitessio/vitess/pull/12248) +#### VTAdmin + * [vtadmin-api] standardize cluster ids [#12803](https://github.com/vitessio/vitess/pull/12803) +#### VTorc + * Cleanup vtorc ssl package [#12423](https://github.com/vitessio/vitess/pull/12423) + * vtgr: Remove unused code [#12975](https://github.com/vitessio/vitess/pull/12975) +#### web UI + * Add nvm and node to vtadmin-up.sh [#12439](https://github.com/vitessio/vitess/pull/12439) +### Performance +#### General + * Use `MarshalVT`/`UnmarshalVT` instead of `proto.Marshal`/`proto.Unmarshal`. [#12525](https://github.com/vitessio/vitess/pull/12525) +#### Online DDL + * OnlineDDL: optimizing --singleton-context conflict check [#12539](https://github.com/vitessio/vitess/pull/12539) +### RFC +#### General + * [RFC] Standardized viper framework for vitess configuration parameters [#11456](https://github.com/vitessio/vitess/pull/11456) +### Regression +#### ACL + * vtgate : Disable Automatically setting immediateCallerID to user from static authentication context [#12961](https://github.com/vitessio/vitess/pull/12961) +#### Query Serving + * gen4 planner bugfix: issue when merging subqueries [#13025](https://github.com/vitessio/vitess/pull/13025) + * gen4 planner: allow last_insert_id with arguments [#13026](https://github.com/vitessio/vitess/pull/13026) +### Release +#### Documentation + * Update the release instructions after v16 code freeze [#12245](https://github.com/vitessio/vitess/pull/12245) + * Improve release instructions post `v16.0.0 GA` release [#12538](https://github.com/vitessio/vitess/pull/12538) + * Re-organize the `releasenotes` directory into `changelog` [#12566](https://github.com/vitessio/vitess/pull/12566) + * Update the release instructions for the local install guide [#12746](https://github.com/vitessio/vitess/pull/12746) + * Fix incorrect path during release notes generation [#12769](https://github.com/vitessio/vitess/pull/12769) + * Fix format error in the `v16.0.2` release notes [#13057](https://github.com/vitessio/vitess/pull/13057) + * consolidate all deprecated flags under one header in release notes [#13163](https://github.com/vitessio/vitess/pull/13163) +#### General + * Fix release script for the version in the docker script [#12284](https://github.com/vitessio/vitess/pull/12284) + * Improve release process post `v16.0.0 GA` code freeze [#12487](https://github.com/vitessio/vitess/pull/12487) + * Copy release notes from release-16.0.0 to main [#12516](https://github.com/vitessio/vitess/pull/12516) + * Copy release notes for v16.0.1, v15.0.3, and v14.0.5 [#12768](https://github.com/vitessio/vitess/pull/12768) + * Copy release notes of v16.0.2 [#13051](https://github.com/vitessio/vitess/pull/13051) + * Review and cleanup of v17 RC release notes summary [#13209](https://github.com/vitessio/vitess/pull/13209) + * Code freeze of release-17.0 [#13210](https://github.com/vitessio/vitess/pull/13210) +#### VTAdmin + * Add the vtadmin `web` directory to the release packages [#12639](https://github.com/vitessio/vitess/pull/12639) +### Testing +#### Build/CI + * testutils: use a rows parser for test comparisons [#12138](https://github.com/vitessio/vitess/pull/12138) + * Don't keep data in upgrade-downgrade tests (#12462) [#12465](https://github.com/vitessio/vitess/pull/12465) + * Flakes: Address Common Unit Test Races [#12546](https://github.com/vitessio/vitess/pull/12546) + * Throttler: Expose Tablet's Config & Leverage to Deflake Tests [#12737](https://github.com/vitessio/vitess/pull/12737) + * Remove TODO from e2e upgrade test [#12792](https://github.com/vitessio/vitess/pull/12792) + * fakesqldb: Guard query log usage with lock [#12813](https://github.com/vitessio/vitess/pull/12813) + * fakedbclient: Add locking to avoid races [#12814](https://github.com/vitessio/vitess/pull/12814) + * test: fix cfc flaky test [#12941](https://github.com/vitessio/vitess/pull/12941) + * [ci] add generator for templated flag testdata [#13150](https://github.com/vitessio/vitess/pull/13150) +#### Evalengine + * evalengine: Try to reduce test flakyness around time [#12819](https://github.com/vitessio/vitess/pull/12819) + * evalengine: Skip integration tests under race detector [#12948](https://github.com/vitessio/vitess/pull/12948) +#### Examples + * Examples: Add ENV Var to Local Examples to Skip VTAdmin [#12547](https://github.com/vitessio/vitess/pull/12547) +#### General + * Fix fullstatus test for backward compat [#12685](https://github.com/vitessio/vitess/pull/12685) + * VDiff: cleanup test log output [#12840](https://github.com/vitessio/vitess/pull/12840) +#### Online DDL + * OnlineDDL endtoend/CI: timestamp deviation toleration [#13046](https://github.com/vitessio/vitess/pull/13046) +#### Query Serving + * Use atomic.Bool for fakesqldb behavior flags [#12603](https://github.com/vitessio/vitess/pull/12603) + * go/{mysql,sync2,vt}: rework tabletserver consolidator test, modify consolidator result type [#12609](https://github.com/vitessio/vitess/pull/12609) + * debug: add ToString to operators to make it easier to debug [#12952](https://github.com/vitessio/vitess/pull/12952) + * fix: deflake TestGetSchemaRPC test [#12977](https://github.com/vitessio/vitess/pull/12977) + * Fix `TestStartFindMysqlPort` test flakiness [#13019](https://github.com/vitessio/vitess/pull/13019) + * Fix benchmarks in `plan_test.go` [#13096](https://github.com/vitessio/vitess/pull/13096) + * tests: stop tests not supported by older versions from running [#13183](https://github.com/vitessio/vitess/pull/13183) +#### VReplication + * Flakes: Use new healthy shard check in vreplication e2e tests [#12502](https://github.com/vitessio/vitess/pull/12502) + * Flakes: Handle Non-Determinstic Shard Ordering in VGTID [#12637](https://github.com/vitessio/vitess/pull/12637) + * Flakes: Handle Non-Deterministic VStream Event Ordering [#12642](https://github.com/vitessio/vitess/pull/12642) + * vreplication: Run all appropriate tests [#12862](https://github.com/vitessio/vitess/pull/12862) + * Generated Invisible Primary Keys: add vreplication test cases [#13004](https://github.com/vitessio/vitess/pull/13004) + * Flakes: enforce no concurrency/parallelism in vreplication unit test framework [#13104](https://github.com/vitessio/vitess/pull/13104) +#### VTAdmin + * [vtctldserver] Remove parallelization from tests that mutate shared state [#12257](https://github.com/vitessio/vitess/pull/12257) +#### VTorc + * Fix flakiness in `TestDeadPrimaryRecoversImmediately` [#13232](https://github.com/vitessio/vitess/pull/13232) + diff --git a/changelog/17.0/17.0.0/release_notes.md b/changelog/17.0/17.0.0/release_notes.md new file mode 100644 index 00000000000..f289ccfb0d4 --- /dev/null +++ b/changelog/17.0/17.0.0/release_notes.md @@ -0,0 +1,460 @@ +# Release of Vitess v17.0.0-rc1 +## Summary + +### Table of Contents + +- **[Major Changes](#major-changes)** + - **[Breaking Changes](#breaking-changes)** + - [Default Local Cell Preference for TabletPicker](#tablet-picker-cell-preference) + - [Dedicated stats for VTGate Prepare operations](#dedicated-vtgate-prepare-stats) + - [VTAdmin web migrated from create-react-app to vite](#migrated-vtadmin) + - [Keyspace name validation in TopoServer](#keyspace-name-validation) + - [Shard name validation in TopoServer](#shard-name-validation) + - [Compression CLI flags removed from vtctld and vtctldclient binaries](#remove-compression-flags-from-vtctld-binaries) + - [VtctldClient command RestoreFromBackup will now use the correct context](#VtctldClient-RestoreFromBackup) + - **[New command line flags and behavior](#new-flag)** + - [Builtin backup: read buffering flags](#builtin-backup-read-buffering-flags) + - [Manifest backup external decompressor command](#manifest-backup-external-decompressor-command) + - [Throttler config via topo enabled by default](#throttler-config-via-topo) + - **[New stats](#new-stats)** + - [Detailed backup and restore stats](#detailed-backup-and-restore-stats) + - [VTtablet Error count with code](#vttablet-error-count-with-code) + - [VReplication stream status for Prometheus](#vreplication-stream-status-for-prometheus) + - **[Online DDL](#online-ddl)** + - [--cut-over-threshold DDL strategy flag](#online-ddl-cut-over-threshold-flag) + - **[VReplication](#vreplication)** + - [Support for MySQL 8.0 `binlog_transaction_compression`](#binlog-compression) + - [Support for the `noblob` binlog row image mode](#noblob) + - **[VTTablet](#vttablet)** + - [VTTablet: Initializing all replicas with super_read_only](#vttablet-initialization) + - [Vttablet Schema Reload Timeout](#vttablet-schema-reload-timeout) + - [Settings pool enabled](#settings-pool) + - **[VTGate](#vtgate)** + - [StreamExecute GRPC API](#stream-execute) + - [Insert Planner Gen4](#insert-planner) + - **[Deprecations and Deletions](#deprecations-and-deletions)** + - [Deprecated Flags](#deprecated-flags) + - [Deprecated Stats](#deprecated-stats) + + +## Major Changes + +### Breaking Changes + +#### Default Local Cell Preference for TabletPicker + +We added options to the `TabletPicker` that allow for specifying a cell preference in addition to making the default behavior to give priority to the local cell *and any alias it belongs to*. We are also introducing a new way to select tablet type preference which should eventually replace the `in_order:` hint currently used as a prefix for tablet types. The signature for creating a new `TabletPicker` now looks like: + +```go +func NewTabletPicker( + ctx context.Context, + ts *topo.Server, + cells []string, + localCell, keyspace, shard, tabletTypesStr string, + options TabletPickerOptions, +) (*TabletPicker, error) {...} +``` + +Where ctx, localCell, option are all new parameters. + +`option` is of type `TabletPickerOptions` and includes two fields, `CellPreference` and `TabletOrder`. +CellPreference`: "PreferLocalWithAlias" (default) gives preference to vtgate's local cell, or "OnlySpecified" which only picks from the cells explicitly passed in by the client +`TabletOrder`: "Any" (default) for no ordering or random, or "InOrder" to use the order specified by the client + +See [PR 12282 Description](https://github.com/vitessio/vitess/pull/12282) for examples on how this changes cell picking behavior. + +#### Default TLS version changed for `vtgr` + +When using TLS with `vtgr`, we now default to TLS 1.2 if no other explicit version is configured. Configuration flags are provided to explicitly configure the minimum TLS version to be used. + +#### Dedicated stats for VTGate Prepare operations + +Prior to v17 Vitess incorrectly combined stats for VTGate Execute and Prepare operations under a single stats key (`Execute`). In v17 Execute and Prepare operations generate stats under independent stats keys. + +Here is a (condensed) example of stats output: + +```json +{ + "VtgateApi": { + "Histograms": { + "Execute.src.primary": { + "500000": 5 + }, + "Prepare.src.primary": { + "100000000": 0 + } + } + }, + "VtgateApiErrorCounts": { + "Execute.src.primary.INVALID_ARGUMENT": 3, + "Execute.src.primary.ALREADY_EXISTS": 1 + } +} +``` + +#### VTAdmin web migrated to vite + +Previously, VTAdmin web used the Create React App framework to test, build, and serve the application. In v17, Create React App has been removed, and [Vite](https://vitejs.dev/) is used in its place. Some of the main changes include: +- Vite uses `VITE_*` environment variables instead of `REACT_APP_*` environment variables +- Vite uses `import.meta.env` in place of `process.env` +- [Vitest](https://vitest.dev/) is used in place of Jest for testing +- Our protobufjs generator now produces an es6 module instead of commonjs to better work with Vite's defaults +- `public/index.html` has been moved to root directory in web/vtadmin + +#### Keyspace name validation in TopoServer + +Prior to v17, it was possible to create a keyspace with invalid characters, which would then be inaccessible to various cluster management operations. + +Keyspace names are restricted to using only ASCII characters, digits and `_` and `-`. TopoServer's `GetKeyspace` and `CreateKeyspace` methods return an error if given an invalid name. + +#### Shard name validation in TopoServer + +Prior to v17, it was possible to create a shard name with invalid characters, which would then be inaccessible to various cluster management operations. + +Shard names are restricted to using only ASCII characters, digits and `_` and `-`. TopoServer's `GetShard` and `CreateShard` methods return an error if given an invalid name. + +#### Compression CLI flags remove from vtctld and vtctldclient binaries + +The CLI flags below were mistakenly added to `vtctld` and `vtctldclient` in v15. In v17, they are no longer present in those binaries. + + * `--compression-engine-name` + * `--compression-level` + * `--external-compressor` + * `--external-compressor-extension` + * `--external-decompressor` + +#### VtctldClient command RestoreFromBackup will now use the correct context + +The VtctldClient command RestoreFromBackup initiates an asynchronous process on the specified tablet to restore data from either the latest backup or the closest one before the specified backup-timestamp. +Prior to v17, this asynchronous process could run indefinitely in the background since it was called using the background context. In v17 [PR#12830](https://github.com/vitessio/vitess/issues/12830), +this behavior was changed to use a context with a timeout of `action_timeout`. If you are using VtctldClient to initiate a restore, make sure you provide an appropriate value for action_timeout to give enough +time for the restore process to complete. Otherwise, the restore will throw an error if the context expires before it completes. + +### Vttablet's transaction throttler now also throttles DML outside of `BEGIN; ...; COMMIT;` blocks + +Prior to v17, `vttablet`'s transaction throttler (enabled with `--enable-tx-throttler`) would only throttle requests done inside an explicit transaction, i.e., a `BEGIN; ...; COMMIT;` block. +In v17 [PR#13040](https://github.com/vitessio/vitess/issues/13037), this behavior was being changed so that it also throttles work outside of explicit transactions for `INSERT/UPDATE/DELETE/LOAD` queries. + +### New command line flags and behavior + +#### Backup --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size + +Prior to v17 the builtin Backup Engine does not use read buffering for restores, and for backups uses a hardcoded write buffer size of 2097152 bytes. + +In v17 these defaults may be tuned with, respectively `--builtinbackup-file-read-buffer-size` and `--builtinbackup-file-write-buffer-size`. + +- `--builtinbackup-file-read-buffer-size`: read files using an IO buffer of this many bytes. Golang defaults are used when set to 0. +- `--builtinbackup-file-write-buffer-size`: write files using an IO buffer of this many bytes. Golang defaults are used when set to 0. (default 2097152) + +These flags are applicable to the following programs: + +- `vtbackup` +- `vtctld` +- `vttablet` +- `vttestserver` + +#### Manifest backup external decompressor command + +Add a new builtin/xtrabackup flag `--manifest-external-decompressor`. When set the value of that flag is stored in the manifest field `ExternalDecompressor`. This manifest field may be consulted when decompressing a backup that was compressed with an external command. + +This feature enables the following flow: + + 1. Take a backup using an external compressor + ``` + Backup --compression-engine=external \ + --external-compressor=zstd \ + --manifest-external-decompressor="zstd -d" + ``` + 2. Restore that backup with a mere `Restore` command, without having to specify `--external-decompressor`. + +#### vttablet --throttler-config-via-topo + +This flag was introduced in v16 and defaulted to `false`. In v17 it defaults to `true`, and there is no need to supply it. + +Note that this flag overrides `--enable-lag-throttler` and `--throttle-threshold`, which now give warnings, and will be removed in v18. + +### New stats + +#### Detailed backup and restore stats + +##### Backup metrics + +Metrics related to backup operations are available in both Vtbackup and VTTablet. + +**BackupBytes, BackupCount, BackupDurationNanoseconds** + +Depending on the Backup Engine and Backup Storage in-use, a backup may be a complex pipeline of operations, including but not limited to: + +* Reading files from disk. +* Compressing files. +* Uploading compress files to cloud object storage. + +These operations are counted and timed, and the number of bytes consumed or produced by each stage of the pipeline are counted as well. + +##### Restore metrics + +Metrics related to restore operations are available in both Vtbackup and VTTablet. + +**RestoreBytes, RestoreCount, RestoreDurationNanoseconds** + +Depending on the Backup Engine and Backup Storage in-use, a restore may be a complex pipeline of operations, including but not limited to: + +* Downloading compressed files from cloud object storage. +* Decompressing files. +* Writing decompressed files to disk. + +These operations are counted and timed, and the number of bytes consumed or produced by each stage of the pipeline are counted as well. + +##### Vtbackup metrics + +Vtbackup exports some metrics which are not available elsewhere. + +**DurationByPhaseSeconds** + +Vtbackup fetches the last backup, restores it to an empty mysql installation, replicates recent changes into that installation, and then takes a backup of that installation. + +_DurationByPhaseSeconds_ exports timings for these individual phases. + +##### Example + +**A snippet of vtbackup metrics after running it against the local example after creating the initial cluster** + +(Processed with `jq` for readability.) + +```json +{ + "BackupBytes": { + "BackupEngine.Builtin.Source:Read": 4777, + "BackupEngine.Builtin.Compressor:Write": 4616, + "BackupEngine.Builtin.Destination:Write": 162, + "BackupStorage.File.File:Write": 163 + }, + "BackupCount": { + "-.-.Backup": 1, + "BackupEngine.Builtin.Source:Open": 161, + "BackupEngine.Builtin.Source:Close": 322, + "BackupEngine.Builtin.Compressor:Close": 161, + "BackupEngine.Builtin.Destination:Open": 161, + "BackupEngine.Builtin.Destination:Close": 322 + }, + "BackupDurationNanoseconds": { + "-.-.Backup": 4188508542, + "BackupEngine.Builtin.Source:Open": 10649832, + "BackupEngine.Builtin.Source:Read": 55901067, + "BackupEngine.Builtin.Source:Close": 960826, + "BackupEngine.Builtin.Compressor:Write": 278358826, + "BackupEngine.Builtin.Compressor:Close": 79358372, + "BackupEngine.Builtin.Destination:Open": 16456627, + "BackupEngine.Builtin.Destination:Write": 11021043, + "BackupEngine.Builtin.Destination:Close": 17144630, + "BackupStorage.File.File:Write": 10743169 + }, + "DurationByPhaseSeconds": { + "InitMySQLd": 2, + "RestoreLastBackup": 6, + "CatchUpReplication": 1, + "TakeNewBackup": 4 + }, + "RestoreBytes": { + "BackupEngine.Builtin.Source:Read": 1095, + "BackupEngine.Builtin.Decompressor:Read": 950, + "BackupEngine.Builtin.Destination:Write": 209, + "BackupStorage.File.File:Read": 1113 + }, + "RestoreCount": { + "-.-.Restore": 1, + "BackupEngine.Builtin.Source:Open": 161, + "BackupEngine.Builtin.Source:Close": 322, + "BackupEngine.Builtin.Decompressor:Close": 161, + "BackupEngine.Builtin.Destination:Open": 161, + "BackupEngine.Builtin.Destination:Close": 322 + }, + "RestoreDurationNanoseconds": { + "-.-.Restore": 6204765541, + "BackupEngine.Builtin.Source:Open": 10542539, + "BackupEngine.Builtin.Source:Read": 104658370, + "BackupEngine.Builtin.Source:Close": 773038, + "BackupEngine.Builtin.Decompressor:Read": 165692120, + "BackupEngine.Builtin.Decompressor:Close": 51040, + "BackupEngine.Builtin.Destination:Open": 22715122, + "BackupEngine.Builtin.Destination:Write": 41679581, + "BackupEngine.Builtin.Destination:Close": 26954624, + "BackupStorage.File.File:Read": 102416075 + }, + "backup_duration_seconds": 4, + "restore_duration_seconds": 6 +} +``` + +Some notes to help understand these metrics: + +* `BackupBytes["BackupStorage.File.File:Write"]` measures how many bytes were read from disk by the `file` Backup Storage implementation during the backup phase. +* `DurationByPhaseSeconds["CatchUpReplication"]` measures how long it took to catch-up replication after the restore phase. +* `DurationByPhaseSeconds["RestoreLastBackup"]` measures to the duration of the restore phase. +* `RestoreDurationNanoseconds["-.-.Restore"]` also measures to the duration of the restore phase. + +#### VTTablet error count with error code + +##### VTTablet Error Count + +We are introducing new error counter `QueryErrorCountsWithCode` for VTTablet. It is similar to existing [QueryErrorCounts](https://github.com/vitessio/vitess/blob/main/go/vt/vttablet/tabletserver/query_engine.go#L174) except it contains errorCode as additional dimension. +We will deprecate `QueryErrorCounts` in v18. + +#### VReplication stream status for Prometheus + +VReplication publishes the `VReplicationStreamState` status which reports the state of VReplication streams. For example, here's what it looks like in the local cluster example after the MoveTables step: + +``` +"VReplicationStreamState": { + "commerce2customer.1": "Running" +} +``` + +Prior to v17, this data was not available via the Prometheus backend. In v17, workflow states are also published as a Prometheus gauge with a `state` label and a value of `1.0`. For example: + +``` +# HELP vttablet_v_replication_stream_state State of vreplication workflow +# TYPE vttablet_v_replication_stream_state gauge +vttablet_v_replication_stream_state{counts="1",state="Running",workflow="commerce2customer"} 1 +``` + +### VTTablet + +#### Initializing all replicas with super_read_only + +In order to prevent SUPER privileged users like `root` or `vt_dba` from producing errant GTIDs on replicas, all the replica MySQL servers are initialized with the MySQL +global variable `super_read_only` value set to `ON`. During failovers, we set `super_read_only` to `OFF` for the promoted primary tablet. This will allow the +primary to accept writes. All of the shard's tablets, except the current primary, will still have their global variable `super_read_only` set to `ON`. This will make sure that apart from +MySQL replication no other component, offline system or operator can write directly to a replica. + +Reference PR for this change is [PR #12206](https://github.com/vitessio/vitess/pull/12206) + +An important note regarding this change is how the default `init_db.sql` file has changed. +This is even more important if you are running Vitess on the vitess-operator. +You must ensure your `init_db.sql` is up-to-date with the new default for `v17.0.0`. +The default file can be found in `./config/init_db.sql`. + +#### Vttablet Schema Reload Timeout + +A new flag, `--schema-change-reload-timeout` has been added to timeout the reload of the schema that Vttablet does periodically. This is required because sometimes this operation can get stuck after MySQL restarts, etc. More details available in the issue https://github.com/vitessio/vitess/issues/13001. + +#### Settings Pool + +This was introduced in v15 and it enables pooling the connection with modified connection settings. +To know more what it does read the [v15 release notes](https://github.com/vitessio/vitess/releases/tag/v15.0.0) or the [blog](https://vitess.io/blog/2023-03-27-connection-pooling-in-vitess/) or [docs](https://vitess.io/docs/17.0/reference/query-serving/reserved-conn/) + +### Online DDL + +#### --cut-over-threshold DDL strategy flag + +Online DDL's strategy now accepts `--cut-over-threshold` (type: `duration`) flag. + +This flag stand for the timeout in a `vitess` migration's cut-over phase, which includes the final locking of tables before finalizing the migration. + +The value of the cut-over threshold should be high enough to support the async nature of vreplication catchup phase, as well as accommodate some replication lag. But it mustn't be too high. While cutting over, the migrated table is being locked, causing app connection and query pileup, consuming query buffers, and holding internal mutexes. + +Recommended range for this variable is `5s` - `30s`. Default: `10s`. + +### VReplication + +#### Support for the `noblob` binlog row image mode + +The `noblob` binlog row image is now supported by the MoveTables and Reshard VReplication workflows. If the source +or target database has this mode, other workflows like OnlineDDL, Materialize and CreateLookupVindex will error out. +The row events streamed by the VStream API, where blobs and text columns have not changed, will contain null values +for those columns, indicated by a `length:-1`. + +Reference PR for this change is [PR #12905](https://github.com/vitessio/vitess/pull/12905) + +#### Support for MySQL 8.0 binary log transaction compression + +MySQL 8.0 added support for [binary log compression via transaction (GTID) compression in 8.0.20](https://dev.mysql.com/blog-archive/mysql-8-0-20-replication-enhancements/). +You can read more about this feature here: https://dev.mysql.com/doc/refman/8.0/en/binary-log-transaction-compression.html + +This can — at the cost of increased CPU usage — dramatically reduce the amount of data sent over the wire for MySQL replication while also dramatically reducing the overall +storage space needed to retain binary logs (for replication, backup and recovery, CDC, etc). For larger installations this was a very desirable feature and while you could +technically use it with Vitess (the MySQL replica-sets making up each shard could use it fine) there was one very big limitation — [VReplication workflows](https://vitess.io/docs/reference/vreplication/vreplication/) +would not work. Given the criticality of VReplication workflows within Vitess, this meant that in practice this MySQL feature was not usable within Vitess clusters. + +We have addressed this issue in [PR #12950](https://github.com/vitessio/vitess/pull/12950) by adding support for processing the compressed transaction events in VReplication, +without any (known) limitations. + +### VTGate + +#### Modified StreamExecute GRPC API + +Earlier VTGate grpc api for `StreamExecute` did not return the session in the response. +Even though the underlying implementation supported transactions and other features that requires session persistence. +With [PR #13131](https://github.com/vitessio/vitess/pull/13131) VTGate will return the session to the client +so that it can be persisted with the client and sent back to VTGate on the next api call. + +This does not impact anyone using the mysql client library to connect to VTGate. +This could be a breaking change for grpc api users based on how they have implemented their grpc clients. + +#### Insert Planning with Gen4 + +Gen4 planner was made default in v14 for `SELECT` queries. In v15 `UPDATE` and `DELETE` queries were moved to Gen4 framework. +With this release `INSERT` queries are moved to Gen4. + +Clients can move to old v3 planner for inserts by using `V3Insert` planner version with `--planner-version` vtgate flag or with comment directive `/*vt+ planner=` for individual query. + +### Deprecations and Deletions + +- The deprecated `automation` and `automationservice` protobuf definitions and associated client and server packages have been removed. +- Auto-population of DDL revert actions and tables at execution-time has been removed. This is now handled entirely at enqueue-time. +- Backwards-compatibility for failed migrations without a `completed_timestamp` has been removed (see https://github.com/vitessio/vitess/issues/8499). +- The deprecated `Key`, `Name`, `Up`, and `TabletExternallyReparentedTimestamp` fields were removed from the JSON representation of `TabletHealth` structures. +- The `MYSQL_FLAVOR` environment variable is no longer used. +- The `--enable-query-plan-field-caching`/`--enable_query_plan_field_caching` vttablet flag was deprecated in v15 and has now been removed. + +#### Deprecated Command Line Flags + +- Flag `vtctld_addr` has been deprecated and will be deleted in a future release. This affects `vtgate`, `vttablet` and `vtcombo`. +- The flag `schema_change_check_interval` used to accept either a Go duration value (e.g. `1m` or `30s`) or a bare integer, which was treated as seconds. + This behavior was deprecated in v15.0.0 and has been removed. + `schema_change_check_interval` now **only** accepts Go duration values. This affects `vtctld`. +- The flag `durability_policy` is no longer used by vtctld. Instead it reads the durability policies for all keyspaces from the topology server. +- The flag `use_super_read_only` is deprecated and will be removed in a later release. This affects `vttablet`. +- The flag `queryserver-config-schema-change-signal-interval` is deprecated and will be removed in a later release. This affects `vttablet`. + Schema-tracking has been refactored in this release to not use polling anymore, therefore the signal interval isn't required anymore. + +In `vttablet` various flags that took float values as seconds have updated to take the standard duration syntax as well. +Float-style parsing is now deprecated and will be removed in a later release. +For example, instead of `--queryserver-config-query-pool-timeout 12.2`, use `--queryserver-config-query-pool-timeout 12s200ms`. +Affected flags and YAML config keys: + +- `degraded_threshold` +- `heartbeat_interval` +- `heartbeat_on_demand_duration` +- `health_check_interval` +- `queryserver-config-idle-timeout` +- `queryserver-config-pool-conn-max-lifetime` +- `queryserver-config-olap-transaction-timeout` +- `queryserver-config-query-timeout` +- `queryserver-config-query-pool-timeout` +- `queryserver-config-schema-reload-time` +- `queryserver-config-schema-change-signal-interval` +- `queryserver-config-stream-pool-timeout` +- `queryserver-config-stream-pool-idle-timeout` +- `queryserver-config-transaction-timeout` +- `queryserver-config-txpool-timeout` +- `queryserver-config-txpool-idle-timeout` +- `shutdown_grace_period` +- `unhealthy_threshold` + +#### Deprecated Stats + +These stats are deprecated in v17. + +| Deprecated stat | Supported alternatives | +|-|-| +| `backup_duration_seconds` | `BackupDurationNanoseconds` | +| `restore_duration_seconds` | `RestoreDurationNanoseconds` | + +------------ +The entire changelog for this release can be found [here](https://github.com/vitessio/vitess/blob/main/changelog/17.0/17.0.0/changelog.md). + +The release includes 432 commits (excluding merges) + +Thanks to all our contributors: @Ayman161803, @GuptaManan100, @L3o-pold, @Phanatic, @WilliamLu99, @adsr, @ajm188, @andylim-duo, @arthurschreiber, @austenLacy, @cuishuang, @dasl-, @dbussink, @deepthi, @dependabot[bot], @ejortegau, @fatih, @frouioui, @github-actions[bot], @harshit-gangal, @hkdsun, @jeremycole, @jhump, @johanstenberg92, @jwangace, @kevinpurwito, @kovyrin, @lixin963, @mattlord, @maxbrunet, @maxenglander, @mdlayher, @moberghammer, @notfelineit, @olyazavr, @pbibra, @pnacht, @rohit-nayak-ps, @rsajwani, @shlomi-noach, @systay, @timvaillancourt, @twthorn, @vbalys, @vinimdocarmo, @vitess-bot[bot], @vmg, @yoheimuta + diff --git a/changelog/17.0/17.0.0/summary.md b/changelog/17.0/17.0.0/summary.md index 306a787fa71..3de3844593c 100644 --- a/changelog/17.0/17.0.0/summary.md +++ b/changelog/17.0/17.0.0/summary.md @@ -14,6 +14,7 @@ - **[New command line flags and behavior](#new-flag)** - [Builtin backup: read buffering flags](#builtin-backup-read-buffering-flags) - [Manifest backup external decompressor command](#manifest-backup-external-decompressor-command) + - [Throttler config via topo enabled by default](#throttler-config-via-topo) - **[New stats](#new-stats)** - [Detailed backup and restore stats](#detailed-backup-and-restore-stats) - [VTtablet Error count with code](#vttablet-error-count-with-code) @@ -29,6 +30,7 @@ - [Settings pool enabled](#settings-pool) - **[VTGate](#vtgate)** - [StreamExecute GRPC API](#stream-execute) + - [Insert Planner Gen4](#insert-planner) - **[Deprecations and Deletions](#deprecations-and-deletions)** - [Deprecated Flags](#deprecated-flags) - [Deprecated Stats](#deprecated-stats) @@ -164,6 +166,12 @@ This feature enables the following flow: ``` 2. Restore that backup with a mere `Restore` command, without having to specify `--external-decompressor`. +#### vttablet --throttler-config-via-topo + +This flag was introduced in v16 and defaulted to `false`. In v17 it defaults to `true`, and there is no need to supply it. + +Note that this flag overrides `--enable-lag-throttler` and `--throttle-threshold`, which now give warnings, and will be removed in v18. + ### New stats #### Detailed backup and restore stats @@ -382,6 +390,13 @@ so that it can be persisted with the client and sent back to VTGate on the next This does not impact anyone using the mysql client library to connect to VTGate. This could be a breaking change for grpc api users based on how they have implemented their grpc clients. +#### Insert Planning with Gen4 + +Gen4 planner was made default in v14 for `SELECT` queries. In v15 `UPDATE` and `DELETE` queries were moved to Gen4 framework. +With this release `INSERT` queries are moved to Gen4. + +Clients can move to old v3 planner for inserts by using `V3Insert` planner version with `--planner-version` vtgate flag or with comment directive /*vt+ planner=` for individual query. + ### Deprecations and Deletions - The deprecated `automation` and `automationservice` protobuf definitions and associated client and server packages have been removed. From 60760325b668f0515d46f9f2497d6a6c0ab2e7e2 Mon Sep 17 00:00:00 2001 From: Florent Poinsard Date: Tue, 6 Jun 2023 16:30:48 +0200 Subject: [PATCH 2/2] Update changelog dir Signed-off-by: Florent Poinsard --- changelog/14.0/README.md | 4 ++++ changelog/15.0/README.md | 4 ++++ changelog/17.0/README.md | 2 ++ 3 files changed, 10 insertions(+) diff --git a/changelog/14.0/README.md b/changelog/14.0/README.md index 10f12867483..1b251dd0b01 100644 --- a/changelog/14.0/README.md +++ b/changelog/14.0/README.md @@ -1,4 +1,8 @@ ## v14.0 +* **[14.0.5](14.0.5)** + * [Changelog](14.0.5/changelog.md) + * [Release Notes](14.0.5/release_notes.md) + * **[14.0.4](14.0.4)** * [Changelog](14.0.4/changelog.md) * [Release Notes](14.0.4/release_notes.md) diff --git a/changelog/15.0/README.md b/changelog/15.0/README.md index f6985e8252b..332f1a7eb24 100644 --- a/changelog/15.0/README.md +++ b/changelog/15.0/README.md @@ -1,5 +1,9 @@ ## v15.0 The dedicated team for this release can be found [here](team.md). +* **[15.0.3](15.0.3)** + * [Changelog](15.0.3/changelog.md) + * [Release Notes](15.0.3/release_notes.md) + * **[15.0.2](15.0.2)** * [Changelog](15.0.2/changelog.md) * [Release Notes](15.0.2/release_notes.md) diff --git a/changelog/17.0/README.md b/changelog/17.0/README.md index 66ade1ffaf4..7e3a43acd03 100644 --- a/changelog/17.0/README.md +++ b/changelog/17.0/README.md @@ -1,2 +1,4 @@ ## v17.0 * **[17.0.0](17.0.0)** + * [Changelog](17.0.0/changelog.md) + * [Release Notes](17.0.0/release_notes.md)