Releases: vitessio/vitess
Vitess 6.0.20-20200525 Patch Update
This issue addresses a regression in vtgate performance, which was significant enough that it was decided to backport to the release branch. In addition to this bug fix, there remains a known issue where XtraBackup is not compatible with MySQL 8.0.20. Percona is aware of the issue, and there are no known fixes at this point in time.
Bugs Fixed
Vitess 6.0.20-20200519 Patch Update
This update fixes a number of issues related to backup discovered in Vitess 6, and is a recommended update for all users. In addition to these bug fixes, there remains a known issue where XtraBackup is not compatible with MySQL 8.0.20. Percona is aware of the issue, and there are no known fixes at this point in time.
Bugs Fixed
- VTTablet: An issue exists where transaction pool timeout did not apply, and connections never expired #6197
- VTTablet: An issue was discovered where copying a backup to S3 failed with an error of
closed network connection: cannot copy data
. This has been worked around by retrying of S3 upload requests. #6199 - VTTablet: The included version of pgzip has been upgraded to v1.2.4. This helps address an occasional panic that was observed with xtrabackup backups. #6198
Vitess 6.0.20-20200508 Patch Update
Known Issues
- While running a builtin backup with large databases, you might sometimes get this error:
use of closed network connection: cannot copy data: MultipartUpload: upload multipart failed.
We believe this is caused by the newer aws-sdk-go version we upgraded to in 5.0. It might be this problem filed upstream. - On xtrabackup backed up tablets on the new build, you might occasionally hit this panic:
Rendezvous of RPC that terminated with (StatusCode.UNKNOWN, caught panic during Backup: bufio: writer returned negative count from Write
This also seems to be related to aws-sdk-go, though we haven't traced it to a known issue. - There is a known regression where the transaction pool timeout will not apply, and connections never expire. We are working on a fix for this issue, and will release an update shortly.
Release Notes
Two regressions were discovered in Vitess 6 after release. While neither are critical, they are visible enough that we wanted to release a patch update:
-
A regression in the packaging of
tar.gz
files moved the examples fromexamples/
toshare/vitess/examples
. This made the install from binary instructions incorrect. #6159 #6160 -
A regression in how
select database()
returned the current database in use broke how the MySQL command-line client changes the currently selected database. This issue is specific to themysql
client, and is not known to affect application usage ofcom_init_db
oruse <dbname>
#6157 #6158
Vitess 6 General Availability
Known Issues
- While running a builtin backup with large databases, you might sometimes get this error:
use of closed network connection: cannot copy data: MultipartUpload: upload multipart failed.
We believe this is caused by the newer aws-sdk-go version we upgraded to in 5.0. It might be this problem filed upstream. - On xtrabackup backed up tablets on the new build, you might occasionally hit this panic:
Rendezvous of RPC that terminated with (StatusCode.UNKNOWN, caught panic during Backup: bufio: writer returned negative count from Write
. This also seems to be related to aws-sdk-go, though we haven't traced it to a known issue. - There is a known regression where the transaction pool timeout will not apply, and connections never expire. We are working on a fix for this issue, and will release an update shortly.
Incompatible Changes
This release includes the following changes, which may result in incompatibilities when upgrading from a previous release:
Changes to Local Examples
The local examples have been updated to use the new VReplication workflows for MoveTables and Resharding. This obsoletes the previous workflows of Vertical Split Clone and Horizontal Sharding. The earlier examples remain available in the “local.legacy” example directory for now. #6016 #6015 #6052 #6072 #6106 #6121
Change to Helm Chart
The helm chart has been updated to default to using Kubernetes as the Topology Server provider. This change was required to remove the dependency on etcd-operator, which is no longer under active support. #5989
Earlier Kubernetes Operator Retired
Support for the Vitess Kubernetes Operator has been removed due to a lack of maintainers. Adoption is understood to be low, but users are encouraged to migrate to using the Helm charts which continue to be supported, or PlanetScale’s Vitess Operator (provided by a third party) #5873
SQL Support: Keywords are no longer lowercase
Vitess no longer lower cases SQL keywords. In corner cases, this may affect applications which made specific use of SQL keywords, and then referred to a column in the result set #5897
SQL Support: Parenthesis are now removed from the AST
Vitess now simplifies expressions to remove parentheses from expressions. In corner cases, this may affect applications that use an expression that contains parens and then refer to it as a column in the result. #5906
Statement Based Replication, SplitQuery and UpdateStream
Code for the unsupported features of Statement-based replication (SBR), SplitQuery and UpdateStream has now been removed. Users using SBR are encouraged to switch to using Row-based replication instead. #5940 #5972 #5928
Messaging: First Class Citizen + API Improvements
The messaging feature has had several breaking changes and feature additions
- Direct inserts are now supported:
INSERT INTO msg_table SELECT col1, col2...
time_scheduled
andtime_created
are no longer required nor automatically populated.priority
is a new required field. Messages with a lower priority will be processed first.- Message retention is now measured after
time_acked
instead oftime_scheduled
- New table options
vt_min_backoff
andvt_max_backoff
set bounds on exponential backoff for message retries - 33% backoff jitter has been added for messages that are getting postponed to prevent thundering herds.
Users of the messages feature are encouraged to read through the full details in issue #5947 #5948 #6114
RPM and DEB Packages
Vitess binaries and examples are now installed in system standard paths, rather than the /vt directory #5836
Vitess Go Driver
Values of type sqltypes.Timestamp
are now automatically converted to time.Time
values
#5788
Simplified VTtablet Schema Load Process
The process of loading vttablet schema has been simplified, as many of the statistics loaded were expensive to gather but remained unused. This may lead to compatibility breakage in the use of the features: Sequences, VStreamer and Messages #5951
Planned Reparent
The implementation of Planned Reparent and Emergency Reparent have been improved, with more graceful handling of failure modes. While this change is functionality neutral, it is important that Vitess components are upgraded in the recommended order. #6050
Bugs Fixed
-
Binlog Client: Vitess components now randomize the server_id when acting as a MySQL replica. #5998
-
heartbeat: Prevent a scenario where tables could be created on a replica leading to errant transactions. Now tables are only created on the master with binlog ON, using the app_user not dba_user #6053
-
Messages: For the mysql protocol, buffered results coming from vitess are flushed a minimum of every 100ms, and configurable by mysql_server_flush_delay. This helps prevent against a case where intermittent streaming could delay messages from being sent. #5879
-
mysqlctl: Correctly escape and quote passwords that contain special characters. #5761
-
VReplication: Several improvements to failure handling and miscellaneous fixes #5753 #5792 #5801 #5842 #5935 #5905 #5845 #5942
-
Vtcompose: Add a check to make sure that files can be read #6024
-
Vtcompose: refactor vtcompose for improved testability #5994
-
Vindexes: Support for a region_json Vindex #6017 #5806 #5955
-
Vtctl: Before applying a VSchema, vtctl now correctly checks that the schema exists. #5931
-
Vtctl: Vtctl now exits early if the topo_global_root was set incorrectly #6119
-
Vtctl: EnsureVSchema and GetOrCreateShard now pass in the cell name. This helps prevent against corruption, which could occur from auto-rebuilding of cells automatically when creating a new shard or keyspace. #5930
-
VTExplain: An issue where incorrectly formatted comments could lead to a hang has been fixed. #5984
-
VTGate: Leading zeros were incorrectly dropped in microsecond timestamps when using prepared statements #5901
-
VTGate: Setting the transaction_mode in a mysql session is now case insensitive. #5963
-
VTGate: When updating consistent indexes, VTGate applies an optimization where it checks if anything really changed before applying an update. In the case that this check results in errors, VTGate will now skip the optimization and execute the update regardless. #5883
-
VTGate: Remove allocation of unused bytes #5874
-
VTGate: The gateway interface functionality has been simplified in anticipation of two new gateway implementations: TabletGateway and VTDirectGateway #5978 #5992
-
VTGate: The process of watching for VSchema updates from the topology server now recovers successfully if it receives an invalid update. This helps prevent against a scenario where a malformed vschema could take down all vtgate servers. #6051
-
VTGate: Previously VTGate performed health checks against all keyspaces, not just keyspaces listed in keyspaces_to_watch. Healthchecks are now limited correctly. #5815
-
VTGate: SQL Support: database() is now only rewritten against dual #5793
-
VTGate: Plan Cache is now disabled for statements with nil instructions #6045
-
VTGate: Make sure to only accept DDL that can be parsed, at least partially #6023
-
VTGate: Log when accepting a new connection fails #5881
-
VTGate: Prepared Statements did not correctly handle signed integers #5974
-
VTGate: Executor statistics did not correctly account for dividing by zero, leading to a potential panic. #5904
-
VTGate: The error message for unsupported DDL creation has been improved #6109
-
VTGate: The metric QueriesProcessedByTable was not correctly incremented with streaming queries processed by vtgate #6057
-
GRPC Client: The default keep alive and keep alive timeout values have been changed from unlimited to 10 seconds. This helps prevent cases where queries could hang while waiting for a failed pod or node. In addition, the defaultMaxMessageSize is now set to 16MB #5922 #3890
-
VTTablet: local_metadata is now populated after an agent has been initialized. This helps prevent against the ambiguous case where both -init_populate_metadata and -restore_from_backup are specified. #5783
-
VTTablet: A hang could occur when shutting down a tablet server when the Topo Server is unavailable. Tablet servers will now time out correctly. #5865
-
VTTablet Health Check: A warning is now logged when the vttablet StreamHealth RPC fails #5917
-
VTTablet Planned Reparent: A new context is now created for calling UndoDemoteMaster. This helps prevent a scenario where the existing context may have expired. #5841
-
VTTablet: A panic could occur when calling
NewDBConnNoPool
#6086 -
VTTablet: Queries of type
STREAM_EXECUTE
were inadvertently dropped from logging #6056 -
Zookeeper Topo: Trailing newlines are now ignored when specifying
-topo_zk_auth_file
#6030 -
Misc: Stats counters would erroneously print warnings to log files when adding a negative value. This warning did not consider the case where counters are reused. #5988
-
Misc: Binary log events with the special name of "mysql.rds_*" are now skipped. This helps improve compatibility with Amazon RDS flavors #5966
Functionality Added or Changed
-
Docker: sysstat and strace are now bundled in vitess/lite images #5814
-
Docker: new lite images based on Red Hat UBI 7 are now available #6100
-
Example Code: An are you alive example has been added #5646
-
Example Code: The demo application has been migrated from Python to Go #6022
-
Example Code: A set of notes and scripts on how to use Vitess with common frameworks #6034
-
Helm charts: now work with the latest Kubernetes version. An issue where the 308_...
Vitess 5 General Availability
Incompatible Changes
This release includes the following changes, which may result in incompatibilities when upgrading from a previous release:
Deprecated Support for Vagrant
Support for deployment on Vagrant is deprecated, and will be removed in the next release of Vitess. The recommended upgrade path is local deployment on macOS.
Deprecation of vtgate option disable_local_gateway
The option disable_local_gateway is parsed but ignored by vtgate. It is now deprecated, and will be removed in a future version #5741
Web_dir2 and web_dir options for vtctld
The options web_dir2 and web_dir are now parsed but ignored. This is a behavior change from 5.0 beta, where the options were removed. We intend to remove these options again in Vitess 6.0, but discovered compatibility issues with existing tooling. #5744 #5747
Bugs Fixed
- Consul: The bundled consul version has been bumped to 1.5.1, addressing CVE-2019-12291 and CVE-2018-19653.
- VReplication: Fixed a panic when connecting to an external MySQL server #5753
- Vttablet: Address #5734 by forming MySQL error packet on query parse error during
- Vtgate: Fixed an issue with prepared statements returning a syntax error #5735
- Mysqlctl: Fixed a regression where the option mysqlctl_mycnf_template was accidentally removed during refactoring. #5717
- Mysqlctl: Fix a regression where refactoring configuration files removed the options relay-log and relay-log-index from included configuration #5745
- Vttablet: Fixed a problem with vttablet consuming too much memory during a restore from backup when running with option -s3_backup_log_level LogDebugWithRequestRetries #5777
Functionality Added or Changed
- Vtctld: The reference to web/vtctld has been removed from the Makefile #5778
- mysqlctld: An option has been added to mysqlctl to allow passdown of LD_PRELOAD libraries. This is useful in the case where a custom memory allocation library needs to be specified. #5730
- Logging: Add datadog plugin for tracing #5681
- Add support for DISTINCTROW as an alias for DISTINCT SQL syntax #5726 #5718
- Vstream: support default values #5736
- VReplication: Add support for ExternalizeVindex #5716
- ApplySchema: Add support for ALTER DATABASE #5733
Build Environment Changes
Vitess 5.0.1 Beta
Incompatible Changes
This release fixes an issue discovered after releasing Vitess 5.0.0 Beta (#5711). We recommend reading the release notes for 5.0.0 when upgrading from an earlier release.
Bugs Fixed
- mysqlctl: Vitess programs would not run if configuration files could not be located #5711
- VReplication: The keyrange was incorrectly being quoted twice #5715
- Release packaging: Fix issues discovered in automated release scripts #5714
- Docker images: provide a workaround for missing configuration files in vitess/lite image #5707
- Change http links to https #5697
- helm: Fix examples to remove undefined flag #5696
Functionality Added or Changed
- VReplication: CreateLookupVindex command with backfill #5692
Vitess 5.0.0 Beta
Known Issues
Important! This release contains a bug, where MySQL configuration files were intended to be bundled in the mysqlctl binary, are not correctly bundled. The workaround is to ensure that $VTROOT/config
is available on your system. We intend to fix this issue before final release of Vitess 5.0.
Incompatible Changes
This release includes the following changes, which may result in incompatibilities when upgrading from a previous release:
Removal of binaries from release packages
The release packages have been improved to no longer ship unneeded dependencies. If you previously installed etcd, zookeeper or consul as part of the Vitess package, you will now need to download and install it separately. #5545 #5645
Refactoring MySQL Configuration Files
The included my.cnf files for MySQL have been simplified and improved. There may be some isolated cases where the difference in configuration may lead to compatibility issues with a previous installation of MySQL. #5326 #5517 #5704
VTROOT, VT_MYSQL_ROOT, VTDATAROOT changes
You are no longer required to specify the environment variables VTROOT, VT_MYSQL_ROOT, VTDATAROOT in order to run the Vitess examples. These variables have been changed to autodetect. Use of these variables is still permitted, but discouraged, as they may be removed in a future release. #5488 #5506 #5654 #5560
Elimination of VTTOP environment variable
The environment variable VTTOP has been eliminated, with all uses changed to VTROOT instead. In most cases VTROOT can also be autodetected, so it is encouraged to remove VTTOP from any scripts. #5527 #5632 #5546 #5548
Removal of vtqueryserver
The unused vtqueryserver utility has been removed. #5583
web_dir2 and web_dir options removed from vtctld
The web files used by vtctld are now embedded into the vtctld binary, simplifying configuration. This results in a behavior change since web_dir and web_dir2 are no longer valid parameters when starting vtctld. Please be sure to update your startup scripts accordingly. #5597
Examples now check for dependencies
The examples now check for shell programs that are required to execute correctly. If you do not have the following programs in your PATH, it will result in an error: mysqld etcd etcdctl curl vtctlclient vttablet vtgate vtctld mysqlctl #5543
Changes to build from source
The build from source instructions have changed. See https://vitess.io/docs/contributing/build-from-source/ for details.
Bugs Fixed
- The stability of prepared statements has been improved #5562 #5441
- The portability of shell scripts has been improved. #5384 #5480
- The semi-sync extension is now optional #5483
- A panic could occur when a sequence name was bad. #5621
- Loading TLS and x509 certificates now consumes less memory. #5423
- Helm no longer complains about nil values in templates. #5484
- Add workaround for common MySQL replication error #5627
- Add null as a valid constant SELECT value for vtexplain #5624
- Creating new shards could overwhelm a topo server. Vitess now no longer performs actions concurrently when creating shards in the topo. #5436
- VTExplain could race when waiting around for the fakesqldb tabletserver #5476
- VTGate now correctly resolves columns in information_schema tables. #5608
- The query SHOW FULL COLUMNS now works correctly with a filter #5558
- Backups now pass the correct filesize to Ceph #5369
- Vtbackup will now stop replication after reaching the goal position. #5437
- The bundled version of log4j has been upgraded #5691
- VTTablet now accepts external tablets in Azure MySQL #5683
- VTGate now advertises protocol support for FOUND ROWS #5676
- A buffered reader is now placed in front of xtrabackup’s STDOUT pipe, potentially reducing memory usage #5666
- Consultopo: Enforce a connection heartbeat for Watch() #5673
- The tabletmanager now ensures that it is fully shutdown before closing the topology client #5667
Functionality Added or Changed
- Several improvements to VReplication #5432 #5628 #5367 #5594 #5575 #5574 #5509 #5500 #5498 #5494 #5491 #5457 #5456 #5454 #5429 #5490 #5515 #5689 #5688 #5669
Including:- Support for file+pos based replication, allowing replication in from MySQL instances which do not have GTIDs enabled
- New Materialize command
- Support for Vdiff, Reshard
- Vindex and multi-column support
- Several Improvements to Docker files. #5592 #5510 #5470 #5458 #5444 #5492 #5661
Including:- A refactoring of vitess/lite files
- Support for aarch64
- The default mysql data files are now removed from the vitess/lite images
- The latest xtrabackup is always retrieved
- Jemalloc and tcmalloc are now bundled.
- Mysqlctl now checks that it is not being executed as root. This helps debugging scenarios, since mysqld will refuse to execute as root by default. #5419 #5644
- The examples have been improved. Improvements include a check that all dependencies are installed, and warn if it looks like selinux or apparmor prevented mysqlctl from starting #5543 #5573 #5461 #5433 #5426 #5386
- The bootstrap process will search for etcd in the PATH, and won’t install it if it is present. In addition, etcd on aarch64 is now supported. #5427 #5481 #5428 #5495 #5434 #5425
- VTGate now supports proxy protocol #5563
- There is now an option for TLS enabled etcd server connections. #5496
- Vtgate now advertises itself as a MySQL 5.7 server #5482
- A vt_monitoring user has been added to init_db.sql. #5486
- Mysqlctl now supports bootstrapping MariaDB 10.4. Please note that MariaDB 10.4 is not yet supported. #5374
- VTGate now provides a scatter of query statistics #5422
- Vttablet now provides better granularity for errors that come out of the transaction pool #5373
- Experimental support for region-aware vindexes, and xxhash vindexes #5390 #5439
- VTTablet servers now support an option to disable caching field names to avoid schema mismatches #5572
- VTGate: Query routing has been improved when using LEFT JOIN queries #5551
- VTGate: The parser now supports additional types of BEGIN statements #5550
- VTTablet servers no longer rechecks the mysql port if it was explicitly provided. This helps ensure operation in proxied environments. #5383
- The 64 bit hash implementation has been changed from 3DES to DES #5529
- Demote a master vttablet to a SPARE instead of REPLICA is now supported #5520
- Vttestserver now supports vschema operations via vtgate #5582
- The vtgate resolver has been simplified #5679
Build Environment Changes
- Several CI tests have been migrated to GitHub Actions, and unit tests are now run against all supported MySQL flavors and versions. #5651 #5635 #5559 #5561 #5414 #5557 #5487
- Several endtoend tests have been converted from Python 2.7 to Go, and now run via GitHub Actions. #5650 #5643 #5630 #5619 #5607 #5604 #5596 #5595 #5587 #5566 #5513 #5479 #5694 #5664
- Improvements to reduce the Flakiness of tests #5642 #5639 #5637 #5625 #5622 #5620 #5615 #5612 #5567 #5648 #5633 #5658 #5657
- The unit race test now tests all go source files #5629 #5499
- Releases are now automated via GitHub Actions #5593 #5589
- The unused files in tools/sauce_connect_*.sh have been removed #5602
- The testsuite now defaults to using Row-based Replication #5588
- Improvements to Git hooks #5544 #5453
- Out of date references to keytar and broken links have been fixed. #5536
- The build system will no longer warn when go is installed via a system package. #5407
- The build system now explicitly sets
GOBIN
toVTROOT/bin
#5468 - Minor code improvements #5501 #5552 #5435 #5526 #5601 #5541 #5472 #5404 #5678 #5663
- Update vagrant bootstrap to install pip #5440
Vitess 4.0.1
An issue was discovered in the previous packages, where packages reported the wrong version.
Bugs Fixed
- Add stronger protection against running examples as root #5431
Vitess 4.0
Important Changes
- Vitess 4.0 no longer supports Statement-Based Replication. Users upgrading will need to switch to Row-Based Replication.
- V2 Routing is now deprecated and will be removed in a future release. This was an undocumented API that was never officially supported.
- The default topology server has switched to etcd. ZooKeeper and Consul remain fully supported.
- Support for InfluxDB as a monitoring backend has been removed. Prometheus is recommended in place.
- VReplication support is experimental. We are satisfied with the stability and test coverage of this feature but may make changes to the command-line interface based on feedback received. These changes will not follow the Vitess deprecation policy.
- Prepared statements support is experimental. Support for prepared statements was merged later in the development cycle, and anticipate that there may still be undiscovered edge-cases.
- The internal representation of topology has been altered between Vitess 3.0 and 4.0. For the most part, it is backward and forwards compatible. The only caveat is to not upgrade to 4.0 while performing horizontal/vertical resharding.
- The internal Vitess tables VReplication, local_metadata and shard_metadata tables have been modified to support multiple databases in the same mysql instance. This schema change is expected to be forwards and backwards compatible with Vitess 3.0.
- Support for VStream is experimental. We invite feedback from users before stabilizing this feature in an upcoming release.
- The included MySQL configuration files bundled with Vitess have been improved. This may lead to cases where a MySQL server started with Vitess 3.0 will not start in Vitess 4.0. Please see the error log for the MySQL server for details.
- The configuration flags timeout, replication_timeout, gcs_backup_storage_project and lock_timeout have been deprecated and will be removed in the next release of Vitess.
- The reparenting process has been significantly improved in Vitess 4.0. It is important that you follow the recommended upgrade order for components so that reparents work properly while components are on mixed versions.
Bugs Fixed
- Handle case where mysqld server replies to initial handshake with ERR packet #5149
- Redact password from error log #5197
- Fixed typos and/or errors in source code comments #5223 #5221 #5199 #5227 #5228 #5230 #5176 #5158 #5135 #5110 #5117 #5112 #5127 #5126 #5134 #5106 #5107 #5100 #5095 #5089 #5079 #4560 #5086 #5024 #5025 #4634 #4588 #4575 #4565 #4570 #4566 #4564 #4809 #5015 #5073 #5074 #4693 #4666 #4658 #4836 #4818 #4665 #4578 #4964 #4612 #4618 #4577 #4648 #4663 #5267 #5248 #5242 #5247 #5246 #5252
- Improve MySQL 8.0 and MariaDB 10.3 support #4402 #4410 #5050 #4977 #4840 #4725
- mysqlctl may not have been able to find mysql_install_db for MySQL 5.6 #4982
- Inserts to lookup tables are now sorted to prevent deadlocks #4859 #4965
- Skip ACL checks for dual pseudo table #4516
- check replication lag on state change before starting query service #5000
- Correctly set basedir when starting mysqld #4993 #5027
- Fix ApplyRoutingRules -rules_file flag #5053
- Fix off-by-1 error for row_differ counting rows processed, not counting empty iteration #5016
- Remove unnecessary usage of FLUSH TABLES WITH READ LOCK #4849
- Add strong warning against production use to example init_db.sql file #4815
- Use jsonpb marshalling for vtctl's GetVSchema #4676
- Kill vtctld commands that outlive the http request context #4189
- Strictly check the structure of mysql static auth json config #4709
Functionality Added or Changed
- Changed default topology server to etcd2 #4869 #5003 #5029
- Remove support for InfluxDB as a monitoring backend #4503
- Improve support for MySQL bool and JSON data types #4609 #5039 #5037 #4842
- The topology server implementation has been refactored and significantly improved #4631 #5156 #5162 #5155 #4392 #4458 #4733 #4734 #4778 #4784 #4685 #4476 #5165 #5103 #5145 #4463 #5010 #4970 #4948 #4916 #4979 #4959 #4767 #4950 #5090 #4674 #4775 #4868 #4619 #4787 #4746 #4753 #4460 #4574
Major improvements are:- Per-cell autonomy has been improved. VTGates and VTTablets rely less upon the global topology to perform their operations.
- Topo to topo comparison tool for sanity checking migrations.
- Support for external workflows: allow storage of arbitrary key-value metadata in topo.
- Support aliases: the ability to configure groups of cells into concepts like regions.
- Experimental support for VRreplication #5179 #5163 #5159 #5130 #5114 #4491 #5093 #4914 #4839 #5048 #5012 #4946 #4844 #4841 #4791 #4829 #4736 #4703 #4637 #4702 #4555 #4571 #4449 #4455 #4981 #4760 #5367
- Instead of using vtworker for performing SplitClone, you can now directly invoke this functionality as vtctl commands. Additionally, these commands can also handle tables with text columns in their primary keys. Following these, you can the existing SplitDiff and Migrate commands just like before.
- VReplication has been componentized. You can compose your own workflows using a combination of VReplication commands, and the new vtctld commands: MigrateReads and MigrateWrites.
- Support for Consistent Lookup Vindexes #4861 #4978
- Support for auto-detecting MYSQL_FLAVOR #4987 #5225 #5161 #5105
- Improve log level and informativeness of error messages #5190 #5233 #5169 #4425 #5049 #4819 #4428 #4422 #5154 #4548 #4523 #4938 #4958 #4576 #4757 #4556 #4889 #4826 #4912 #4890 #4897 #4895 #4894 #4647 #5241
- Improve bundled my.cnf files and defaults #4999 #4410 #4872 #5366 #5309
- Support for Tracing #4610 #5195 #5175 #5118
- Several improvements to the helm chart #5144 #4443 #4430 #4478 #4494 #4493 #4473 #4988 #5115 #4952 #4764 #4536 #4521 #4506 #4664 #4726
- Improvements to the backup and restore features #5092 #5192 #5147 #5047 #5071 #5153 #5129 #5052 #5059 #4858 #4929 #4898 #4845 #4508 #4524 #4549 #4695 #4699 #4653 #5193 #5177 #5066 #5065 #4803 #4805 #4552 #4909 #5218 #5254 #5160 #5000 #5351 #5348 #5314 #5379 #5380
- Support for xtrabackup
- Ability to restore keyspace to an older time using backups
- Improve memory usage with ceph
- Update GCS client version to avoid backup corruption issues
- Vtbackup binary: validate previous backup before taking a new backup
- Detect if a previous restore failed and clean up before attempting to restore again
- Support backup with MyRocks
- Allow backup on master with -allow_master flag
- Vttablet should go back to healthy / serving after failed backup
- Tablets should not serve stale data after backup / restore, they should stay non-serving until they catch up to within the defined thresholds.
- Improvements to SQL execution engine #5146 #5121 #5097 #5111 #4432 #4400 #5094 #5141 #4519 #4512 #4067 #4486 #4510 #5150 #4384 #5124 #5080 #4470 #4527 #4526 #4528 #4513 #4507 #4533 #4581 #4616 #4739 #4822 #4830 #4785 #4853 #4732 #4773 #4930 #4900 #4917 #4662 #4833 #4743 #5140 #5072 #5136 #4996 #4420 #4882 #4966 #5120 #5209 #5096 #3862 #5018 #4534 #4558 #4936 #4824 #5023 #4469 #5020 #4336 #4761 #4933 #4881 #4949 #4867 #4854 #4041 #5277 #5219 #5186 #5259 #5338 #5336 #5324 #5301 #5398
- More ALTER VSCHEMA constructs
- Support for multi-statements
- Direct pass-through of autocommitable statements.
- Support for distinct aggregates like count(distinct…)
- Support for ordering of cross-shard queries and joins.
- Table equivalence. You can specify that two tables are equivalent. This will allow VTGate to choose the best plan option for the given query. This is useful if you use vreplication to materialize tables from one keyspace to another.
- Improvements to Resharding #5198 #5213 #5211 #5178 #5064 #5087 #5057 #5054 #5005 #4541 #4686 #4850 #5347 #5325 #5315
- Improvements to Java client #4991 #4441 #4594 #4680 #4484 #5040 #4599 #4864
- Improvements to MultiSplitDiff #4281 #4500 #4539 #5290
- Improve examples #4770 #4435 #4670 #4537 #5280 #5361 #5360 #5349 #5323
- vtexplain improvements #4796 #4774 #4755 #4681 #5139
- Improve Orchestrator integration #4961 #4788 #5143
- Improvements to VTTablet #5157 #4903 #4814 #4871 #4832 #4727 #4827 #4811 #4694 #4403 #4745 #4689 #4586 #4756 #4515 #5264 #5244 #5271
- Improvements to VTGate #4924 #4502 #4899 #4919 #4339 #4797 #4285 #4162 #4499 #5258 #5291
- Improvements to connection pooling #4769 #5184 #4729
- Experimental support for Vstream #4927 #4865 #4837 #5216
- Improvements to VTWorker #4592 #4585 #4633 #4243 #5232
- Improvements to reparenting #5235 #5226 #5236 #5293 #4550 #5151 #5210 #5391 #5376 #5363 #5316 #5300
- Add /debug/liveness handler for liveness checks #4935
- Add option to re-resolve DNS names #4793
- Surface consistent snapshot option in the UI for vertical splits #4759
- servenv: add new keepalive flag to grpc server #4737
- Chain grpc interceptors #4657
- Add setting the DB server name using a parameter #4408
- reload db-credentials-file upon SIGHUP #4514
- make FindSlaves keys be addresses #4595
- Add read-only security_policy to block admin HTTP endpoints #5321
- Vtcompose external db support #5306
Build Environment Changes
- Convert to building via Go modules #5215 #5109 #5185 #5167
- Improvements to build environment #4489 #5091 #4495 #4891 #4781 #4487 #5081 #4684 #4446 #4766 #4995 #4584 #4490 #4782 #5337 #5329 #5328 #5327 #5299
- Bump dependencies #4410 #4010 #4583 #4582 #4580 #4598 #4590 #5047 #5013 #5017 #4467 #4520 #4705 #5260 #5240 #4976 #5234 #5372 #5370
- Improve tests and/or stability of tests #5187 #4492 #4545 #4645 #4744 #5068 #5051 #5022 #4605 #4692 #5042 #4960 #4763 #4635 #5274 #5266 #5256 #5389 #5304 #5296 #5312
- Deb/rpm packages #4963
- Improvements to Docker #4951 #4794 #4706 #4423 #4554 #5262
- Functionality neutral code changes #4621 #4623 #4625 #4611 #4617 #4907 #4816 #4...
Vitess 3.0
Vitess 3.0 many new features over 2.2.
Download
- Server binaries are available in the Docker image "vitess/lite:v3.0.0-rc.1".
- Java client packages are published on Maven Central (To be uploaded).
- Source code is available via the tag "v3.0.0-rc.1".
Upgrading
All Vitess 3.0 changes are backward compatible with the 2.2 release. Please make sure that you follow the recommended upgrade order: vttablet->vtctld->vtgate.
Changes since 2.2
Usability
- Tutorial that runs on Minikube
- Tutorial that runs on local host
- Simplified db parameters for vttablet
- Formal support for externally managed mysql
Monitoring
- Prometheus plug-in
SQL constructs
- Comment directives:
QUERY_TIMEOUT_MS=1000
,MULTI_SHARD_AUTOCOMMIT
,SKIP_QUERY_PLAN_CACHE
- Partial Scatter
- New
ALTER VSCHEMA
syntax - Multi-table DDL statements
- Workbench connectivity
- Cross-shard subqueries
- Improved handling of
select *
constructs
Performance
- vtbench: Benchmarking tool
- MySQL protocol performance improvements
- Faster reparents
- Parallelized backups (4X faster)
- Snappier InitShardMaster and PlannedReparentShard
- Improved coordination with Orchestrator
New MySQL versions support
- MySQL 8.0 and MariaDB 10.3
Build
- Pure Go 1.11. You can now build with
CGO_ENABLED=0
.
Resharding and VReplication
- ShowResharding
- CancelResharding
- Safer MigrateServedTypes
- Auto-reverse direction of MigrateServedTypes to allow rollback of a previous MigrateServedTypes
- Workflow SplitDiff parallelized across shards
- SplitClone uses destination master for comparison