Vitess v3.0.0-rc.3

Fix a couple of issues in local example:

• restore vschema.sql to keep vagrant working
• teardown correct vttablets after horizontal resharding

Vitess v3.0.0-rc.2

• Fixed a typo in helm charts
• Removed settings that are incompatible with 8.0

Vitess v3.0.0-rc.1

This is the first release candidate of the 3.0 release.

We expect minimal changes between this and the final 3.0 release.

Download

• Server binaries are available in the Docker image "vitess/lite:v3.0.0-rc.1".
• Java client packages are published on Maven Central (To be uploaded).
• Source code is available via the tag "v3.0.0-rc.1".

Usability

• Tutorial that runs on Minikube
• Tutorial that runs on local host
• Simplified db parameters for vttablet
• Formal support for externally managed mysql

Monitoring

• Prometheus plug-in

SQL constructs

• Comment directives: QUERY_TIMEOUT_MS=1000, MULTI_SHARD_AUTOCOMMIT, SKIP_QUERY_PLAN_CACHE
• Partial Scatter
• New ALTER VSCHEMA syntax
• Multi-table DDL statements
• Workbench connectivity
• Cross-shard subqueries
• Improved handling of select * constructs

Performance

• vtbench: Benchmarking tool
• MySQL protocol performance improvements
• Faster reparents
• Parallelized backups (4X faster)
• Snappier InitShardMaster and PlannedReparentShard
• Improved coordination with Orchestrator

New MySQL versions support

• MySQL 8.0 and MariaDB 10.3

Build

• Pure Go 1.11. You can now build with CGO_ENABLED=0.

Resharding and VReplication

• ShowResharding
• CancelResharding
• Safer MigrateServedTypes
• Auto-reverse direction of MigrateServedTypes to allow rollback of a previous MigrateServedTypes
• Workflow SplitDiff parallelized across shards
• SplitClone uses destination master for comparison

Vitess 2.2

Vitess 2.2 brings many new features over 2.1.1.

Download

• Server binaries are available in the Docker image "vitess/lite:v2.2".
• Java client packages are published on Maven Central.
• Source code is available via the tag "v2.2".

Upgrading

All Vitess 2.2 changes are backward compatible with the 2.1.1 release. Please make sure that you follow our recommended upgrade order.

Changes since the 2.1.1 Release

VTGate

• Query normalization
• Streamlog and plan stats
• Instant-commit for autocommit
• Smarter pass-through of SQL statements for unsharded keyspaces or single-shard targets.
• Pass-through support for information_schema and dual table
• Improved master detection (if multiple vttablets declare themselves to be master)
• Multi-column vindexes

VTTablet

• Heartbeat
• Can run on a separate machine from its MySQL
• Hot row protection for DMLs
• Per-user transaction limits

Improved RBR support

• Vttablet allows all ‘INSERT..ON DUPLICATE KEY’ constructs if MySQL is in RBR mode.
• JSON support for RBR filtered replication and update stream
• A separate flag to pass-through all DMLs without rewriting them.

New Session-based protocol with support for TargetString and autocommit

• Legacy mode flag for those transitioning to the new behavior
• Auth plugin

MySQL protocol behaves more like mysql

• Autocommit is on by default, and mimics MySQL for set autocommit, begin & commit commands.
• Support of use statement with the ability to target keyspace, shard, tablet type or a combination of those.
• Support for CLIENT_FOUND_ROWS connection flag
• Support streaming

New constructs:

• SET TRANSACTION_MODE=[single,multi,2pc]
• SELECT NEXT :n VALUES FROM seq
• SHOW VINDEXES

Messages

• Multi-column support
• ACL
• New STREAM sql syntax

More query constructs

• Scatter aggregates, order by and limit (count and offset)
• Cross-shard delete
• INSERT ON DUPLICATE KEY with lookup vindexes
• INSERT IGNORE with lookup vindexes
• Cross-shard sorting of date/time and text columns
• UPDATE of secondary lookup vindex columns

Vindexes

• Autocommit support for lookup vindexes
• Write_only support which can be used for migration
• Keyrange support
• Null vindex

GRPC

• Upgrade to 1.10.0, faster and more efficient
• Allow setting of max message size for GRPC

Misc

• VTExplain: new tool for previewing query plans end-to-end
• Vtqueryserver: a new mysql proxy that has all the guardrails of vttablet, but can be used directly from an app.
• New docker-compose example
• New kubernetes helm charts
• Apache V2 license
• Move to CNCF
• Java client moved under io.vitess root
• Query logging support to output as JSON and log to a file
• Improved caching of topology server state at vtgate to allow queries to continue when the topo service is unavailable.
• Optional flags to limit the time a query will wait for a connection from the pool and the total number of outstanding requests.
• Options to redact and/or truncate sensitive queries in the debug UIs

Deprecated

• Queryserver-config-strict-mode
• PHP client
• V1 toposerver scheme

Vitess v2.2.0-rc.1

This is the first release candidate of the 2.2 release.

We expect minimal changes between this and the final 2.2 release.

Download

• Server binaries are available in the Docker image "vitess/lite:v2.2.0-rc.1".
• Java client packages are published on Maven Central.
• Source code is available via the tag "v2.2.0-rc.1".

Upgrading

All Vitess 2.2 changes are backward compatible with the 2.1.1 release. Please make sure that you follow our recommended upgrade order.

Changes since the 2.1.1 Release

VTGate

• Query normalization
• Streamlog and plan stats
• Instant-commit for autocommit
• Smarter pass-through of SQL statements for unsharded keyspaces or single-shard targets.
• Pass-through support for information_schema and dual table
• Improved master detection (if multiple vttablets declare themselves to be master)
• Multi-column vindexes

VTTablet

• Heartbeat
• Can run on a separate machine from its MySQL
• Hot row protection for DMLs
• Per-user transaction limits

Improved RBR support

• Vttablet allows all ‘INSERT..ON DUPLICATE KEY’ constructs if MySQL is in RBR mode.
• JSON support for RBR filtered replication and update stream
• A separate flag to pass-through all DMLs without rewriting them.

New Session-based protocol with support for TargetString and autocommit

• Legacy mode flag for those transitioning to the new behavior
• Auth plugin

MySQL protocol behaves more like mysql

• Autocommit is on by default, and mimics MySQL for set autocommit, begin & commit commands.
• Support of use statement with the ability to target keyspace, shard, tablet type or a combination of those.
• Support for CLIENT_FOUND_ROWS connection flag
• Support streaming

New constructs:

• SET TRANSACTION_MODE=[single,multi,2pc]
• SELECT NEXT :n VALUES FROM seq
• SHOW VINDEXES

Messages

• Multi-column support
• ACL
• New STREAM sql syntax

More query constructs

• Scatter aggregates, order by and limit (count and offset)
• Cross-shard delete
• INSERT ON DUPLICATE KEY with lookup vindexes
• INSERT IGNORE with lookup vindexes
• Cross-shard sorting of date/time and text columns
• UPDATE of secondary lookup vindex columns

Vindexes

• Autocommit support for lookup vindexes
• Write_only support which can be used for migration
• Keyrange support
• Null vindex

GRPC

• Upgrade to 1.10.0, faster and more efficient
• Allow setting of max message size for GRPC

Misc

• VTExplain: new tool for previewing query plans end-to-end
• Vtqueryserver: a new mysql proxy that has all the guardrails of vttablet, but can be used directly from an app.
• New docker-compose example
• New kubernetes helm charts
• Apache V2 license
• Move to CNCF
• Java client moved under io.vitess root
• Query logging support to output as JSON and log to a file
• Improved caching of topology server state at vtgate to allow queries to continue when the topo service is unavailable.
• Optional flags to limit the time a query will wait for a connection from the pool and the total number of outstanding requests.
• Options to redact and/or truncate sensitive queries in the debug UIs

Deprecated

• Queryserver-config-strict-mode
• PHP client
• V1 toposerver scheme

Vitess 2.1.1

Bugfix release for Vitess 2.1.

Changes since 2.1:

• Fixed issue that auto inc columns for unsharded keyspaces were not working (#2804).

If you are upgrading from a version < 2.1.0, please read the "Upgrading" section from the 2.1 release notes: https://github.com/youtube/vitess/releases/tag/v2.1.0

Vitess 2.1

Vitess 2.1 brings many new features since the last major release 2.0.

Highlights are Two-Phase Commit (2PC) support and early support for the native MySQL protocol.

See below for full details.

Download

• Server binaries are available in the Docker image "vitess/lite:v2.1.0".
• Java client packages are published on Maven Central.
• Source code is available via the tag "v2.1.0".

Upgrading

All Vitess 2.1 changes are backward compatible with the 2.0 release. Please make sure that you follow our recommended upgrade order.

If you use custom start scripts, you will have to change the command line flags which specify the topology (zookeeper or etcd) addresses. New flags are:

-topo_implementation zookeeper|etcd|zk2|etcd2|consul
-topo_global_server_address global_server1,global_server2
-topo_global_root /vitess/global

Additionally, for the zk2/etcd2/consul implementations, you'll have to register cell local topology servers with the vtctl AddCellInfo command. See our documentation for details.

This version also adds new topology plugins ("zk2", "etcd2") with a different storage format. The legacy plugins "zookeeper" and "etcd" will continue to work in 2.1 but will be deprecated in the next major version. Therefore, we recommend to migrate to one of the new plugins once you updated to 2.1.

Changes since the 2.0 Release

• Our website vitess.io has a new logo! :)
• Kubernetes:
  • Added Helm chart and support for StatefulSet.
• all:
  • Topology:
    • Rewrote plugins for etcd and Zookeeper (etcd2, zk2). Please see the doc how to migrate to the new storage format.
      Command line flags have changed as well (shipped examples are updated).
    • New plugin: Consul.
  • Internal: We have changed the internal error representation in the RPC messages in a backward compatible way.
• clients:
  • Java:
    • TLS support in client and JDBC driver.
    • Unified package structure under new root io.vitess.
    • 2.1.0 packages are published on Maven Central e.g. our JDBC driver as artifact vitess-jdbc.
  • Map Reduce:
    • Faster execution on tables with a non-uniform distribution by sampling rows and splitting the table into equal-sized row ranges.
• vtctld:
  • New Angular 2 based UI.
  • Online Schema Swap: Apply complex schema changes without any downtime.
  • Resharding workflow. Automatically runs the resharding steps and shows them in the Workflow UI Tab. New tutorials added which use the workflow.
• vtgate:
  • 2PC: Support for distributed transactions.
  • Native MySQL server protocol (alpha).
  • Direct access to shards via "keyspace:shard" database name.
  • Buffer master traffic during failovers and retry to avoid client errors (disabled by default).
  • Support for SHOW statements to expose Vitess metadata.
  • Increase large installation scalability with (optional) two-layer vtgate pools (l2vtgate, applicable to 100+ shard installations).
• vttablet:
  • Native MySQL client protocol (instead of using libmysqlclient) (disabled by default).
  • Support for MySQL Row-based-Replication e.g. needed when resharding (alpha).
  • Update Stream: Applications can subscribe to a change stream (e.g. to invalidate an app level cache).
  • Messages: Native support for a queue processing framework (alpha).
  • Hot Row Protection: Serializes write transaction for the same row (range) to prevent that all transaction pool slots can be exhausted (disabled by default).
  • Transaction Throttler: Rejects new transactions with retryable error if replication lag on replicas is too high (disabled by default).

Other Noteworthy Changes

• Minimum required Go version changed to 1.8 (from 1.5).
• Updated to gRPC 1.0.4 (from 1.0.0).

Vitess v2.1.0-rc.1

This is the first release candidate of the 2.1 release.

We expect minimal changes between this and the final 2.1 release.

Download

• Server binaries are available in the Docker image "vitess/lite:v2.1.0-rc.1".
• Java client packages are published on Maven Central.
• Source code is available via the tag "v2.1.0-rc.1".

Upgrading

All Vitess 2.1 changes are backward compatible with the 2.0 release. Please make sure that you follow our recommended upgrade order.

Changes since the 2.0 Release

• Our website vitess.io has a new logo! :)
• Kubernetes:
  • Added Helm chart and support for StatefulSet.
• all:
  • Topology:
    • Rewrote plugins for etcd and Zookeeper (etcd2, zk2). Please see the doc how to migrate to the new storage format.
      Command line flags have changed as well (shipped examples are updated).
    • New plugin: Consul.
  • Internal: We have changed the internal error representation in the RPC messages in a backward compatible way.
• clients:
  • Java:
    • TLS support in client and JDBC driver.
    • Unified package structure under new root io.vitess.
    • 2.1.0-rc.1 packages are published on Maven Central e.g. our JDBC driver as artifact vitess-jdbc.
  • Map Reduce:
    • Faster execution on tables with a non-uniform distribution by sampling rows and splitting the table into equal-sized row ranges.
• vtctld:
  • New Angular 2 based UI.
  • Online Schema Swap: Apply complex schema changes without any downtime.
  • Resharding workflow. Automatically runs the resharding steps and shows them in the Workflow UI Tab. New tutorials added which use the workflow.
• vtgate:
  • 2PC: Support for distributed transactions.
  • Native MySQL server protocol (alpha).
  • Direct access to shards via "keyspace:shard" database name.
  • Buffer master traffic during failovers and retry to avoid client errors (disabled by default).
  • Support for SHOW statements to expose Vitess metadata.
  • Increase large installation scalability with (optional) two-layer vtgate pools (l2vtgate, applicable to 100+ shard installations).
• vttablet:
  • Native MySQL client protocol (instead of using libmysqlclient) (disabled by default).
  • Support for MySQL Row-based-Replication e.g. needed when resharding (alpha).
  • Update Stream: Applications can subscribe to a change stream (e.g. to invalidate an app level cache).
  • Messages: Native support for a queue processing framework (alpha).
  • Hot Row Protection: Serializes write transaction for the same row (range) to prevent that all transaction pool slots can be exhausted (disabled by default).
  • Transaction Throttler: Rejects new transactions with retryable error if replication lag on replicas is too high (disabled by default).

Other Noteworthy Changes

• Minimum required Go version changed to 1.8 (from 1.5).
• Updated to gRPC 1.0.4 (from 1.0.0).

Vitess v2.1.0-alpha.1

This is the first alpha release of the upcoming 2.1 version.

2.1 will be a feature release which will bring the following new features:

• Support for distributed transactions, using 2 phase commit.
• Resharding workflow improvements, to increase manageability of the process.
• Online schema swap, to apply complex schema changes without any downtime.
• New dynamic UI (vtctld), rewritten from scratch in angular2.
• Update Stream functionality, for applications to subscribe to a change stream (for cache invalidation, for instance).
• Improved Map-Reduce support, for tables with non-uniform distributions.
• Increase large installation scalability with two-layer vtgate pools (l2vtgate, applicable to 100+ shard installations).
• Better Kubernetes support (Helm support, better scripts, ...).

Note that some of these features are still in development and not fully documented. This will change before we release the final 2.1 version.

Other noteworthy changes:

• Minimum required Go version changed to 1.7 (from 1.5).
• Updated to gRPC 1.0.0 (from 0.13.0).
• Added Percona Docker Images (changes contributed by Stitch Labs).

Vitess v2.0.0

Release to Manufacturing

This is the code cut for what will become the first GA release of Vitess 2.0.
We're now focused on updating documentation in preparation for General Availability.

Changes since v2.0.0-rc.2

Code Cleanup

• vtctld: Remove experimental VSchema editor UI