14 Jan 21:29

frouioui

638ce93

Test FPO 2 Latest

Latest

test

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>

Assets 5

14 Jan 21:13

frouioui

testfpo

5e22200

Test FPO

Modify Assign Milestone workflow (#373)

* Modify Assign Milestone workflow

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>

* Add comments

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>

* Harden the create_release.yml workflow

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>

* Modify the docker_build_images.yml workflow

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>

* Disable persist-credentials where possible

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>

---------

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>

Assets 2

08 Nov 23:58

frouioui

v21.0.1

9b2cd2c

Vitess v21.0.1

Release of Vitess v21.0.1

Assets 5

20 Jun 22:40

frouioui

v20.0.0

3a06a54

Vitess v20.0.0

Release of Vitess v20.0.0

Summary

Major Changes
Minor Changes

Major Changes

Deletion

`--vreplication_tablet_type` flag

The previously deprecated flag --vreplication_tablet_type has been deleted.

Pool Capacity Flags

The previously deprecated flags --queryserver-config-query-pool-waiter-cap, --queryserver-config-stream-pool-waiter-cap and --queryserver-config-txpool-waiter-cap have been deleted.

`vitess/base` and `vitess/k8s` Docker images

Since we have deleted MySQL from our vitess/lite image, we are removing the vitess/base and vitess/k8s images.

These images are no longer useful since we can use vitess/lite as the base of many other Docker images (vitess/vtgate, vitess/vtgate, ...).

`gh-ost` binary and endtoend tests

Vitess 20.0 drops support for gh-ost DDL strategy.

vttablet binary no longer embeds a gh-ost binary. Users of gh-ost DDL strategy will need to supply a gh-ost binary on the vttablet host or pod. Vitess will look for the gh-ost binary in the system PATH; otherwise the user should supply vttablet --gh-ost-path.

Vitess' endtoend tests no longer use nor test gh-ost migrations.

Legacy `EmergencyReparentShard` stats

The following EmergencyReparentShard stats were deprecated in Vitess 18.0 and are removed in Vitess 20.0:

ers_counter
ers_success_counter
ers_failure_counter

These counters are replaced by the following stats (introduced in Vitess 18.0):

emergency_reparent_counts - Number of times EmergencyReparentShard has been run. It is further subdivided by the keyspace, shard and the result of the operation.
planned_reparent_counts - Number of times PlannedReparentShard has been run. It is further subdivided by the keyspace, shard and the result of the operation.

Also, the reparent_shard_operation_timings stat was added to provide per-operation timings of reparent operations.

Breaking Changes

Metric Name Changes in VTOrc

The following metric names have been changed in VTOrc. The old metrics are still available in /debug/vars for this release, but will be removed in later releases. The new metric names and the deprecated metric names resolve to the same metric name on prometheus, so there is no change there.

Old Metric Name	New Metric Name	Name in Prometheus
`analysis.change.write`	`AnalysisChangeWrite`	`vtorc_analysis_change_write`
`audit.write`	`AuditWrite`	`vtorc_audit_write`
`discoveries.attempt`	`DiscoveriesAttempt`	`vtorc_discoveries_attempt`
`discoveries.fail`	`DiscoveriesFail`	`vtorc_discoveries_fail`
`discoveries.instance_poll_seconds_exceeded`	`DiscoveriesInstancePollSecondsExceeded`	`vtorc_discoveries_instance_poll_seconds_exceeded`
`discoveries.queue_length`	`DiscoveriesQueueLength`	`vtorc_discoveries_queue_length`
`discoveries.recent_count`	`DiscoveriesRecentCount`	`vtorc_discoveries_recent_count`
`instance.read`	`InstanceRead`	`vtorc_instance_read`
`instance.read_topology`	`InstanceReadTopology`	`vtorc_instance_read_topology`
`emergency_reparent_counts`	`EmergencyReparentCounts`	`vtorc_emergency_reparent_counts`
`planned_reparent_counts`	`PlannedReparentCounts`	`vtorc_planned_reparent_counts`
`reparent_shard_operation_timings`	`ReparentShardOperationTimings`	`vtorc_reparent_shard_operation_timings_bucket`

ENUM and SET column handling in VTGate VStream API

The VTGate VStream API now returns ENUM and SET column type values in VEvent messages (in the embedded RowChange messages) as their string values instead of the integer based ones — in both the copy/snapshot phase and the streaming phase. This change was done to make the VStream API more user-friendly, intuitive, and to align the behavior across both phases. Before this change the values for ENUM and SET columns were string values in the copy phase but integer values (which only have an internal meaning to MySQL) in the streaming phase. This inconsistency led to various challenges and issues for each VStream client/consumer (e.g. the Debezium Vitess connector failed to properly perform a snapshot for tables containing these column types). Now the behavior is intuitive — clients need the string values as the eventual sink is often not MySQL so each consumer needed to perform the mappings themselves — and consistent. While this is a (potentially) breaking change, a new boolean field has been added to the FieldEvent message called EnumSetStringValues. When that field is false (in Vitess v19 and older) then the consumer will need to perform the mappings during streaming phase, but not during copy phase. When this field is true, then no mapping is required. This will help to ensure a smooth transition for all consumers over time. To demonstrate, let's look at the textual output (printing the received VEvents as strings) when streaming a single enum_set_test table from the unsharded commerce keyspace so that we can see what the VStream looks like before and after when we start a new VStream in copy/snapshot mode and then transition to streaming mode for the following table:

CREATE TABLE `enum_set_test` (
  `id` int NOT NULL AUTO_INCREMENT,
  `name` varchar(120) DEFAULT NULL,
  `shirt_size` enum('small','medium','large','xlarge'...

Contributors

frouioui

Assets 5

23 May 16:10

frouioui

v20.0.0-rc1

056060f

Vitess v20.0.0-rc1 Pre-release

Pre-release

Release of Vitess v20.0.0

Major Changes

Deletion

`--vreplication_tablet_type` flag

The previously deprecated flag --vreplication_tablet_type has been deleted.

Pool Capacity Flags

The previously deprecated flags --queryserver-config-query-pool-waiter-cap, --queryserver-config-stream-pool-waiter-cap and --queryserver-config-txpool-waiter-cap have been deleted.

MySQL binaries in the `vitess/lite` Docker images

In v19.0.0 we had deprecated the mysqld binary in the vitess/lite Docker image.
Making MySQL/Percona version specific image tags also deprecated.

Starting in v20.0.0 we no longer build the MySQL/Percona version specific image tags.
Moreover, the mysqld binary is no longer present on the vitess/lite image.

Here are the images we will no longer build and push:

Image	Available
`vitess/lite:v20.0.0`	YES
`vitess/lite:v20.0.0-mysql57`	NO
`vitess/lite:v20.0.0-mysql80`	NO
`vitess/lite:v20.0.0-percona57`	NO
`vitess/lite:v20.0.0-percona80`	NO

If you have not done it yet, you can use an official MySQL Docker image for your mysqld container now such as: mysql:8.0.30.
Below is an example of a kubernetes yaml file before and after upgrading to an official MySQL image:

# before:

# you are still on v19 and are looking to upgrade to v20
# the image used here includes MySQL 8.0.30 and its binaries

    mysqld:
      mysql80Compatible: vitess/lite:v19.0.0-mysql80

# after:

# if we still want to use MySQL 8.0.30, we now have to use the
# official MySQL image with the 8.0.30 tag as shown below 

    mysqld:
      mysql80Compatible: mysql:8.0.30 # or even mysql:8.0.34 for instance

`vitess/base` and `vitess/k8s` Docker images

Since we have deleted MySQL from our vitess/lite image, we are removing the vitess/base and vitess/k8s images.

These images are no longer useful since we can use vitess/lite as the base of many other Docker images (vitess/vtgate, vitess/vtgate, ...).

`gh-ost` binary and endtoend tests

Vitess 20.0 drops support for gh-ost DDL strategy.

Vitess' endtoend tests no longer use nor test gh-ost migrations.

Breaking Changes

ENUM and SET column handling in VTGate VStream API

CREATE TABLE `enum_set_test` (
  `id` int NOT NULL AUTO_INCREMENT,
  `name` varchar(120) DEFAULT NULL,
  `shirt_size` enum('small','medium','large','xlarge','xxlarge') DEFAULT NULL,
  `hobbies` set('knitting','cooking','pickleball','biking','hiking','motorcycle','video games','reading') DEFAULT NULL,
  PRIMARY KEY (`id`)
)

And with the table having this data when we start our VStream and begin the copy/snapshot phase:

mysql> select * from enum_set_test;
+----+-----------+------------+-------------------------+
| id | name      | shirt_size | hobbies                 |
+----+-----------+------------+-------------------------+
|  1 | Billy Bob | xlarge     | cooking,reading         |
|  2 | Sally Mae | medium     | knitting,cooking,hiking |
+----+-----------+------------+-------------------------+
2 rows in set (0.00 sec)

And finally we will perform the following inserts and updates to the table during the streaming phase:

insert into enum_set_test values (3, "Matt Lord", 'medium', 'pickleball,biking,hiking,motorcycle,video games,reading');
insert into enum_set_test values (4, "Jerry Badyellow", 'large', '');
update enum_set_test set shirt_size = 'small', hobbies = 'knitting,cooking,hiking,reading' where id = 2;

Vitess v19 and older:

[type:BEGIN keyspace:"commerce" shard:"0" type:FIELD field_event:{table_name:"commerce.enum_set_test" fields:{name:"id" type:INT32 table:"enum_set_test" org_table:"enum_set_test" database:"vt_commerce" org_name:"id" column_length:11 charset:63 flags:49667 column_type:"int"} fields:{name:"name" type:VARCHAR table:"enum_set_test" org_table:"enum_set_test" database:"vt_commerce" org_name:"name" column_length:480 charset:255 column_type:"varchar(120)"} fields:{name:"shirt_size" type:ENUM table:"enum_set_test" org_table:"enum_set_test" database:"vt_commerce" org_name:"shirt_size" column_length:28 charset:255 flags:256 column_type:"enum('small','medium','large','xlarge','xxlarge')"} fields:{name:"hobbies" type:SET table:"enum_set_test" org_table:"enum_set_test" database:"vt_commerce" org_name:"hobbies" column_length:288 charset:255 flags:2048 column_type:"set('knitting','cooking','pickleball','biking','hiking','motorcycle','video games','reading')"} keyspace:"commerce" shard:"0"} keyspace:"commerce" shard:"0"]
[type:VGTID vgtid:{shard_gtids:{...

Contributors

frouioui

Assets 5

20 Feb 21:25

frouioui

v19.0.1

18e944c

Vitess v19.0.1

Release of Vitess v19.0.1

Assets 5

20 Feb 21:14

frouioui

v19.0.0

11c27a6

Vitess v19.0.0

Release of Vitess v19.0.0

Summary

Major Changes
Minor Changes
- Apply VSchema
  - --strict sub-flag and strict gRPC field

Major Changes

Dropping Support for MySQL 5.7

Oracle has marked MySQL 5.7 end of life as of October 2023. Vitess is also dropping support for MySQL 5.7 from v19 onwards. Users are advised to upgrade to MySQL 8.0 while on v18 version of Vitess before
upgrading to v19.

Vitess will however, continue to support importing from MySQL 5.7 into Vitess even in v19.

Deprecations and Deletions

The MYSQL_FLAVOR environment variable is now removed from all Docker Images.

VTTablet Flags

The following flags — which were deprecated in Vitess 7.0 — have been removed:
--vreplication_healthcheck_topology_refresh, --vreplication_healthcheck_retry_delay, and --vreplication_healthcheck_timeout.
The --vreplication_tablet_type flag is now deprecated and ignored.

Docker Image vitess/lite

The mysqld binary is now deprecated in the vitess/lite Docker image and will be removed in a future release.
This means that the MySQL/Percona version specific image tags for the vitess/lite image are deprecated.

Below is a full list of available tags for v19.0.0 and their deprecation status:

Image	Deprecated
`vitess/lite:v19.0.0`	NO
`vitess/lite:v19.0.0-mysql57`	YES
`vitess/lite:v19.0.0-mysql80`	YES
`vitess/lite:v19.0.0-percona57`	YES
`vitess/lite:v19.0.0-percona80`	YES

If you are currently using vitess/lite as your mysqld image in your vitess-operator deployment we invite you to use an official MySQL image, such as mysql:8.0.30.

Below is an example of a kubernetes yaml file before and after upgrading to an official MySQL image:

# before:

# the image used here includes MySQL 8.0.30 and its binaries

    mysqld:
      mysql80Compatible: vitess/lite:v19.0.0-mysql80

# after:

# if we still want to use MySQL 8.0.30, we now have to use the
# official MySQL image with the 8.0.30 tag as shown below 

    mysqld:
      mysql80Compatible: mysql:8.0.30 # or even mysql:8.0.34 for instance

Explain Statement Format

Explain statement format vitess and vexplain were deprecated in v16 and removed in v19 version.
Use VExplain Statement for understanding Vitess plans.

Breaking Changes

ExecuteFetchAsDBA rejects multi-statement SQL

vtctldclient ExecuteFetchAsDBA (and similarly the vtctl and vtctlclient commands) now reject multi-statement SQL with error.

For example, vtctldclient ExecuteFetchAsDBA my-tablet "stop replica; change replication source to auto_position=1; start replica will return an error, without attempting to execute any of these queries.

Previously, ExecuteFetchAsDBA silently accepted multi statement SQL. It would (attempt to) execute all of them, but:

It would only indicate error for the first statement. Errors on 2nd, 3rd, ... statements were silently ignored.
It would not consume the result sets of the 2nd, 3rd, ... statements. It would then return the used connection to the pool in a dirty state. Any further query that happens to take that connection out of the pool could get unexpected results.
As another side effect, multi-statement schema changes would cause schema to be reloaded with only the first change, leaving the cached schema inconsistent with the underlying database.

ExecuteFetchAsDBA does allow a specific use case of multi-statement SQL, which is where all statements are in the form of CREATE TABLE or CREATE VIEW. This is to support a common pattern of schema initialization, formalized in ApplySchema --batch-size which uses ExecuteFetchAsDBA under the hood.

New Stats

Stream Consolidations

Prior to 19.0 VTTablet reported how much time non-streaming executions spend waiting for consolidations to occur. In 19.0, VTTablet reports a similar stat for streaming executions in /debug/vars stat Waits.Histograms.StreamConsolidations.

Build Version in `/debug/vars`

The build version (e.g., 19.0.0-SNAPSHOT) has been added to /debug/vars, allowing users to programmatically inspect Vitess components' build version at runtime.

Planned Reparent Shard

`--tolerable-replication-lag` Sub-flag

A new sub-flag --tolerable-replication-lag has been added to the command PlannedReparentShard that allows users to specify the amount of replication lag that is considered acceptable for a tablet to be eligible for promotion when Vitess makes the choice of a new primary.
This feature is opt-in and not specifying this sub-flag makes Vitess ignore the replication lag entirely.

A new flag in VTOrc with the same name has been added to control the behaviour of the PlannedReparentShard calls that VTOrc issues.

Query Compatibility

Multi Table Delete Support

Support is added for sharded multi-table delete with target on single table using multiple table join.

Example: Delete t1 from t1 join t2 on t1.id = t2.id join t3 on t1.col = t3.col where t3.foo = 5 and t2.bar = 7

More details about how it works is available in MySQL Docs

`SHOW VSCHEMA KEYSPACES` Query

A SQL query, SHOW VSCHEMA KEYSPACES is now supported in Vitess. This query prints the vschema information
for all the keyspaces. It is useful for seeing the foreign key mode, whether the keyspace is sharded, and if there is an
error in the VSchema for the keyspace.

An example output of the query looks like -

mysql> show vschema keyspaces;
+----------+---------+-------------+---------+
| Keyspace | Sharded | Foreign Key | Comment |
+----------+---------+-------------+---------+
| ks       | true    | managed     |         |
| uks      | false   | managed     |         |
+----------+---------+-------------+---------+
2 rows in set (0.01 sec)

`FOREIGN_KEY_CHECKS` is now a Vitess Aware Variable

When VTGate receives a query to change the FOREIGN_KEY_CHECKS value for a session, instead of sending the value down to MySQL, VTGate now keeps track of the value and changes the queries by adding SET_VAR(FOREIGN_KEY_CHECKS=On/Off) style query optimizer hints wherever required.

Explain Statement

Explain statement can handle routed table queries now. Explain is unsupported when the tables involved in the query refers more than one keyspace. Users should use VExplain Statement in those cases.

Partial Multi-shard Commit Warnings

When using multi transaction mode (the default), it is possible for Vitess to successfully commit to one shard, but fail to commit to a subsequent shard, thus breaking the atomicity of a multi-shard transaction.

In v19.0, VTGate reports partial-success commits in warnings, e.g.:

mysql> commit;
ERROR 1317 (70100): target: customer.-80.primary: vttablet: rpc error: code = Aborted desc = transaction 1703182545849001001: ended at 2023-12-21 14:07:41.515 EST (exceeded timeout: 30s) (CallerID: userData1)
mysql> show warnings;
+---------+------+----------------------------------------------------------+
| Level   | Code | Message                                                  |
+---------+------+----------------------------------------------------------+
| Warning |  301 | multi-db commit failed after committing to 1 shards: 80- |
+---------+------+----------------------------------------------------------+
1 row in set, 1 warning (0.00 sec)

Vttestserver

`--vtcombo-bind-host` flag

A new flag --vtcombo-bind-host has been added to vttestserver that allows the users to configure the bind host that vtcombo uses. This is especially useful wh...

Contributors

frouioui

Assets 5

15 Feb 20:05

frouioui

v18.0.3

40944e9

Vitess v18.0.3

Release of Vitess v18.0.3

Summary

Major Changes
- Breaking Changes
  - ExecuteFetchAsDBA rejects multi-statement SQL

It would only indicate error for the first statement. Errors on 2nd, 3rd, ... statements were silently ignored.
It would not consume the result sets of the 2nd, 3rd, ... statements. It would then return the used connection to the pool in a dirty state. Any further query that happens to take that connection out of the pool could get unexpected results.
As another side effect, multi-statement schema changes would cause schema to be reloaded with only the first change, leaving the cached schema inconsistent with the underlying database.

The entire changelog for this release can be found here.

The release includes 1 merged Pull Requests.

Thanks to all our contributors: @frouioui

Contributors

frouioui

Assets 5

21 Dec 08:37

frouioui

v18.0.2

cdee9fc

Vitess v18.0.2

Release of Vitess v18.0.2

The entire changelog for this release can be found here.

The release includes 7 merged Pull Requests.

Thanks to all our contributors: @frouioui

Contributors

frouioui

Assets 5

26 Jun 18:26

frouioui

86e9bef

Test vtadmin build

tt

Merge pull request #2 from frouioui/optimize-vtadmin-build

Assets 2

Releases: frouioui/vitess

Test FPO 2

Test FPO

Vitess v21.0.1

Release of Vitess v21.0.1

Vitess v20.0.0

Release of Vitess v20.0.0

Summary

Table of Contents

Contributors

Vitess v20.0.0-rc1

Release of Vitess v20.0.0

Summary

Table of Contents

Contributors

Vitess v19.0.1

Release of Vitess v19.0.1

Vitess v19.0.0

Release of Vitess v19.0.0

Summary

Table of Contents

Contributors

Vitess v18.0.3

Release of Vitess v18.0.3

Summary

Table of Contents

Contributors

Vitess v18.0.2

Release of Vitess v18.0.2

Contributors

Test vtadmin build