Skip to content
This repository has been archived by the owner on Dec 16, 2022. It is now read-only.

Merge upstream 2019-04-25 #130

Merged
merged 158 commits into from
Apr 26, 2019
Merged

Merge upstream 2019-04-25 #130

merged 158 commits into from
Apr 26, 2019

Conversation

setassociative
Copy link

It's been busy and exciting in vitess upstream. Some changes that felt worth calling out as they could impact assumptions about behavior:

Slack changes
@demmer vitessio#4827 Correct suppress logging for begin...commit in autocommit
@rafael vitessio#4824 adds timeouts for all statements
@demmer vitessio#4826 adds logging of stack traces as opt-in
@demmer vitessio#4819 remove begin/commit logs from autocommit txns
@demmer vitessio#4796 improves support for vtexplain for begin/dml/dml/commit txns

Non slack changes
vitessio#4839 Improve behavior with reference table routing & vreplication
vitessio#4833 Support query routing given there could be multiple targets for a table
vitessio#4832 tablets get new health check behavior (health checks time out)
vitessio#4837 refresh bug around row streamer handling creds
vitessio#4830 apply a default value to the db_name attribute in local_metadata
vitessio#4785 SHOW SCHEMAS aliased to SHOW DATABASES
vitessio#4829 add experimental support for split clone & vertical split clone via vreplication
vitessio#4822 do not normalize in order by
vitessio#4791 Adds vreplication row streaming support
vitessio#4814 command line flags now used regardless of management mode
vitessio#4811 vttablet sets super_read_only during planned reparent
vitessio#4803 xtrabackup testing
vitessio#4727 support multiple vttablets running against one mysql instance
vitessio#4746 TopoCat can now produce JSON output
vitessio#4805 mysqld is now restarted after a failed backup
vitessio#4685 macos bootstrap bug
vitessio#4874 ZK opts and java land
vitessio#4695 Adds support for xtrabackup
vitessio#4794 fixes build failures
vitessio#4725 changes mysql8.0 start args
vitessio#4736 introduce new states to support vreplication
vitessio#4788 expands orc error

Docs etc
vitessio#4831 lint
vitessio#4827 docs
vitessio#4816 const declaration cleanup
vitessio#4820 const declaration cleanup
vitessio#4825 docs
vitessio#4818 docs
vitessio#4809 docs
vitessio#4812 moves consts around
vitessio#4813 docs
vitessio#4808 docs
vitessio#4800 docs
vitessio#4795 docs

gak and others added 30 commits February 19, 2019 11:16
Instead of downloading Linux binaries when running macOS, detect platform
and download the appropriate binary.

Signed-off-by: Gerald Kaszuba <gak@gak0.com>
* Deprecate the concept of regions.
* Favor aliases instead

Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
Signed-off-by: dleibovic <dleibovic@etsy.com>
…unsupported_cases.txt

vitessio#4772

Signed-off-by: David Weitzman <dweitzman@pinterest.com>
Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
…meout.

Signed-off-by: Michael Pawliszyn <mikepaw@squareup.com>
…gr_limitation

Document that left-joins don't recognize transitive relationships in …
…check-backoff

Limit the connection retry delay to no longer than the healthcheck timeout
Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
Signed-off-by: Adam Saponara <as@php.net>
Signed-off-by: Anthony Yeh <enisoc@planetscale.com>
orchestrator: Add more detail to error messages.
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
vplayer used to previously pass through the regular expressions.
With this change, regular expressions are only used to identify
target tables. And then, the request sent to vstreamer requests
specific table names and the filter is converted to use select
statememnts.

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
vreplication: table copying phase 1: create list of tables to copy
Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
…k warnings

Signed-off-by: deepthi <deepthi@planetscale.com>
…storing xtrabackup

Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
sougou and others added 27 commits April 22, 2019 20:02
In this change the query routing takes the possibility that
there could be multiple target options for a given table. The
design for this is explained in vitessio#4790.

At a high level:
* VSchema.FindTableOrVindex function can return a list of
  tables instead of a single one.
* The route planbuilder creates multiple routeOptions, one
  for each table returned.
* All actions that affected the plan of a route are changed
  to update all routeOptions.
* If a particular routeOption cannot accommodate a pushed
  down construct, it's removed from the list. Previously,
  this was an error case. But if no options are left, then
  we return an error.
* If two routeOptions qualify for a merge of routes, then
  all other combinations that don't qualify are discarded.
  This is the case for joins, subqueries and unions.

More details:
vindexTable was renamed to the more appropriate vschemaTable.

In order to achieve this, a new routeOption data type was
introduced, and route was changed to contain a list of
routeOptions.

In symtab, tables used to point at the vschema table that
was used to build them. Since a table can now represent
multiple target tables, this field has been moved into
routeOption.

In symtab, columns used to contain a vindex member. Since
this can change depending on the target table, the routeOption
now contains a map of column to vindexes instead.

The routeOption also contains the vschemaTable. DMLs use
this information. Since DMLs have to be more deterministic
about the table they write to, they always choose the
first option.

At the beginning of the Wireup phase, we evaluate all existing
options and decide on the best available.

To be done:
When a table has multiple targets, the targets can have different
names than the original table. If so, the queries have to be
rewritten to address the new target tables. In order to do this,
each routeOption will contain a list of substitutions that will
be made during the Wireup phase.

Tests have to be written for the new flows.

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
The code has also been refactored and cleaned up
for improved readability.

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
sqlparser: support for SHOW SCHEMAS
Signed-off-by: deepthi <deepthi@planetscale.com>
The row streamer needed to refresh credentials before connecting
to mysql.

Also fixed some flaky tests.

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
…cently

This logic already existed in go/cmd/vttablet/healthz.go for /healthz, but
did not exist for broadcastHealth() and did not exist for deciding whether
to return a cached health value to a newly-connected streaming health client.

Signed-off-by: David Weitzman <dweitzman@pinterest.com>
vstreamer: bug fix on mysql connection
…me_streaming

tabletmanager: only report healthy if a health check has succeeded re…
Using const() defines constants together (part:2)
Using const() defines constants together (part:1)
Signed-off-by: SataQiu <qiushida@beyondcent.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
…o-misleading-logs

properly suppress begin...commit in autocommit logs
query routing: multi-table plan options
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Also make SelectNext more type safe

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Mark Solters <msolters@gmail.com>
Signed-off-by: Mark Solters <msolters@gmail.com>
Copy link

@guidoiaquinti guidoiaquinti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

@rafael rafael left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@setassociative setassociative merged commit 3489b15 into master Apr 26, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.