Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changed FTWRL to produce error #7712

Merged
merged 4 commits into from
Oct 12, 2018
Merged

Changed FTWRL to produce error #7712

merged 4 commits into from
Oct 12, 2018

Conversation

morgo
Copy link
Contributor

@morgo morgo commented Sep 15, 2018

What problem does this PR solve?

Many programs use FLUSH TABLES WITH READ LOCK (ftwrl) to create a synchronization point between multi-threaded programs. For example, mydumper works like this:

Main thread: FLUSH TABLES WITH READ LOCK;
Worker threads: START TRANSACTION WITH CONSISTENT SNAPSHOT;
Main thread: UNLOCK TABLES;

Because FTWRL is parsed but ignored, these programs are not getting a synchronization point, leading to inconsistent backups. The alternatives for these programs are:

  • Use single threaded execution (start transaction is sufficient to provide repeatable-read consistency)
  • Use a tidb_snapshot read. Our fork of mydumper can do this (although it doesn't by default). I have a PR for mydumper upstream.

What is changed and how it works?

This changes FTWRL to be an error, similar to KILL x (not supported, but judged not safe to parse but ignore). An alternative implimentation could be to raise a warning, but error is my first preference.

Check List

Tests

  • Unit test

Code changes

  • Has exported function/method change

Side effects

  • Breaking backward compatibility

Related changes

  • Need to update the documentation
  • Need to be included in the release note

Important! This change is not backwards compatible. It will cause an error in existing mydumpers (both upstream and our fork). However, mydumper also checks for warnings: so lessening the severity will not help.

Assuming the mydumper PR is pulled, we should document that TiDB requires a version of mydumper of X.Y.Z or greater.

@morgo
Copy link
Contributor Author

morgo commented Sep 15, 2018

/run-all-tests

@shenli
Copy link
Member

shenli commented Sep 15, 2018

@morgo This change may lead to a compatible issue. We need to consider this carefully.

@morgo
Copy link
Contributor Author

morgo commented Sep 15, 2018

/run-all-tests tidb-test=pr/626

@coocood
Copy link
Member

coocood commented Sep 17, 2018

@morgo
Do we need to return the error if WITH READ LOCK is not specified?

@morgo
Copy link
Contributor Author

morgo commented Sep 17, 2018

@coocood No. You are correct, it is safe for FLUSH TABLES (no additions) to be a no-op.

@shenli
Copy link
Member

shenli commented Sep 23, 2018

@morgo Should we merge this PR after the PR for mydumper upstream merged?

@shenli
Copy link
Member

shenli commented Sep 23, 2018

LGTM

@shenli shenli added type/compatibility status/LGT1 Indicates that a PR has LGTM 1. labels Sep 23, 2018
@morgo
Copy link
Contributor Author

morgo commented Sep 23, 2018

@shenli Yes, that would be my preference. The release notes can say minimum mydumper version required.

@zhexuany
Copy link
Contributor

zhexuany commented Sep 26, 2018

It seems that this PR is blocked by mydumper/mydumper#155
but LGTM

Copy link
Contributor

@zhexuany zhexuany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhexuany zhexuany added status/LGT2 Indicates that a PR has LGTM 2. status/DNM and removed status/LGT1 Indicates that a PR has LGTM 1. labels Sep 26, 2018
@winoros
Copy link
Member

winoros commented Oct 8, 2018

We need to wait for that pr merged?

@morgo
Copy link
Contributor Author

morgo commented Oct 8, 2018

@winoros I have a backup plan, which is for these two PRs to merge:
pingcap/mydumper#4
pingcap/docs#644

@morgo
Copy link
Contributor Author

morgo commented Oct 11, 2018

This PR is now unblocked and Ready for Merge.

Copy link
Contributor

@winkyao winkyao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@winkyao winkyao merged commit 508a836 into pingcap:master Oct 12, 2018
@morgo morgo deleted the ftwrl-error branch October 12, 2018 02:41
@sre-bot sre-bot added the contribution This PR is from a community contributor. label Dec 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution This PR is from a community contributor. status/LGT2 Indicates that a PR has LGTM 2. type/compatibility
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants