Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sink/mysql: support translated to insert SQL if old value enabled #955

Merged
merged 3 commits into from
Sep 16, 2020

Conversation

amyangfei
Copy link
Contributor

@amyangfei amyangfei commented Sep 16, 2020

What problem does this PR solve?

In MySQL sink, insert is translated to REPLACE SQL, update is translated to DELETE + REPLACE SQL

When old value feature is enabled, we can distinguish whether a SQL is update or insert, so we can translate insert to INSERT SQL and update to UPDATE SQL to increase executing performance.

But in some cases, update and insert is not reentrant, we also need the safe-mode mechanism to force write data.

Besides, the more convenient safe-mode switch will be implemented in another PR.

What is changed and how it works?

  • Introduce a new safe-mode option in MySQL sink
  • The SQL translator behavior, when the old value feature is enabled, is as follows
insert update
in safe mode REPLACE DELETE + REPLACE
not in safe mode INSERT UPDATE

Check List

Tests

  • Unit test
  • Integration test

Release note

  • Support to translate to more execution efficient SQLs in MySQL sink when the old value feature is enabled.

@amyangfei amyangfei added the component/sink Sink component. label Sep 16, 2020
@amyangfei
Copy link
Contributor Author

/run-integration-tests tidb=release-4.0 tikv=release-4.0 pd=release-4.0

@amyangfei
Copy link
Contributor Author

/run-integration-tests tidb=release-4.0 tikv=release-4.0 pd=release-4.0

@overvenus
Copy link
Member

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Sep 16, 2020
@amyangfei
Copy link
Contributor Author

/run-integration-tests tidb=release-4.0 tikv=release-4.0 pd=release-4.0

@amyangfei amyangfei added the status/ptal Could you please take a look? label Sep 16, 2020
@liuzix
Copy link
Contributor

liuzix commented Sep 16, 2020

/lgtm

@ti-srebot ti-srebot removed the status/LGT1 Indicates that a PR has LGTM 1. label Sep 16, 2020
@ti-srebot ti-srebot added the status/LGT2 Indicates that a PR has LGTM 2. label Sep 16, 2020
@amyangfei
Copy link
Contributor Author

/run-integration-tests tidb=release-4.0 tikv=release-4.0 pd=release-4.0

@codecov-commenter
Copy link

Codecov Report

Merging #955 into master will increase coverage by 0.0315%.
The diff coverage is 35.4545%.

@@               Coverage Diff                @@
##             master       #955        +/-   ##
================================================
+ Coverage   33.5889%   33.6204%   +0.0315%     
================================================
  Files           100        100                
  Lines         11998      12073        +75     
================================================
+ Hits           4030       4059        +29     
- Misses         7573       7615        +42     
- Partials        395        399         +4     

@amyangfei amyangfei merged commit cc341fe into pingcap:master Sep 16, 2020
@amyangfei amyangfei deleted the old-value-insert-update branch September 16, 2020 10:00
@amyangfei amyangfei added this to the v4.0.7 milestone Sep 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/sink Sink component. status/LGT2 Indicates that a PR has LGTM 2. status/ptal Could you please take a look?
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants