Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(dm): enable new charset for TiDB in test #3736

Merged
merged 17 commits into from
Dec 8, 2021

Conversation

lance6716
Copy link
Contributor

@lance6716 lance6716 commented Dec 6, 2021

What problem does this PR solve?

enable new charset for TiDB in test, so we can test about charset and collations

also include pingcap/tidb#30173 pingcap/tidb#30292 pingcap/tidb#29291

What is changed and how it works?

  • as title
  • adjust some SQL to use explict COLLATE because the default value of TiDB is not same as MySQL
  • fix a bug that test script didn't return non-zero when error, so comment out lightning tls test
  • disable drop PK test

Check List

Tests

  • No code

Code changes

Side effects

Related changes

Release note

 `None`.

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Dec 6, 2021

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • glorv

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 6, 2021
@lance6716 lance6716 added the area/dm Issues or PRs related to DM. label Dec 6, 2021
@lance6716
Copy link
Contributor Author

/run-dm-integration-test

@codecov-commenter
Copy link

codecov-commenter commented Dec 6, 2021

Codecov Report

Merging #3736 (f0927fe) into master (eed6f9b) will increase coverage by 0.1718%.
The diff coverage is 63.6186%.

Flag Coverage Δ
cdc 58.8386% <74.0298%> (+0.4014%) ⬆️
dm 56.3316% <44.1340%> (-0.0437%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master      #3736        +/-   ##
================================================
+ Coverage   57.3452%   57.5170%   +0.1718%     
================================================
  Files           460        461         +1     
  Lines         54927      55321       +394     
================================================
+ Hits          31498      31819       +321     
- Misses        20180      20239        +59     
- Partials       3249       3263        +14     

@lance6716
Copy link
Contributor Author

/run-dm-integration-test

@lance6716
Copy link
Contributor Author

/run-dm-integration-test

@ti-chi-bot ti-chi-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 6, 2021
@lance6716
Copy link
Contributor Author

/run-dm-integration-test

@lance6716 lance6716 changed the title test(dm): enable new charset for TiDB in test [WIP]test(dm): enable new charset for TiDB in test Dec 6, 2021
@ti-chi-bot ti-chi-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 6, 2021
@lance6716
Copy link
Contributor Author

/run-dm-integration-test

1 similar comment
@lance6716
Copy link
Contributor Author

/run-dm-integration-test

@lance6716
Copy link
Contributor Author

/run-dm-integration-test

@lance6716
Copy link
Contributor Author

/run-dm-integration-test

@lance6716
Copy link
Contributor Author

/run-dm-integration-test

1 similar comment
@lance6716
Copy link
Contributor Author

/run-dm-integration-test

@lance6716
Copy link
Contributor Author

/run-dm-integration-test
/run-dm-integration-test

@lance6716
Copy link
Contributor Author

/run-dm-integration-test

@lance6716 lance6716 changed the title [WIP]test(dm): enable new charset for TiDB in test test(dm): enable new charset for TiDB in test Dec 7, 2021
@ti-chi-bot ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 7, 2021
@lance6716
Copy link
Contributor Author

@lance6716
Copy link
Contributor Author

/run-dm-integration-test

# turn on collation framework https://docs.pingcap.com/tidb/stable/character-set-and-collation#new-framework-for-collations
rm $tmp_config || true
cat >$tmp_config <<EOF
new_collations_enabled_on_first_bootstrap = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not let cases which depends on this flag opens it separately?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change the default test behaviour to avoid we forget collation compatibility in future (such as a new developer add a test and reviews forget checking collation).

echo "check data"
check_sync_diff $WORK_DIR $cur/conf/diff_config.toml
mv "$cur/conf/task-ca.pem.bak" "$cur/conf/task-ca.pem"
# TODO: revert this when lightning support tls
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can comment the whole test test_worker_ha_when_enable_source_tls 😂

Copy link
Contributor Author

@lance6716 lance6716 Dec 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hope you can finish it soon so we'll revert it soon😂

Copy link
Contributor

@D3Hunter D3Hunter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot
Copy link
Member

@D3Hunter: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

LGTM

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@lance6716 lance6716 added the require-LGT1 Indicates that the PR requires an LGTM. label Dec 7, 2021
@niubell
Copy link
Contributor

niubell commented Dec 7, 2021

LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Dec 7, 2021
@lance6716
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: f0927fe

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Dec 7, 2021
@lance6716
Copy link
Contributor Author

/run-integration-test
/run-kafka-integration-test

@ti-chi-bot
Copy link
Member

@lance6716: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@Ehco1996
Copy link
Contributor

Ehco1996 commented Dec 7, 2021

/run-dm-integration-tests

@lance6716
Copy link
Contributor Author

/run-dm-integration-test

@ti-chi-bot ti-chi-bot merged commit 1415a7a into pingcap:master Dec 8, 2021
okJiang pushed a commit to okJiang/tiflow that referenced this pull request Dec 8, 2021
@lance6716 lance6716 deleted the enable-new-charset branch October 13, 2022 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/dm Issues or PRs related to DM. release-note-none Denotes a PR that doesn't merit a release note. require-LGT1 Indicates that the PR requires an LGTM. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT1 Indicates that a PR has LGTM 1.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants