*: reset infoschema v2 data before a full load #54831

tiancaiamao · 2024-07-23T07:25:09Z

What problem does this PR solve?

Issue Number: close #54796

Problem Summary:

What changed and how does it work?

As the comment says:

https://github.com/pingcap/tidb/compare/master...tiancaiamao:issue54796?expand=1#diff-e33e11641dcb529d51aab53c02179aa98f1261cf704d4dfb6189e96489b4f2caR869-R887

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

tiprow · 2024-07-23T07:25:31Z

Hi @tiancaiamao. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

tiancaiamao · 2024-07-23T07:26:44Z

pkg/infoschema/infoschema_v2.go

@@ -250,6 +250,121 @@ func (isd *Data) deleteDB(dbInfo *model.DBInfo, schemaVersion int64) {
 	isd.schemaID2Name.Set(schemaIDName{schemaVersion: schemaVersion, id: dbInfo.ID, name: dbInfo.Name.O, tomb: true})
 }

+// resetBeforeFullLoad is called before a full recreate operation within builder.InitWithDBInfos().
+// TODO: write a generics version to avoid repeated code.


In fact, I tried to do that and failed, just wast triple time than the non-generic version.

codecov · 2024-07-23T08:09:53Z

Codecov Report

Attention: Patch coverage is 98.50746% with 3 lines in your changes missing coverage. Please review.

Project coverage is 56.7046%. Comparing base (d6ee4b8) to head (8282893).
Report is 5 commits behind head on master.

Additional details and impacted files

@@                Coverage Diff                @@
##             master     #54831         +/-   ##
=================================================
- Coverage   74.8037%   56.7046%   -18.0991%     
=================================================
  Files          1557       1679        +122     
  Lines        363606     612424     +248818     
=================================================
+ Hits         271991     347273      +75282     
- Misses        71916     241441     +169525     
- Partials      19699      23710       +4011

Flag	Coverage Δ
integration	`38.5345% <28.8557%> (?)`
unit	`72.0074% <98.5074%> (-1.7121%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`52.9656% <ø> (-2.2339%)`	⬇️
parser	`∅ <ø> (∅)`
br	`62.0019% <ø> (+14.3076%)`	⬆️

tiancaiamao · 2024-07-23T11:32:14Z

/retest

#54846

tiprow · 2024-07-23T11:32:36Z

@tiancaiamao: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

#54846

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

tiancaiamao · 2024-07-23T13:45:07Z

/retest

tiprow · 2024-07-23T13:45:29Z

@tiancaiamao: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

pkg/domain/domain.go

Benjamin2037 · 2024-07-24T05:12:50Z

Will the reset operations cause performance problems when there are 1M tables in the cluster?

pkg/infoschema/infoschema_v2.go

Benjamin2037

rest LGTM

ti-chi-bot · 2024-07-24T05:24:57Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Benjamin2037, wjhuang2016

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~pkg/ddl/OWNERS~~ [Benjamin2037,wjhuang2016]
~~pkg/domain/OWNERS~~ [wjhuang2016]
~~pkg/infoschema/OWNERS~~ [Benjamin2037,wjhuang2016]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2024-07-24T05:25:00Z

[LGTM Timeline notifier]

Timeline:

2024-07-23 11:08:23.684837651 +0000 UTC m=+956925.675779120: ☑️ agreed by wjhuang2016.
2024-07-24 05:24:59.666923767 +0000 UTC m=+1022721.657865239: ☑️ agreed by Benjamin2037.

Benjamin2037 · 2024-07-24T05:25:23Z

/hold

Benjamin2037 · 2024-07-24T05:25:57Z

I hold because there are some comments need to be reply.

Benjamin2037 · 2024-07-24T05:48:49Z

/unhold

tiancaiamao · 2024-07-24T06:39:10Z

/retest

tiprow · 2024-07-24T06:39:32Z

@tiancaiamao: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

pkg/infoschema/builder.go

tiancaiamao · 2024-07-24T08:05:20Z

/test pull-integration-ddl-test

tiprow · 2024-07-24T08:05:42Z

@tiancaiamao: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test pull-integration-ddl-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

tiancaiamao · 2024-07-24T08:39:09Z

/test check-dev2

tiprow · 2024-07-24T08:39:31Z

@tiancaiamao: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

*: reset infoschema v2 data before a full load

738ae4e

ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/needs-triage-completed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 23, 2024

tiancaiamao commented Jul 23, 2024

View reviewed changes

ti-chi-bot bot removed the do-not-merge/needs-triage-completed label Jul 23, 2024

tiancaiamao requested review from ywqzzy, D3Hunter and wjhuang2016 July 23, 2024 07:28

tiancaiamao added 2 commits July 23, 2024 15:40

try to make the lint check happy

fe8f438

make lint happy

65cc8fd

tiancaiamao added 2 commits July 23, 2024 16:32

Merge branch 'master' into issue54796

245eb94

address comment

22ed60d

ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 23, 2024

tiny optimize

ed223f3

wjhuang2016 approved these changes Jul 23, 2024

View reviewed changes

ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 23, 2024

tiancaiamao added 4 commits July 23, 2024 19:41

Merge branch 'master' into issue54796

38218ab

Merge branch 'master' into issue54796

a463096

ci stable?

5b39f2d

ci stable

d618cfb

Benjamin2037 reviewed Jul 24, 2024

View reviewed changes

pkg/domain/domain.go Show resolved Hide resolved

Benjamin2037 reviewed Jul 24, 2024

View reviewed changes

pkg/infoschema/infoschema_v2.go Show resolved Hide resolved

Benjamin2037 approved these changes Jul 24, 2024

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 24, 2024

ti-chi-bot bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 24, 2024

ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 24, 2024

tiancaiamao mentioned this pull request Jul 24, 2024

flaky test TestSelectHiddenColumn #54856

Closed

D3Hunter reviewed Jul 24, 2024

View reviewed changes

pkg/infoschema/builder.go Outdated Show resolved Hide resolved

Merge branch 'master' into issue54796

8282893

ti-chi-bot bot merged commit 3148c80 into pingcap:master Jul 24, 2024
21 checks passed

tiancaiamao deleted the issue54796 branch July 24, 2024 10:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: reset infoschema v2 data before a full load #54831

*: reset infoschema v2 data before a full load #54831

tiancaiamao commented Jul 23, 2024 •

edited by hawkingrei

Loading

tiprow bot commented Jul 23, 2024

tiancaiamao Jul 23, 2024

codecov bot commented Jul 23, 2024 •

edited

Loading

tiancaiamao commented Jul 23, 2024

tiprow bot commented Jul 23, 2024

tiancaiamao commented Jul 23, 2024

tiprow bot commented Jul 23, 2024

Benjamin2037 commented Jul 24, 2024

Benjamin2037 left a comment

ti-chi-bot bot commented Jul 24, 2024

ti-chi-bot bot commented Jul 24, 2024

Benjamin2037 commented Jul 24, 2024

Benjamin2037 commented Jul 24, 2024

Benjamin2037 commented Jul 24, 2024

tiancaiamao commented Jul 24, 2024

tiprow bot commented Jul 24, 2024

tiancaiamao commented Jul 24, 2024

tiprow bot commented Jul 24, 2024

tiancaiamao commented Jul 24, 2024

tiprow bot commented Jul 24, 2024

*: reset infoschema v2 data before a full load #54831

*: reset infoschema v2 data before a full load #54831

Conversation

tiancaiamao commented Jul 23, 2024 • edited by hawkingrei Loading

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

tiprow bot commented Jul 23, 2024

tiancaiamao Jul 23, 2024

Choose a reason for hiding this comment

codecov bot commented Jul 23, 2024 • edited Loading

Codecov Report

tiancaiamao commented Jul 23, 2024

tiprow bot commented Jul 23, 2024

tiancaiamao commented Jul 23, 2024

tiprow bot commented Jul 23, 2024

Benjamin2037 commented Jul 24, 2024

Benjamin2037 left a comment

Choose a reason for hiding this comment

ti-chi-bot bot commented Jul 24, 2024

ti-chi-bot bot commented Jul 24, 2024

[LGTM Timeline notifier]

Benjamin2037 commented Jul 24, 2024

Benjamin2037 commented Jul 24, 2024

Benjamin2037 commented Jul 24, 2024

tiancaiamao commented Jul 24, 2024

tiprow bot commented Jul 24, 2024

tiancaiamao commented Jul 24, 2024

tiprow bot commented Jul 24, 2024

tiancaiamao commented Jul 24, 2024

tiprow bot commented Jul 24, 2024

tiancaiamao commented Jul 23, 2024 •

edited by hawkingrei

Loading

codecov bot commented Jul 23, 2024 •

edited

Loading