Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

statistics: add upper bound of retry for sync load #52658

Merged
merged 13 commits into from
Apr 22, 2024
Merged

statistics: add upper bound of retry for sync load #52658

merged 13 commits into from
Apr 22, 2024

Conversation

hawkingrei
Copy link
Member

@hawkingrei hawkingrei commented Apr 17, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: close #52657

Problem Summary:

What changed and how does it work?

We find if the sync load task fails to run, it will retry forever.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked labels Apr 17, 2024
@codecov Codecov
Copy link

codecov bot commented Apr 17, 2024 •
edited
Loading

Codecov Report

Merging #52658 (30fa279) into master (a9e3398) will increase coverage by 1.9236%.
Report is 10 commits behind head on master.
The diff coverage is 88.2352%.

Additional details and impacted files 
@@               Coverage Diff                @@
##             master     #52658        +/-   ##
================================================
+ Coverage   72.3821%   74.3057%   +1.9236%     
================================================
  Files          1471       1473         +2     
  Lines        427187     428516      +1329     
================================================
+ Hits         309207     318412      +9205     
+ Misses        98761      90238      -8523     
- Partials      19219      19866       +647
Flag Coverage Δ
integration 48.7894% <47.0588%> (?)
unit 71.1762% <88.2352%> (-0.0983%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 53.9957% <ø> (ø)
parser ∅ <ø> (∅)
br 49.4332% <ø> (+8.3204%) ⬆️

@hawkingrei
Copy link
Member Author

/retest

1 similar comment
@hawkingrei
Copy link
Member Author

/retest

@ti-chi-bot ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Apr 17, 2024
@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 17, 2024
@hawkingrei
Copy link
Member Author

/retest

@hawkingrei
statistics: add upper bound of retry for sync load
ba41bc9 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
@hawkingrei
statistics: add upper bound of retry for sync load
6d906cd 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
@hawkingrei
update
551c02c 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
@hawkingrei
update
72fa39d 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
@hawkingrei
update
2f3c90c 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
hawkingrei
hawkingrei commented Apr 18, 2024
View reviewed changes
pkg/statistics/handle/syncload/stats_syncload.go
return task.Retry <= 3
}

func (s *statsSyncLoad) handleOneItemTask(sctx sessionctx.Context, task *statstypes.NeededItemTask) (err error) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the clean code. the origin's result *stmtctx.StatsLoadResult is unnecessary.

@hawkingrei
update
f2d5def 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
@hawkingrei
Copy link
Member Author

@hi-rustin PTAL

@hawkingrei
update
fee47d8 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
@hawkingrei
update
e9d8b6f 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Apr 22, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hi-rustin, winoros

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Apr 22, 2024
@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Apr 22, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-04-17 09:00:29.237240412 +0000 UTC m=+415048.348286861: ☑️ agreed by winoros.
  • 2024-04-22 10:44:11.21615385 +0000 UTC m=+853270.327200300: ☑️ agreed by hi-rustin.

@ti-chi-bot ti-chi-bot bot merged commit cfcc770 into pingcap:master Apr 22, 2024
12 checks passed
@hawkingrei
Copy link
Member Author

/cherrypick release-8.1
/cherrypick release-7.5
/cherrypick release-7.1
/cherrypick release-6.5

@hawkingrei hawkingrei deleted the avoid_cannot_stop_retry branch April 22, 2024 15:04
@ti-chi-bot
Copy link
Member

@hawkingrei: new pull request created to branch release-6.5: #52816.

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5
/cherrypick release-7.1
/cherrypick release-6.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Apr 22, 2024
@hawkingrei @ti-chi-bot
This is an automated cherry-pick of pingcap#52658
27d7681 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Apr 22, 2024
Merged
13 tasks
@ti-chi-bot
Copy link
Member

@hawkingrei: new pull request created to branch release-7.1: #52817.

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5
/cherrypick release-7.1
/cherrypick release-6.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Apr 22, 2024
@hawkingrei @ti-chi-bot
This is an automated cherry-pick of pingcap#52658
15b2ad8 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Apr 22, 2024
Merged
13 tasks
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Apr 22, 2024
@hawkingrei @ti-chi-bot
This is an automated cherry-pick of pingcap#52658
d3e5ae3 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request Apr 22, 2024
Merged
13 tasks
@ti-chi-bot
Copy link
Member

@hawkingrei: new pull request created to branch release-7.5: #52818.

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5
/cherrypick release-7.1
/cherrypick release-6.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

@hawkingrei: new pull request created to branch release-8.1: #52819.

In response to this:

/cherrypick release-8.1
/cherrypick release-7.5
/cherrypick release-7.1
/cherrypick release-6.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot mentioned this pull request Apr 22, 2024
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this pull request Apr 23, 2024
@ti-chi-bot
statistics: add upper bound of retry for sync load (#52658) (#52819)
cfeca9b 
close #52657
ti-chi-bot bot pushed a commit that referenced this pull request May 16, 2024
@ti-chi-bot
statistics: add upper bound of retry for sync load (#52658) (#52818)
d9788b9 
close #52657
hawkingrei added a commit to ti-chi-bot/tidb that referenced this pull request Jun 5, 2024
@hawkingrei
This is an automated cherry-pick of pingcap#52658
b9ef08c 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
ti-chi-bot bot pushed a commit that referenced this pull request Jun 5, 2024
@ti-chi-bot
statistics: add upper bound of retry for sync load (#52658) (#52816)
55bccae 
close #52657
hawkingrei pushed a commit to hawkingrei/tidb that referenced this pull request Jul 1, 2024
@ti-chi-bot @hawkingrei
statistics: add upper bound of retry for sync load (pingcap#52658) (p…
30911a2 
…ingcap#52816)

close pingcap#52657
hawkingrei added a commit to ti-chi-bot/tidb that referenced this pull request Aug 28, 2024
@hawkingrei
This is an automated cherry-pick of pingcap#52658
551e603 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
ti-chi-bot bot pushed a commit that referenced this pull request Aug 28, 2024
@ti-chi-bot
statistics: add upper bound of retry for sync load (#52658) (#52817)
3764806 
close #52657
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Rustin170506 Rustin170506 Rustin170506 approved these changes

@winoros winoros winoros approved these changes

@elsa0520 elsa0520 Awaiting requested review from elsa0520

@AilinKid AilinKid Awaiting requested review from AilinKid

Assignees
No one assigned
Labels
approved lgtm release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

sync load cannot stop to retry when to load empty stats
4 participants
@hawkingrei @ti-chi-bot @winoros @Rustin170506