-
Notifications
You must be signed in to change notification settings - Fork 5.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
statistics: support global singleflight for sync load (#52796) #53340
statistics: support global singleflight for sync load (#52796) #53340
Conversation
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
/retest |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## release-7.5 #53340 +/- ##
================================================
Coverage ? 71.9971%
================================================
Files ? 1411
Lines ? 409847
Branches ? 0
================================================
Hits ? 295078
Misses ? 94911
Partials ? 19858
Flags with carried forward coverage won't be shown. Click here to find out more.
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: hawkingrei, hi-rustin The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
[LGTM Timeline notifier]Timeline:
|
This is an automated cherry-pick of #52796
What problem does this PR solve?
Issue Number: close #52797
Problem Summary:
What changed and how does it work?
before
All items the query needs will be put into the global task channel( the default capacity is 1000). And workers will merge the same task with the singleflight. So performance depends on the number of tasks that can be concurrently executed.
Under poor performance conditions, he will not be able to handle a sufficient number of tasks, resulting in slow processing. But there are still many same tasks in the channel. So they still need to check whether to sync load. And They will fill up this channel. Query can not insert the sync load task into it and load necessary stats.
after
So we need to refactor this code. We need first to let the query pass through singleflight to merge identical tasks. Then insert the task into the channel to make the worker syncload it.
So why is it that during the startup of Wetech, the queries are very unstable and there are severe timeouts with sync load? The problem lies right here.
Check List
Tests
Side effects
Documentation
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.