Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lock contention problem when creating too many operators #7837

Closed
rleungx opened this issue Feb 22, 2024 · 0 comments · Fixed by #7836
Closed

Lock contention problem when creating too many operators #7837

rleungx opened this issue Feb 22, 2024 · 0 comments · Fixed by #7836
Labels
type/enhancement The issue or PR belongs to an enhancement.

Comments

@rleungx
Copy link
Member

rleungx commented Feb 22, 2024 •
edited
Loading

Enhancement Task

Consider that we create lots of scatter region operators, some of them are created and the others might be finished and removed. But they all need to lock the operator map. So there might be a lock contention problem due to the count update being O(n^2). See

img_v3_028a_03a9fd45-cc7c-49f3-9e7b-df3be43efc3g

img_v3_0289_5384bb66-ecff-4617-8ae2-694b0884c46g
img_v3_0289_09e0e749-4179-4ad6-ac6a-3704e8c6dbfg
img_v3_0289_9a5c36d9-20ba-4160-b6f5-0b1ac384f1bg
@rleungx rleungx added the type/enhancement The issue or PR belongs to an enhancement. label Feb 22, 2024
@Leavrth Leavrth mentioned this issue Feb 28, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Feb 29, 2024
@Leavrth @HuSharp
scheduler: avoid to updateCounts frequently (#7836)
b207e51 
close #7837

Use O(1) method to update `counts`.

Signed-off-by: Leavrth <jianjun.liao@outlook.com>

Co-authored-by: Hu# <jinhao.hu@pingcap.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

scheduler: avoid to updateCounts frequently Leavrth/pd
1 participant
@rleungx