Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Executor: dynamically changing the concurrency of the Scan operator #18448

Open
fzhedu opened this issue Jul 8, 2020 · 2 comments
Open

Executor: dynamically changing the concurrency of the Scan operator #18448

fzhedu opened this issue Jul 8, 2020 · 2 comments
Labels
type/enhancement The issue or PR belongs to an enhancement.

Comments

@fzhedu
Copy link
Contributor

fzhedu commented Jul 8, 2020

Feature Request

Is your feature request related to a problem? Please describe:

#18444, after increasing the concurrency of operators but decreasing the concurrency of Scan, the OOM disappears.

Describe the feature you'd like:

dynamically channging the concurrency of the scan operator, because the default 15 concurrency would buffer so many regions that occupy a lot of memory. This feature would avoid OOM in many cases.

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

@fzhedu fzhedu added the type/feature-request Categorizes issue or PR as related to a new feature. label Jul 8, 2020
@zz-jason zz-jason removed the type/feature-request Categorizes issue or PR as related to a new feature. label Jul 9, 2020
@SunRunAway
Copy link
Contributor

Does #17324 fix this issue?

@fzhedu
Copy link
Contributor Author

fzhedu commented Jul 9, 2020

Does #17324 fix this issue?

no. I planned let the PR fix this issue, but let it as the TODO work. It should not open too many copWorkers at the beginning.

@ghost ghost added the type/enhancement The issue or PR belongs to an enhancement. label Jul 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

No branches or pull requests

3 participants