Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

estimate the memory consuming spike for the coprocessor #21427

Open
Yisaer opened this issue Dec 2, 2020 · 0 comments
Open

estimate the memory consuming spike for the coprocessor #21427

Yisaer opened this issue Dec 2, 2020 · 0 comments
Labels
type/feature-request Categorizes issue or PR as related to a new feature.

Comments

@Yisaer
Copy link
Contributor

Yisaer commented Dec 2, 2020

Feature Request

Is your feature request related to a problem? Please describe:

In this request #21168, we use ratelimit to limit the max count of cached response size in the tidb-server for one query. The coprocessor is response for the fetching data from storage and works as a producer/consumer model. There exists multiple workers(as producer) try to fetch data from storage and only one Iterator(as consumer) try to receive data and deliver them to the upon operator.

The initial count of workers is depended on the tidb_distsql_scan_concurrency(default 15) which may cause the memory consuming spike at the beginning for the coprocessor.

Describe the feature you'd like:

Find a way to estimate the memory consuming spike for the coprocessor. For example, we could modify the count of the workers with a proper count at the beginning.

Note that there could exist multiple coprocessor operator for one query, we should also consider and test this case.

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

@Yisaer Yisaer added the type/feature-request Categorizes issue or PR as related to a new feature. label Dec 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature-request Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

No branches or pull requests

1 participant