Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support batch processing with the GetRegion stream call #8690

Open
4 tasks
JmPotato opened this issue Oct 11, 2024 · 1 comment
Open
4 tasks

Support batch processing with the GetRegion stream call #8690

JmPotato opened this issue Oct 11, 2024 · 1 comment
Labels
type/development The issue belongs to a development tasks

Comments

@JmPotato
Copy link
Member

We have encountered several issues related to the GetRegion call and region cache updates, such as #2843 and #8426. At present, a highly feasible solution to alleviate this type of problem is to change GetRegion from a unary gRPC call to a stream call that supports batch processing, which should reduce the CPU pressure on PD in scenarios such as cache expiration and error retries strom.

  • Implement the gRPC stream version of GetRegion which supports handling various queries, such as GetPrevRegion and GetRegionByID.
  • Implement the batch processing logic within the PD client.
  • Integrate the new GetRegion method into the client-go/TiDB.
  • Conduct the benchmark tests with the previous version to evaluate the QPS/CPU usage performance.
@JmPotato JmPotato added the type/development The issue belongs to a development tasks label Oct 11, 2024
@JmPotato
Copy link
Member Author

/cc @JackL9u

This was referenced Oct 15, 2024
Draft
Draft
@JmPotato JmPotato mentioned this issue Nov 8, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Nov 11, 2024
@JmPotato
client: genericize the batch controller (#8793)
1265c8c 
ref #8690

Genericize the batch controller for the later reuse.

Signed-off-by: JmPotato <ghzpotato@gmail.com>
@JmPotato JmPotato mentioned this issue Nov 11, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Nov 12, 2024
@JmPotato
*: aggregate the utils package within the client (#8799)
2637331 
ref #8690

Aggregate the utils package within the client.

Signed-off-by: JmPotato <ghzpotato@gmail.com>
@JmPotato JmPotato pinned this issue Nov 12, 2024
This was referenced Nov 12, 2024
Closed
Merged
ti-chi-bot bot pushed a commit that referenced this issue Nov 14, 2024
@JmPotato
*: independent the client option (#8813)
71745a1 
ref #8690

Independent the client option to reduce the number of files under the client root directory.

Signed-off-by: JmPotato <ghzpotato@gmail.com>
@JmPotato JmPotato mentioned this issue Nov 18, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Nov 19, 2024
@JmPotato
client: split the meta storage client (#8822)
36cc3f4 
ref #8690

Split the meta storage client from the client definitions.

Signed-off-by: JmPotato <ghzpotato@gmail.com>
This was referenced Nov 19, 2024
Merged
Merged
ti-chi-bot bot added a commit that referenced this issue Nov 22, 2024
@JmPotato @ti-chi-bot
client: separate the metrics package (#8833)
20c4157 
ref #8690

Separate the client metrics package.

Signed-off-by: JmPotato <ghzpotato@gmail.com>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot pushed a commit that referenced this issue Nov 25, 2024
@JmPotato
*: independent the service discovery package (#8825)
ec77762 
ref #8690

Make the service discovery module of the client a separate package.

Signed-off-by: JmPotato <ghzpotato@gmail.com>
@JmPotato JmPotato mentioned this issue Nov 25, 2024
Merged
ti-chi-bot bot added a commit that referenced this issue Nov 26, 2024
@JmPotato @ti-chi-bot
client: separate the TSO client implementation (#8848)
176ab23 
ref #8690

Separate the TSO client implementation.

Signed-off-by: JmPotato <ghzpotato@gmail.com>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
@JmPotato JmPotato mentioned this issue Nov 26, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Nov 26, 2024
@JmPotato
*: move some common libraries to the client/pkg directory (#8858)
da0000a 
ref #8690

Move some common libraries to the `client/pkg` directory.

Signed-off-by: JmPotato <ghzpotato@gmail.com>
@JmPotato JmPotato mentioned this issue Dec 5, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Dec 9, 2024
@JmPotato
client: introduce an independent region client interface (#8874)
536ae49 
ref #8690

Introduce an independent region client interface.

Signed-off-by: JmPotato <ghzpotato@gmail.com>
@JmPotato JmPotato mentioned this issue Dec 11, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Dec 11, 2024
@JmPotato
client: reduce redundant concepts in the client filenames (#8894)
fbfcdb8 
ref #8690

Reduce the redundant concepts in the client filenames.

Signed-off-by: JmPotato <ghzpotato@gmail.com>
@JmPotato JmPotato mentioned this issue Dec 11, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Dec 17, 2024
@JmPotato
client/batch: allow tokenCh of batch controller to be nil (#8903)
e8bef5f 
ref #8690

Allow `tokenCh` of batch controller be nil.

Signed-off-by: JmPotato <ghzpotato@gmail.com>
@rleungx rleungx mentioned this issue Dec 18, 2024
Merged
ti-chi-bot bot added a commit that referenced this issue Dec 18, 2024
@rleungx @ti-chi-bot
client: move circuit breaker to pkg (#8932)
ecb31de 
ref #8678, ref #8690

Signed-off-by: Ryan Leung <rleungx@gmail.com>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
This was referenced Dec 19, 2024
Draft
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/development The issue belongs to a development tasks
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@JmPotato