tune default setting: async_pollers_per_cq #4359

bestwoody · 2022-03-21T07:02:44Z

Signed-off-by: bestwoody bestwoody@163.com

What problem does this PR solve?

Issue Number: close #xxx

Problem Summary:

What is changed and how it works?

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

None

Signed-off-by: bestwoody <bestwoody@163.com>

ti-chi-bot · 2022-03-21T07:02:45Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

fuzhe1989
windtalker

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

windtalker · 2022-03-21T09:09:45Z

dbms/src/Interpreters/Settings.h

@@ -356,7 +356,7 @@ struct Settings
    M(SettingBool, enable_async_grpc_client, true, "Enable async grpc in MPP.")                                                                                                                                                                \
    M(SettingUInt64, grpc_completion_queue_pool_size, 0, "The size of gRPC completion queue pool. 0 means using hardware_concurrency.")\
    M(SettingBool, enable_async_server, true, "Enable async rpc server.")                                                                                                                                                               \
-    M(SettingUInt64, async_pollers_per_cq, 100, "grpc async pollers per cqs")                                                                                                                                                           \
+    M(SettingUInt64, async_pollers_per_cq, 200, "grpc async pollers per cqs")                                                                                                                                                           \


Does this mean TiFlash will use 400 threads for async poll? Maybe we should set it according to the cpu cores?

windtalker

LGTM

bestwoody · 2022-03-21T10:06:58Z

/merge

ti-chi-bot · 2022-03-21T10:06:59Z

@bestwoody: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2022-03-21T10:07:05Z

This pull request has been accepted and is ready to merge.

Commit hash: b1ba774

bestwoody · 2022-03-21T10:09:25Z

/merge

ti-chi-bot · 2022-03-21T10:09:26Z

@bestwoody: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

sre-bot · 2022-03-21T10:39:21Z

Coverage for changed files

Filename                                                                                        Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/home/jenkins/agent/workspace/tiflash-build-common/tiflash/dbms/src/Interpreters/Settings.h           1                 0   100.00%           1                 0   100.00%           1                 0   100.00%           0                 0         -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                 1                 0   100.00%           1                 0   100.00%           1                 0   100.00%           0                 0         -

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
17027      9572             43.78%    191821  97034        49.41%

full coverage report (for internal network access only)

ti-chi-bot · 2022-03-21T11:12:40Z

[FORMAT CHECKER NOTIFICATION]

Notice: To remove the do-not-merge/needs-linked-issue label, please provide the linked issue number on one line in the PR body, for example: Issue Number: close #123 or Issue Number: ref #456.

_{📖 For more info, you can check the "Contribute Code" section in the development guide.}

bestwoody · 2022-03-21T11:22:04Z

/merge

ti-chi-bot · 2022-03-21T11:22:05Z

@bestwoody: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

sre-bot · 2022-03-21T11:49:26Z

Coverage for changed files

Filename                                                                                        Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/home/jenkins/agent/workspace/tiflash-build-common/tiflash/dbms/src/Interpreters/Settings.h           1                 0   100.00%           1                 0   100.00%           1                 0   100.00%           0                 0         -
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                 1                 0   100.00%           1                 0   100.00%           1                 0   100.00%           0                 0         -

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
17031      9576             43.77%    191844  97050        49.41%

full coverage report (for internal network access only)

ti-chi-bot · 2022-03-21T11:51:09Z

In response to a cherrypick label: new pull request created: #4365.

tune async_pollers_per_cq

00696f7

Signed-off-by: bestwoody <bestwoody@163.com>

bestwoody requested review from windtalker, fuzhe1989 and yibin87 March 21, 2022 07:02

ti-chi-bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Mar 21, 2022

fuzhe1989 approved these changes Mar 21, 2022

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 21, 2022

windtalker reviewed Mar 21, 2022

View reviewed changes

windtalker approved these changes Mar 21, 2022

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 21, 2022

Merge branch 'master' into tune_async_pollers_per_cq

b1ba774

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 21, 2022

bestwoody added needs-cherry-pick-release-6.0 Type: Need cherry pick to release-6.0 and removed do-not-merge/needs-linked-issue labels Mar 21, 2022

Merge branch 'master' into tune_async_pollers_per_cq

afcf1cf

ti-chi-bot added the do-not-merge/needs-linked-issue label Mar 21, 2022

bestwoody removed the do-not-merge/needs-linked-issue label Mar 21, 2022

Merge branch 'master' into tune_async_pollers_per_cq

7a40bfe

ti-chi-bot added the do-not-merge/needs-linked-issue label Mar 21, 2022

bestwoody removed the do-not-merge/needs-linked-issue label Mar 21, 2022

ti-chi-bot merged commit 3566265 into pingcap:master Mar 21, 2022

ti-chi-bot mentioned this pull request Mar 21, 2022

tune default setting: async_pollers_per_cq (#4359) #4365

Merged

12 tasks

ti-chi-bot added a commit that referenced this pull request Mar 22, 2022

tune default setting: async_pollers_per_cq (#4359) (#4365)

f2b0aa7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tune default setting: async_pollers_per_cq #4359

tune default setting: async_pollers_per_cq #4359

bestwoody commented Mar 21, 2022

ti-chi-bot commented Mar 21, 2022 •

edited

Loading

windtalker Mar 21, 2022

windtalker left a comment

bestwoody commented Mar 21, 2022

ti-chi-bot commented Mar 21, 2022

ti-chi-bot commented Mar 21, 2022

bestwoody commented Mar 21, 2022

ti-chi-bot commented Mar 21, 2022

sre-bot commented Mar 21, 2022

ti-chi-bot commented Mar 21, 2022

bestwoody commented Mar 21, 2022

ti-chi-bot commented Mar 21, 2022

sre-bot commented Mar 21, 2022

ti-chi-bot commented Mar 21, 2022

tune default setting: async_pollers_per_cq #4359

tune default setting: async_pollers_per_cq #4359

Conversation

bestwoody commented Mar 21, 2022

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Mar 21, 2022 • edited Loading

windtalker Mar 21, 2022

Choose a reason for hiding this comment

windtalker left a comment

Choose a reason for hiding this comment

bestwoody commented Mar 21, 2022

ti-chi-bot commented Mar 21, 2022

ti-chi-bot commented Mar 21, 2022

bestwoody commented Mar 21, 2022

ti-chi-bot commented Mar 21, 2022

sre-bot commented Mar 21, 2022

Coverage for changed files

Coverage summary

ti-chi-bot commented Mar 21, 2022

bestwoody commented Mar 21, 2022

ti-chi-bot commented Mar 21, 2022

sre-bot commented Mar 21, 2022

Coverage for changed files

Coverage summary

ti-chi-bot commented Mar 21, 2022

ti-chi-bot commented Mar 21, 2022 •

edited

Loading