Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resource_control: bypass some internal urgent request #884

Merged
merged 6 commits into from
Jul 14, 2023

Conversation

nolouch
Copy link
Contributor

@nolouch nolouch commented Jul 12, 2023

Background

In tidb, the source request will be tiny but urgent. we can consider skip it by resource control

const (
	// InternalTxnOthers is the type of requests that consume low resources.
	// This reduces the size of metrics.
	InternalTxnOthers = util.InternalTxnOthers
	// InternalTxnBootstrap is the type of TiDB bootstrap txns.
	InternalTxnBootstrap = InternalTxnOthers
	// InternalTxnMeta is the type of the miscellaneous meta usage.
	InternalTxnMeta = util.InternalTxnMeta ==> util.InternalTxnOthers
	// InternalTxnCacheTable is the type of cache table usage.
	InternalTxnCacheTable = InternalTxnOthers
	// InternalTxnBindInfo is the type of bind info txn.
	InternalTxnBindInfo = InternalTxnOthers
	// InternalTxnSysVar is the type of sys var txn.
	InternalTxnSysVar = InternalTxnOthers
	// InternalTxnTelemetry is the type of telemetry.
	InternalTxnTelemetry = InternalTxnOthers
	// InternalTxnPrivilege is the type of privilege txn.
	InternalTxnPrivilege = InternalTxnOthers
)

As the upper show, bootstrap, cached table, bindinfo, sysvar, telemetry, privilege, meta relative internal SQL can be skipped first.
Search code with kv.WithInternalSourceType(context.Background(), kv.InternalTxnBindInfo) can see the origin of the SQL.

Manual Test:

It's based on cse repo. I set the group with low RU:

  {
    "name": "1",
    "mode": 1,
    "r_u_settings": {
      "r_u": {
        "settings": {
          "fill_rate": 0
          "burst_limit": 1
        },
        "state": {
          "last_update": "2023-07-14T01:23:21.99892+08:00",
          "initialized": true
        }
      }
    },
    "priority": 0
  }

Before:

image

After
image

@nolouch nolouch changed the base branch from master to cse-region-client July 12, 2023 11:11
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch nolouch closed this Jul 13, 2023
@nolouch nolouch reopened this Jul 13, 2023
nolouch added 2 commits July 14, 2023 03:33
Signed-off-by: nolouch <nolouch@gmail.com>
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch nolouch changed the title Cse add bypass resource_control: bypass some internal urgent request Jul 14, 2023
@nolouch nolouch marked this pull request as ready for review July 14, 2023 08:10
util/request_source.go Outdated Show resolved Hide resolved
Signed-off-by: nolouch <nolouch@gmail.com>
@zeminzhou
Copy link
Contributor

Can this pr fix Error 8027 (HY000): Information schema is out of date: schema failed to update in 1 lease, please make sure TiDB can connect to TiKV.?

@nolouch
Copy link
Contributor Author

nolouch commented Jul 14, 2023

I think Information schema is out of date should pass the DDL.

@zeminzhou
Copy link
Contributor

If we do it, all DDL will be bypass?

@nolouch
Copy link
Contributor Author

nolouch commented Jul 14, 2023

@zeminzhou I checked it, it's relative to the reload of the schema from tikv, and it will label as util. InternalTxnMeta, it's belongs to InternalTxnOthers, so I think This PR will fix the reported error.

@ystaticy
Copy link
Contributor

How about bypass ddl request in this PR ?

@zeminzhou
Copy link
Contributor

That's great, thanks!

…ightning/br (tikv#868)

Signed-off-by: nolouch <nolouch@gmail.com>
Signed-off-by: nolouch <nolouch@gmail.com>
@nolouch
Copy link
Contributor Author

nolouch commented Jul 14, 2023

commit 7b5da57
is cherry-pick from #868 to make tidb-compatibility pass.

Copy link
Contributor

@zeminzhou zeminzhou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~

@iosmanthus iosmanthus merged commit 19ef437 into tikv:cse-region-client Jul 14, 2023
@nolouch nolouch deleted the cse-add-bypass branch July 14, 2023 16:40
nolouch added a commit to nolouch/client-go that referenced this pull request Aug 11, 2023
nolouch added a commit to nolouch/client-go that referenced this pull request Aug 11, 2023
Signed-off-by: nolouch <nolouch@gmail.com>
iosmanthus pushed a commit that referenced this pull request Aug 11, 2023
* resource_control: bypass some internal urgent request (#884)

Signed-off-by: nolouch <nolouch@gmail.com>

* resourcecontrol: fix nil pointer (#900)

Signed-off-by: nolouch <nolouch@gmail.com>

---------

Signed-off-by: nolouch <nolouch@gmail.com>
iosmanthus added a commit that referenced this pull request Aug 11, 2023
* client-go: add some key range info to error when PD returned no region (#862)

Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>

* *: refine non-global stale-read request retry logic (#863)

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* Fix the issue that primary pessimistic lock may be left not cleared after GC (#866)

* Fix the issue that primary pessimistic lock may be left not cleared after GC

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix mysteriously shown up thing that makes compilation failed

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix test effectiveness (forgot to set txn2 to pessimistic txn); add more strict checks

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Address comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

---------

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* add explicit request source type to label the external request like lightning/br (#868)

Signed-off-by: nolouch <nolouch@gmail.com>

* use '%d' instead of '%q' for some int values in error message (#875)

Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>

* format key in error message in method `scanRegions` (#876)

Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>

* make cop request timeout a config paramter (#865)

* update

Signed-off-by: Spade A <u6748471@anu.edu.au>

* update

Signed-off-by: Spade A <u6748471@anu.edu.au>

* update

Signed-off-by: Spade A <u6748471@anu.edu.au>

* update

Signed-off-by: Spade A <u6748471@anu.edu.au>

---------

Signed-off-by: Spade A <u6748471@anu.edu.au>

* region_cache: support check pending tiflash peer (#821)

Signed-off-by: guo-shaoge <shaoge1994@163.com>
Co-authored-by: disksing <i@disksing.com>

* *: add `SnapshotIterReverse` and make `iterReverse` supports `lowerBound` (#883)

Signed-off-by: Jason Mo <mohangjie1995@gmail.com>

* *: fix stale read ops metric (#878) (#889)

Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: disksing <i@disksing.com>

* add gc options (#828)

Signed-off-by: weedge <weege007@gmail.com>
Co-authored-by: disksing <i@disksing.com>

* reload region cache when store is resolved from invalid status (#843)

Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: disksing <i@disksing.com>

* ci: update setup-go action (#904)

Signed-off-by: disksing <i@disksing.com>

* fix unexpected slow query during GC running after stop 1 tikv-server (#899) (#909)

* fix unexpected slow query during GC running after stop 1 tikv-server

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* resource_manager: ignore ru metrics for background request (#872)

Signed-off-by: husharp <jinhao.hu@pingcap.com>
Co-authored-by: disksing <i@disksing.com>

* add more log for diagnose (#915)

* add more log for diagnose

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more log for diagnose

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more log

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* use context logger as much as possible (#908)

* use context logger as much as possible

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* Resume max retry time check for stale read retry with leader option(#903) (#911)

* Resume max retry time check for stale read retry with leader option

Signed-off-by: cfzjywxk <lsswxrxr@163.com>

* add cancel

Signed-off-by: cfzjywxk <lsswxrxr@163.com>

---------

Signed-off-by: cfzjywxk <lsswxrxr@163.com>

* request_source: remove default label (#890)

* request_source: remove default label

Signed-off-by: nolouch <nolouch@gmail.com>

* add a function to set request source task type (#925)

* add a function to set request source task type

Signed-off-by: glorv <glorvs@163.com>

* ci: update go version (#936)

* ci: update go version

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* use tidb_kv_read_timeout as first kv request timeout (#919)

* support tidb_kv_read_timeout as first round kv request timeout

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix ci

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix ci

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix ci

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix ci

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix ci

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* update comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* [pick] resource_control: bypass some internal urgent request (#938)

* resource_control: bypass some internal urgent request (#884)

Signed-off-by: nolouch <nolouch@gmail.com>

* resourcecontrol: fix nil pointer (#900)

Signed-off-by: nolouch <nolouch@gmail.com>

---------

Signed-off-by: nolouch <nolouch@gmail.com>

---------

Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Signed-off-by: nolouch <nolouch@gmail.com>
Signed-off-by: Spade A <u6748471@anu.edu.au>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: Jason Mo <mohangjie1995@gmail.com>
Signed-off-by: weedge <weege007@gmail.com>
Signed-off-by: you06 <you1474600@gmail.com>
Signed-off-by: disksing <i@disksing.com>
Signed-off-by: husharp <jinhao.hu@pingcap.com>
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
Signed-off-by: glorv <glorvs@163.com>
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
Co-authored-by: 王超 <cclcwangchao@hotmail.com>
Co-authored-by: crazycs <crazycs520@gmail.com>
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: ShuNing <nolouch@gmail.com>
Co-authored-by: Spade  A <71589810+SpadeA-Tang@users.noreply.github.com>
Co-authored-by: guo-shaoge <shaoge1994@163.com>
Co-authored-by: disksing <i@disksing.com>
Co-authored-by: Hangjie Mo <mohangjie1995@gmail.com>
Co-authored-by: weedge <weege007@gmail.com>
Co-authored-by: you06 <you1474600@gmail.com>
Co-authored-by: Hu# <jinhao.hu@pingcap.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
Co-authored-by: glorv <glorvs@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants