api: add HTTP rate limit middleware and config API #4667

CabinfeverB · 2022-02-22T08:39:22Z

Signed-off-by: Cabinfever_B cabinfeveroier@gmail.com

What problem does this PR solve?

Issue Number: Close #4666
This PR is used to support the HTTP rate limit
This PR should be merged after #4501 #4683

What is changed and how it works?

Add rate limit middleware for HTTP API.
Define Ratelimiter interface.

Add rate limit middleware for HTTP API.

Check List

Tests

Unit test
Integration test

Code changes

Has HTTP API interfaces change (Don't forget to add the declarative for API)

Side effects

Possible performance regression

Release note

Add rate limit middleware for HTTP API.

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

ti-chi-bot · 2022-02-22T08:39:23Z

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

codecov · 2022-02-23T08:25:23Z

Codecov Report

Merging #4667 (79f53cf) into master (085a40e) will decrease coverage by 0.01%.
The diff coverage is 92.10%.

@@            Coverage Diff             @@
##           master    #4667      +/-   ##
==========================================
- Coverage   75.37%   75.35%   -0.02%     
==========================================
  Files         295      295              
  Lines       28780    28944     +164     
==========================================
+ Hits        21693    21811     +118     
- Misses       5174     5208      +34     
- Partials     1913     1925      +12

Flag	Coverage Δ
unittests	`75.35% <92.10%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/apiutil/apiutil.go	`70.29% <57.14%> (-2.12%)`	⬇️
server/server.go	`75.96% <88.88%> (+0.50%)`	⬆️
server/api/admin.go	`73.80% <92.75%> (+22.93%)`	⬆️
pkg/ratelimit/option.go	`91.11% <92.85%> (+6.49%)`	⬆️
pkg/ratelimit/limiter.go	`100.00% <100.00%> (ø)`
server/api/config.go	`64.58% <100.00%> (+0.75%)`	⬆️
server/api/middleware.go	`95.08% <100.00%> (+8.54%)`	⬆️
server/api/router.go	`97.86% <100.00%> (-1.72%)`	⬇️
server/api/util.go	`50.00% <100.00%> (+0.76%)`	⬆️
server/config/config.go	`78.31% <100.00%> (+0.18%)`	⬆️
... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 085a40e...79f53cf. Read the comment docs.

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

server/server.go

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

nolouch · 2022-04-18T06:30:01Z

Can it be closed?

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

rleungx · 2022-04-24T04:13:48Z

pkg/ratelimit/limiter.go

@@ -30,7 +49,10 @@ type Limiter struct {

 // NewLimiter returns a global limiter which can be updated in the later.
 func NewLimiter() *Limiter {
-	return &Limiter{labelAllowList: make(map[string]struct{})}
+	return &Limiter{
+		labelAllowList: make(map[string]struct{}),


Should we regard it as a part of config?

I don't think we need to do that, because putting it into configuration would lose the sense of constraint for some API.

pkg/ratelimit/option.go

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

ti-chi-bot · 2022-04-28T00:59:53Z

@CabinfeverB: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

CabinfeverB · 2022-05-18T10:21:02Z

already split

CabinfeverB added 3 commits December 24, 2021 18:23

close tikv#4500: add QPS rate limiter

8334bbe

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

close tikv#4500: add QPS rate limiter

a1c1ecc

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

add rate limit middleware

9f7b3b5

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

ti-chi-bot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Feb 22, 2022

ti-chi-bot requested review from disksing and rleungx February 22, 2022 08:39

merge master

8357e05

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 22, 2022

CabinfeverB added 9 commits February 23, 2022 13:15

add rate limiter

2ff4873

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

add rate limiter

4654a05

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

merge master

3f13679

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

merge master

7fbb733

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fit new limiter

2a82a71

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fit new limiter

9e4e4ac

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

adapt option

2b607b8

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

adapt option

4bbf80b

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

adapt option

3785601

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 23, 2022

CabinfeverB added 5 commits February 23, 2022 15:48

fix static check

3518655

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fix static check

871789a

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fix static check

66edd0c

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fix static check

d51e031

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fix static check

a6935dd

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

CabinfeverB added 5 commits February 23, 2022 16:50

fix test

721fce0

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fix static check

89ec7a9

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fix static check

58d05e1

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fix static check

d0bf8eb

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fix static check

9ef9b50

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

CabinfeverB added 2 commits March 11, 2022 14:31

merge master

4dce217

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

merge master

e83b0b2

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

ti-chi-bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Mar 11, 2022

merge master

4f79dab

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 18, 2022

CabinfeverB added 6 commits March 21, 2022 13:06

refactor service middleware

20ac5a9

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

refactor service middleware

e5270b5

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

refactor route register

30838f4

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

refactor route register

b3f7185

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

refactor route register

5ca5fa1

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

refactor route register

45ece1e

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

rleungx reviewed Mar 22, 2022

View reviewed changes

server/server.go Outdated Show resolved Hide resolved

merge master

b50aa04

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 25, 2022

CabinfeverB added 2 commits April 19, 2022 12:47

merge master

641a620

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

persist rate limit config

ace4b0d

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 19, 2022

CabinfeverB added 3 commits April 19, 2022 19:02

fix

a087b68

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fix json type

7ab6a1c

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fix config bug

0cd8005

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

rleungx reviewed Apr 24, 2022

View reviewed changes

CabinfeverB added 3 commits April 24, 2022 15:32

fix config bug

5a236cf

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fix config bug

1db5023

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

fix config bug

79f53cf

Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>

ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 28, 2022

CabinfeverB closed this May 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

api: add HTTP rate limit middleware and config API #4667

api: add HTTP rate limit middleware and config API #4667

CabinfeverB commented Feb 22, 2022 •

edited

Loading

ti-chi-bot commented Feb 22, 2022

codecov bot commented Feb 23, 2022 •

edited

Loading

nolouch commented Apr 18, 2022

rleungx Apr 24, 2022

CabinfeverB Apr 24, 2022

ti-chi-bot commented Apr 28, 2022

CabinfeverB commented May 18, 2022

api: add HTTP rate limit middleware and config API #4667

api: add HTTP rate limit middleware and config API #4667

Conversation

CabinfeverB commented Feb 22, 2022 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Feb 22, 2022

codecov bot commented Feb 23, 2022 • edited Loading

Codecov Report

nolouch commented Apr 18, 2022

rleungx Apr 24, 2022

Choose a reason for hiding this comment

CabinfeverB Apr 24, 2022

Choose a reason for hiding this comment

ti-chi-bot commented Apr 28, 2022

CabinfeverB commented May 18, 2022

CabinfeverB commented Feb 22, 2022 •

edited

Loading

codecov bot commented Feb 23, 2022 •

edited

Loading