-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
fix(aliyundrive_open): limit rate for every request (close #724) #1011
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…limit for `List` and `Link` (close #724)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: 火星大王 <34576789+huoxingdawang@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: 火星大王 <34576789+huoxingdawang@users.noreply.github.com>
|
感谢,所以实际上是其他所有的API共享15QPS的限流? 我再下去修一轮,把其他所有API共享限流的逻辑给写了再合并还是怎么说? 是直接向那个仓库交PR,还是考虑撤掉提交 7e9cdd8 中的修改重新把 亦或者我看 |
|
另外我注意到 |
|
另外我测试的时候还注意到,这个限流在阿里云盘那边应该是针对同一个账户共享请求频率限制,即便token或者挂载的rootid不一样,只要是同一个账户就会有相同的配额。 我实际测试下来如果用多个driver挂载了同一用户的多个不同的路径,这些路径之间会相互触发limit。。。。 如果要改的话,应该需要启动的时候从 |
Remove and MakeDir; reduce limit for List and Link (close #724) |
我根据文档,从 我注意到所有的请求都经过
|
这个也改造一下吧,然后 go mod 里面就可以删了,辛苦啦! |
彳亍口巴 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements rate limiting for Aliyun Drive Open API requests to address TooManyRequest errors mentioned in issue #724. The implementation reduces API call rates from the documented limits to provide a safety margin and adds rate limiting to previously unprotected operations.
- Replaces the
rateglibrary with a custom rate limiting solution usinggolang.org/x/time/rate - Implements per-user rate limiting with global fallback for initial requests
- Adds rate limiting to Remove and MakeDir operations that were experiencing TooManyRequest errors
Reviewed Changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| go.mod | Removes rateg dependency and moves gopsutil from indirect to direct dependency |
| drivers/aliyundrive_share/limiter.go | Adds new rate limiter implementation for share driver with fixed rate limits |
| drivers/aliyundrive_share/util.go | Updates methods to accept context and use new rate limiting |
| drivers/aliyundrive_share/driver.go | Integrates new rate limiter and removes rateg usage |
| drivers/aliyundrive_open/limiter.go | Adds sophisticated per-user rate limiter with cleanup mechanism |
| drivers/aliyundrive_open/util.go | Updates request methods to use new rate limiting system |
| drivers/aliyundrive_open/upload.go | Adds rate limiting to upload-related operations |
| drivers/aliyundrive_open/driver.go | Integrates per-user rate limiting and removes rateg dependencies |
已处理 |
根据 #724 中的现象,我推测阿里云盘的上游对API调用都有限速。但是现有的限速给的太极限了,导致出现很多几毫秒几十毫秒的TooManyRequest,我适当降低了
List和Link两个API的限速。同时我注意到
Remove和MakeDir也会出现TooManyRequest,大概都在3300ms左右。我给这两个API也进行了限速。现在看起来工作的比较稳定,但是很慢,一直在限速。。。。