-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question: Add client.MaxResponseBody*Read*Size? #1724
Comments
A naive modification could be to treat func (resp *Response) ReadBody(r *bufio.Reader, maxBodySize int) (err error) {
bodyBuf := resp.bodyBuffer()
bodyBuf.Reset()
contentLength := resp.Header.ContentLength()
switch {
case contentLength >= 0:
+ if maxBodySize > 0 && contentLength > maxBodySize {
+ contentLength = maxBodySize
+ }
bodyBuf.B, err = readBody(r, contentLength, maxBodySize, bodyBuf.B)
if err == ErrBodyTooLarge && resp.StreamBody {
resp.bodyStream = acquireRequestStream(bodyBuf, r, &resp.Header)
err = nil
}
case contentLength == -1:
... By doing so, I think it could mitigate the issue (maybe not) of fasthttp always reading the whole body into memory. |
Something like that could work but it would still have to return an error. It will leave the connection in a broken state with a partially read body. It will also fail completely if the response uses chunk encoding and the first chunk by itself is already too big. It will always exclude the last chunk as it can't read that without fitting in the max size. And there are probably other edge cases that need to be considered. I'm afraid I don't have time to work on this, a pull request is always welcome. |
Hi, thanks for the great lib.
I'm encountering the same issue as described in issue #686. By setting
Client.MaxResponseBodySize
, it returnsErrBodyTooLarge
when the response body length exceeds this value.However, I would still like to read part of the response body instead of receiving an error, and also prevent potential DOS attacks caused by large responses.
Would it be possible to provide a configuration similar to
client.MaxResponseBody*Read*Size
? This would be similar to what's implemented in retryablehttp-go.Thanks!
The text was updated successfully, but these errors were encountered: