Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is my proposed fix to https://github.com/crowdmob/goamz/issues/247
Basically connections are not being reused between requests. This causes a few problems:
These issue come up in my use case where I would like to download S3 objects at 200+ qps. I run out of local tcp sockets to use for new connections since they are all in the TIME_WAIT state. This fix solved my problems.
This PR adds an ReuseHTTPClient parameter to the S3 struct and the http.Client will be created on first request. The main issue is that changes to the ConnectTimeout and ReadTimeout will have not effect if made after the first request. (maybe we should check if they've been changed on each request?)
I understand that people have tried to solve this problem before and would be interested in their insights as I'm not an expert on TCP connection reuse and weird states it can find itself in.