Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HTTP Protocol reads the whole content when calling close() on the stream #463

Closed
jnioche opened this issue May 8, 2017 · 2 comments
Closed

HTTP Protocol reads the whole content when calling close() on the stream #463

jnioche opened this issue May 8, 2017 · 2 comments
Assignees
@jnioche
Labels
bug core fetcher wontfix

Comments

@jnioche
Copy link
Contributor

jnioche commented May 8, 2017 •
edited
Loading

In case of very large content, we end up pulling the whole thing even though we want to trim it. This can, in turn, lead to timeouts.

Here are some very large URLs to test on

http://get.gamexp.ru/BGO_160224/BGO_20160224-2.bin
http://get.gamexp.ru/BGO_160224/BGO_20160224-4.bin
http://get.gamexp.ru/BGO_160224/BGO_20160224-6.bin
http://get.gamexp.ru/BGO_160224/BGO_20160224-3.bin
http://get.gamexp.ru/BGO_160224/BGO_20160224-1.bin
http://get.gamexp.ru/BGO_160224/BGO_20160224-5.bin

what's worse, CloseableHTTTPClient calls EntityUtils.consume(entity); so it looks like we are trying to close it twice anyway.
@jnioche jnioche self-assigned this May 8, 2017
jnioche added a commit that referenced this issue May 8, 2017
@jnioche
HttpProtocol: no need to close the stream as CloseableHTTPClient does…
7d2e3ac 
… it for us. See #463
@jnioche jnioche mentioned this issue May 15, 2017
Closed
@jnioche
Copy link
Contributor Author

jnioche commented May 30, 2017

OKHTTP can handle that, not HTTP client. Won't fix

@jnioche jnioche closed this as completed May 30, 2017
@jnioche
Copy link
Contributor Author

jnioche commented Jul 6, 2017

See https://issues.apache.org/jira/browse/HTTPCLIENT-1861

@jnioche jnioche mentioned this issue Jul 6, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jnioche jnioche

Labels
bug core fetcher wontfix
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jnioche