Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #929 implement a utility class to save large downloads to a file #12292

Open
wants to merge 7 commits into
base: jetty-12.0.x
Choose a base branch
from

Conversation

arsenalzp
Copy link

@arsenalzp arsenalzp commented Sep 19, 2024

Hello,
This PR is trying to implement feature described in the issue #929.

Fixes: #929

Signed-off-by: Oleksandr Krutko <alexander.krutko@gmail.com>
Signed-off-by: Oleksandr Krutko <alexander.krutko@gmail.com>
Signed-off-by: Oleksandr Krutko <alexander.krutko@gmail.com>
Signed-off-by: Oleksandr Krutko <alexander.krutko@gmail.com>
Signed-off-by: Oleksandr Krutko <alexander.krutko@gmail.com>
Signed-off-by: Oleksandr Krutko <alexander.krutko@gmail.com>
@arsenalzp arsenalzp changed the title Issue# 929 save large file Issue #929 implement a utility class to save large downloads to a file Sep 19, 2024
@joakime joakime linked an issue Sep 19, 2024 that may be closed by this pull request
@joakime
Copy link
Contributor

joakime commented Sep 19, 2024

The CI build failed due to javadoc issues.

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.7.0:javadoc-no-fork (default) on project jetty-client: An error has occurred in Javadoc report generation: 
[ERROR] Exit code: 1
[ERROR] /home/jenkins/agent/workspace/jetty.project_PR-12292/jetty-core/jetty-client/src/main/java/org/eclipse/jetty/client/PathResponseListener.java:40: error: reference not found
[ERROR]  * Implementation of {@link Listener} that produces an {@link Path}
[ERROR]                             ^
[ERROR] /home/jenkins/agent/workspace/jetty.project_PR-12292/jetty-core/jetty-client/src/main/java/org/eclipse/jetty/client/PathResponseListener.java:42: error: unknown tag: URL
[ERROR]  * like curl <URL> -o file.bin does.
[ERROR]              ^
[ERROR] /home/jenkins/agent/workspace/jetty.project_PR-12292/jetty-core/jetty-client/src/main/java/org/eclipse/jetty/client/PathResponseListener.java:50: error: unknown tag: Path
[ERROR]  *  CompletableFuture<Path> completable = PathResponseListener.write(request, Path.of("/tmp/file.bin"));
[ERROR]                      ^
[ERROR] /home/jenkins/agent/workspace/jetty.project_PR-12292/jetty-core/jetty-client/src/main/java/org/eclipse/jetty/client/PathResponseListener.java:52: error: unexpected end tag: </p>
[ERROR]  * </p>
[ERROR]    ^
[ERROR] 4 errors

You can see your CI builds here -> https://jenkins.webtide.net/blue/organizations/jenkins/jetty.project/activity?branch=PR-12292

Signed-off-by: Oleksandr Krutko <alexander.krutko@gmail.com>
@arsenalzp
Copy link
Author

The CI build failed due to javadoc issues.

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.7.0:javadoc-no-fork (default) on project jetty-client: An error has occurred in Javadoc report generation: 
[ERROR] Exit code: 1
[ERROR] /home/jenkins/agent/workspace/jetty.project_PR-12292/jetty-core/jetty-client/src/main/java/org/eclipse/jetty/client/PathResponseListener.java:40: error: reference not found
[ERROR]  * Implementation of {@link Listener} that produces an {@link Path}
[ERROR]                             ^
[ERROR] /home/jenkins/agent/workspace/jetty.project_PR-12292/jetty-core/jetty-client/src/main/java/org/eclipse/jetty/client/PathResponseListener.java:42: error: unknown tag: URL
[ERROR]  * like curl <URL> -o file.bin does.
[ERROR]              ^
[ERROR] /home/jenkins/agent/workspace/jetty.project_PR-12292/jetty-core/jetty-client/src/main/java/org/eclipse/jetty/client/PathResponseListener.java:50: error: unknown tag: Path
[ERROR]  *  CompletableFuture<Path> completable = PathResponseListener.write(request, Path.of("/tmp/file.bin"));
[ERROR]                      ^
[ERROR] /home/jenkins/agent/workspace/jetty.project_PR-12292/jetty-core/jetty-client/src/main/java/org/eclipse/jetty/client/PathResponseListener.java:52: error: unexpected end tag: </p>
[ERROR]  * </p>
[ERROR]    ^
[ERROR] 4 errors

You can see your CI builds here -> https://jenkins.webtide.net/blue/organizations/jenkins/jetty.project/activity?branch=PR-12292

Hello,
Thank you for your advice, fixed.

@arsenalzp
Copy link
Author

All tests passed on my laptop with JDK17. Should I also run tests with JDK22?

Copy link
Contributor

@sbordet sbordet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arsenalzp good effort!
Please follow my review, and your implementation should become much simpler and with less code.
Thanks!

* CompletableFuture&gt;Path&gt; completable = PathResponseListener.write(request, Path.of("/tmp/file.bin"));
* </pre>
*/
public class PathResponseListener implements CompleteListener, Response.ContentListener
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make this class implement Response.Listener instead, like BufferingResponseListener, so users have all events available if they want to.
This class should extend CompletableFuture<Path>, so users can get its result once the file is written (or failed).

Comment on lines +91 to +94
if (response.getStatus() != HttpStatus.OK_200)
{
throw new HttpResponseException(String.format("HTTP status code of this response %d", response.getStatus()), response);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this check to onHeaders() so it is done only once.
See examples in BufferingResponseListener.


public static CompletableFuture<Path> write(Request request, Path path)
{
return CompletableFuture.supplyAsync(() ->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method should be implemented in terms of PathResponseListener, not by rewriting the same logic again.

Comment on lines +136 to +154
public Response get(long timeout, TimeUnit unit) throws InterruptedException, TimeoutException, ExecutionException
{
boolean expired = !responseLatch.await(timeout, unit);
if (expired && this.response == null)
throw new TimeoutException();
try (AutoLock ignored = lock.lock())
{
// If the request failed there is no response.
if (response == null)
throw new ExecutionException(failure);
return response;
}
}

public Response get() throws InterruptedException, ExecutionException
{
this.completable.get();
return this.response;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove these methods, as this class should be a CompletableFuture.

@sbordet
Copy link
Contributor

sbordet commented Sep 20, 2024

All tests passed on my laptop with JDK17. Should I also run tests with JDK22?

Our CI environment will do it, but sure if you want to try, give it a go.

@sbordet sbordet self-requested a review September 20, 2024 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 👀 In review
Development

Successfully merging this pull request may close these issues.

Implement a utility class to save large downloads to a file
3 participants