-
Notifications
You must be signed in to change notification settings - Fork 203
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RqMultipart.Base.make() leaves temporary files until the VM terminates #254
Comments
@dmzaytsev totally agree, thanks for catching this |
@dmzaytsev I will find a developer for the task soon... |
@dmzaytsev there is no milestone yet, so I set it to |
@dmzaytsev thanks for the ticket, your account was topped for 15 mins, payment 56894648 |
@Bertram25 this ticket is yours now, please proceed, and keep in mind this. Any technical questions you should ask right here... Total fixed cost of this task is 30 mins (see this for more info) |
@dmzaytsev Hi, :) What kind of method would be the most suitable to you? Proposal: Collect the files in an iterable and add a method to make those cleanable before the VM stops (as you requested), leave the deleteOnExit(); statement in place, and add a finalize() method cleaning the file handles, so that we are sure the temp files are either deleted when the gc triggers the parent request object deletion or at least (and still) when the VM stops. Quick-made Example showing the part permitting to delete the files when the request is deleted by the gc (as said above, we'd still keep deleteOnExit() call to make sure the files get deleted at least when the VM stops.): // In RqMultipart.Base class
// ...
// file handler collection
private final Iterable<File> files;
private void cleanTempFiles() {
for(File file : files) {
// TODO: Check for file existence before deleting it
// if we're going to keep deleteOnExit() in make()
// add try catch
file.delete();
}
}
// Handles the deletion of files when the object is cleaned up.
@Override
protected void finalize() throws Throwable
{
try {
cleanTempFiles();
} catch (Throwable t) {
throw t;
} finally {
super.finalize();
}
} |
@Bertram25 hi,
We don't know when the garbage collector will call |
@dmzaytsev @Bertram25 I think the best place to delete the file is in Request req = // .. incoming request
final Request part = new RqMultipart.Base(req).part("file").iterator().next();
final InputStream body = part.body();
final byte[] content = new byte[1_000_000];
body.read(content);
body.close(); // boom! here we delete the file make sense? |
@yegor256 @Bertram25 Let's try, I agree |
@yegor256 @dmzaytsev Thanks for the hints! Am I doing it the right way? If so, and you agree to give me one or two more days, I'll finish this commit with proper style and an unit test. Best regards, |
@Bertram25 Why do we need I think you should use a decorator for InputStream here this decorator have to use @yegor256 correct me if I wrong |
@Bertram25 I'm with @dmzaytsev on this |
For yegor256#254 I thus added a decorator to InputStream to permit deletion of the temp file when calling close() as advised by @dmzaytsev Thanks to @yegor256 and @dmzaytsev for the hints.
…dies are closed. For yegor256#254 I thus added a decorator to InputStream to permit deletion of the temp file when calling close() as advised by @dmzaytsev Thanks to @yegor256 and @dmzaytsev for the hints.
@yegor256 @dmzaytsev Indeed! Please accept my apologize for my own lack of sleep and coffee. :) Thanks for all the advice. |
@alexey-krylov can you please help? Keep in mind this. If you have any technical questions, don't hesitate to ask right here... The budget of this task is 30 mins. This is exactly how much time will be compensated, when the task is completed. Read about our Definition of Done |
@davvd sorry, i think i have not enough knowledge of 'takes' to fix that in this time. Find someone else please. |
@davvd let me finish it. |
@alexey-krylov deducted 30 from your rating :( |
@alexey-krylov all right, we'll find someone else for this task |
@exper0 the task is yours please proceed |
@dmzaytsev PR #536 merged and closed. Please close request |
@exper0 thanks! |
@yegor256 could you make a new release ? thanks |
@dmzaytsev just in case - in scope of this task the problem with temporary files / file descriptors is not fully solved. There are a couple of puzzles created in scope of this task. Briefly - in order to clean the file and close actual file descriptor |
@dmzaytsev 2 puzzles were created here: |
@ypshenychka please, let us know what do you think about this ticket, according to our QA rules |
@davvd Quality is good. |
@ypshenychka thanks a lot ;) |
@exper0 added 10 mins to @ypshenychka, for QA review, payment code is |
@dmzaytsev the last puzzle |
RqMultipart.Base.make() uses deleteOnExit() option for temporary files. It means such files be deleted when the virtual machine terminates. We will get a large number of temporary files with heavy use applications.
I believe we need to use another method for deletion these temporary files. Since temp files in this method uses as buffer for a large requests we could try collect names and force to delete the files every some time interval
- ~~`254-d68bc52e`/#575~~ (by Armin Braun) - `254-40a8e0fe`/#576 (by Nicolas FILOTTO)
The text was updated successfully, but these errors were encountered: