Use of File.deleteOnExit leaks memory

[Tomas-Kraus]

The method MemoryData.createNext calls File.deleteOnExit. Unfortunately, this permanently adds an entry to a list of files. In our long running server application, the end result is a significant memory leak.
In our case, we don't have much use for this call to deleteOnExit anyway, as our system already cleans up all temporary files when the server stops. So if there is no better way, I would request a simple system property to disable this call.

Affected Versions

[1.9.4]

[Tomas-Kraus]

@glassfishrobot Commented
Reported by basdebakker14

[Tomas-Kraus]

@glassfishrobot Commented
Was assigned to snajper

[Tomas-Kraus]

@glassfishrobot Commented
phraktle2 said:
+1 for addressing this problem.

For long-running server processes, using deleteOnExit is not appropriate, due to the memory leak. It's a better tradeoff to actually leak the files in the rare event of a crash, than it is to constantly leak memory during normal operations.

If your target is Java 7+, you could also open files with the "delete on close" hint:

http://docs.oracle.com/javase/7/docs/api/java/nio/file/StandardOpenOption.html#DELETE_ON_CLOSE

[Tomas-Kraus]

@glassfishrobot Commented
phraktle2 said:
Any plans on fixing this? This affects long running server-apps, resulting in OOM.

[Tomas-Kraus]

@glassfishrobot Commented
hjf1223 said:
removed this line will fixed the issue:

tempFile.deleteOnExit();

[Tomas-Kraus]

@glassfishrobot Commented
hjf1223 said:
kohsuke/mimepull#2

[Tomas-Kraus]

@glassfishrobot Commented
This issue was imported from java.net JIRA MIMEPULL-13

[Tomas-Kraus]

@noppo0831 Commented

Problem

• Calling org.jvnet.mimepull.MemoryData#createNext (DataHead dataHead, ByteBuffer buf) causes memory leak.

environment

Java 7, Java 8, Java 9(build 9+181)

Cause

java.io.File#deleteOnExit called in org.jvnet.mimepull.MemoryData#createNext;

String suffix = config.getTempFileSuffix();
File tempFile = TempFiles.createTempFile (prefix, suffix, config.getTempDir());
// delete the temp file when VM exits as a last resort for file clean up
tempFile.deleteOnExit();
if (LOGGER.isLoggable (Level.FINE)) {
    LOGGER.log (Level.FINE, `Created temp file = {0}`, tempFile);
}

tempFile.deleteOnExit() has the following code

public void deleteOnExit() {
    SecurityManager security = System.getSecurityManager();
    if (security! = null) {
        security.checkDelete(path);
    }
    if (isInvalid()) {
        return;
    }
    DeleteOnExitHook.add(path);
}

java.io.DeleteOnExitHook.add(path); is called as the processing of java.io.File#deleteOnExit

The code of DeleteOnExitHook#add are as follows

static synchronized void add (String file) {
        if (files == null) {
            // DeleteOnExitHook is running. Too late to add a file
            throw new IllegalStateException (`Shutdown in progress`);
        }

        files.add (file);
}

Sting file add to LinkedHashSet<String> files in java.io.DeleteOnExitHook#add, LinkedHashSet<String> files is
It is not initialized until javaVM restarts.

Therefore, memory of the size of String path is consumed when org.jvnet.mimepull.MemoryData#createNext is called.

Solution idea

NO1.

tempfile is deleted when dataHead.dataFile = new DataFile (tempFile); is closed.
Therefore, DeleteOnExitHook will cause memory leak, so it will not be called

Advantages: There are few changes
Disadvantage: A temporary file remains when the VM is shut down before dataHead.dataFile.close is called

NO2.

http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6664633

There is an idea in the above URL.
There is also an idea to extend File and DeleteOnExitHook and remove the target path from DeleteOnExitHook files when File.delete is called.

Advantages: temporary files do not disappear even if VM is shut down before dataHead.dataFile.close is called
Disadvantage: There are many changes in the code

No.3.

Like No2., we extend and add remove method DeleteOnExitHook and call it when delete tmpFile in mimepull. (Perhaps org.jvnet.mimepull.WeakDataFile#close only)
also instead of calling File.deleteOnExit, it calls add of DeleteOnExitHook extension class and adds path to files.

Advantages: There are fewer code changes than No2.
temporary files disappear even if the VM is shut down before dataHead.dataFile.close is called.
Disadvantage: There are more code change parts than No1.

I recommended No3.

[Tomas-Kraus]

@jingtingjian Commented
This might be the cause of https://github.com/jersey/jersey/issues/3259.

[Tomas-Kraus]

• Issue Imported From: Use of File.deleteOnExit leaks memory javaee/metro-mimepull#13
• Original Issue Raised By:@glassfishrobot
• Original Issue Assigned To: @glassfishrobot

[iprem]

@glassfishrobot @Tomas-Kraus Any ETA on when the issue will be fixed?