This is a plain and simple leak detector for finding stream (and file descriptor) leaks in a Java application.
NOTE: Please refer j8
branch for an older implementation covering Java 8 etc.
Leaving behind unclosed streams could cause great harm, especially if your application is long-running; e.g. a web or application server that is supported to run indefinitely. It could make you run out of file descriptors, exhaust your file caches (we’re big fans!), and in some cases, even hang your process entirely.
Once we had been constantly fighting with file leaks in our own SaaS AS2 exchange server. And this tool, even in its primitive stage, helped us enormously.
Almost all Java stream classes derive from one of two pairs of base classes:
-
java.io.FileInputStream
andjava.io.FileOutputStream
-
java.io.FilterInputStream
andjava.io.FilterOutputStream
By simply patching these two, we can get a global view of all stream openings and closings in your Java process. Simple.
The most straightforward way - which this project also follows - is to take the source of the core Java classes and insert our custom logic.
In our case we use a timer-driven detector. When the stream is created, it:
-
captures the stacktrace of the current thread, which includes all classes and calls that led to the stream opening
-
starts a timer that would fire after a predefined interval, printing this info into stdout or stderr
When close()
is invoked, we cancel the timer (there’s effectively no leak now).
If close()
wasn’t invoked within the stipulated time, we can suspect that the stream has leaked;
in that case the timer would fire automatically, and dump the stream identity (e.g. filename) and opener stacktrace.
Simple.
We can build a JAR with compiled copies of these classes, drop it into our application, and sit back and relax while the detector does all the magic.
BUT…
This is a bit tricky (and might even be impossible, depending on your Java version).
java.io
is one of the core Java packages, that gets loaded very early - while the JVM is still getting initialized.
Because of Java’s classloading policies,
there’s no way to load our patched class later on and "replace" the originally loaded version.
So we use the --patch-module
mechanism to merge our custom classes into Java’s native java.base
module,
so all streams opened in our JVM would go through our patched logic - and our leak detector.
java <params> --patch-module java.base=path/to/your/JAR <more params> <classname, or -jar {JAR file}>
If the JAR was loaded successfully, you would see this printed to stdout/stderr about 1 second after the process was started:
LeakDumper [FILES,FILTERS]: 2000ms initial grace, 10000ms min-life
If it sees a suspiciously long-living open stream, the detector will dump something like this:
/tmp/prefix1879581817937119981suffix
id: 2 created: 1570066845396
java.io.FileInputStream.<init>(FileInputStream.java:45)
x.y.Z.loadThatFile(Z.java:36)
...
g.h.I.doThat(I.java:15)
d.e.F.doThis(F.java:20)
a.b.C.main(C.java:10)
This shows the chain of classes and method calls that led to the stream opening, which is almost always sufficient to identify the culprit.
-
id
is an incremental index - the number of streams monitored so far during this JVM session. -
created
is the timestamp when the stream was opened
For convenience, the detector accepts the following parameters as environment variables, falling back to defaults if unspecified:
variable | purpose | default |
---|---|---|
|
initial grace period (ms) to ignore detection, right after JVM startup; without this, unnecessary streams (JVM- and framework-level (e.g. Spring) JAR/classloading etc.) may also be captured |
|
|
minimum lifetime (ms) of a stream for it to be considered as a leak, i.e. maximum time allowed for a stream to close normally |
|
|
whether to capture leaks for |
|
|
whether to capture leaks for |
|
|
to which standard output stream ( |
|
-
The trick will not work on any JVMs where the class-load endorsing/whitelisting/prioritization is not supported
-
The tool may produce false positives if the stream is closed via some means other than
close()
: I’m not aware of any "other means" though. -
The tool would warn you, regardless of whether the stream is actually backed by system level resources (e.g. in case of mock or in-memory streams/buffers a long-lived stream may not raise an issue).
-
In some cases like long-lived TCP sockets, it may be acceptable to have the stream open for prolonged periods of time - may not be a major concern because each stream is warned for exactly once.
-
If your system or JVM is too busy, the timers may not get fired exactly on-time.
-
Under heavy load with high-frequency stream ops, the tool may degrade performance (console dumps; several extra objects being created for each opened stream; extra thread scheduling; etc.)
-
Finally, worst case, under load, it could hang or crash your JVM entirely.
This tool is not licensed; use it at your free will. I would appreciate it if you would mention me (and my company, which made it all possible) when referring to this tool elsewhere.