Native memory leak in zstd compression handler #14028
Description
Jetty version(s)
12.1.4
Jetty Environment
ee10
HTTP version
http/1.1, http/2
Java version/vendor
openjdk version "21.0.5" 2024-10-15 LTS
OpenJDK Runtime Environment Temurin-21.0.5+11 (build 21.0.5+11-LTS)
OpenJDK 64-Bit Server VM Temurin-21.0.5+11 (build 21.0.5+11-LTS, mixed mode, sharing)
OS type/version
Ubuntu 24.04.3 LTS
Description
When zstd compression is used for data served as resources by ServletContextHandler, there is native memory leak. It happens on both http/1.1 and http/2.
Scenario: Page with lot of small resources, 5× refreshes in Brave browser
Scenario - Variant A: compressPreferredEncodings(List.of("gzip"))
Scenario - Variant B: compressPreferredEncodings(List.of("zstd","gzip"))
When I continue to hit F5 for refresh few more times, RSS memory usage rises until there is no more free memory and oom killer terminates process.
JVM heap and non-heap memory seems to be fine, but process memory rises rapidly. Values from chart are obtained using:
cat /proc/<PID>/status | grep Vm
We are using same library for zstd compression (com.github.luben:zstd-jni:1.5.7-6) as jetty-compression-zstandard for the storing and retrieving data in our application and we are not experiencing memory leak in our storage routines. It might suggest that memory leak is not in zstd-jni library by itself.
How to reproduce?
Just configure compressPreferredEncodings to value, where "zstd" is on the first place
CompressionConfig compressionConfig = CompressionConfig.builder().defaults()
.compressPreferredEncodings(List.of("zstd","gzip"))
.build();
compressionHandler.putConfiguration("/*", compressionConfig);
compressionHandler.putCompression(new ZstandardCompression());
compressionHandler.putCompression(new GzipCompression());
And open a page with huge amount of small resources in Chromium based browser and hit refresh few times.