JarLauncher fails to load large jar files

[KaurKadakWise]

I have an issue when updating to SB 3.2 and trying to use the new Jar launcher.

I changed my JarLauncher reference address from
org.springframework.boot.loader.JarLauncher to org.springframework.boot.loader.launch.JarLauncher
after SB 3.2 upgrade, as per instructions, and started getting this error when trying build an image from my dockerfile.

I am suspicious that the reasoning here is my large jar size (~2.6GB), because other services that I've upgraded have not faced this issue.

The error happens on the RUN java ... command

 => ERROR [builder 3/3] RUN java -Djarmode=layertools -jar service.jar extract                                                                                                                                                                                                                                                                                                                                                0.3s
------
 > [builder 3/3] RUN java -Djarmode=layertools -jar service.jar extract:
0.311 Exception in thread "main" java.lang.reflect.InvocationTargetException
0.313   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
0.313   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
0.313   at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
0.313   at java.base/java.lang.reflect.Method.invoke(Method.java:569)
0.313   at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:91)
0.314   at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:53)
0.314   at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:58)
0.314 Caused by: java.lang.AssertionError: Package org.springframework.util has already been defined but it could not be found
0.314   at org.springframework.boot.loader.net.protocol.jar.JarUrlClassLoader.tolerateRaceConditionDueToBeingParallelCapable(JarUrlClassLoader.java:165)
0.315   at org.springframework.boot.loader.net.protocol.jar.JarUrlClassLoader.definePackageIfNecessary(JarUrlClassLoader.java:128)
0.315   at org.springframework.boot.loader.net.protocol.jar.JarUrlClassLoader.loadClass(JarUrlClassLoader.java:98)
0.315   at org.springframework.boot.loader.launch.LaunchedClassLoader.loadClass(LaunchedClassLoader.java:91)
0.315   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
0.315   at org.springframework.boot.loader.launch.JarModeRunner.main(JarModeRunner.java:40)
0.315   ... 7 more
0.315 Caused by: java.lang.IllegalArgumentException: Size must not be negative and must be within bounds
0.315   at org.springframework.boot.loader.zip.FileDataBlock.slice(FileDataBlock.java:147)
0.315   at org.springframework.boot.loader.zip.FileDataBlock.slice(FileDataBlock.java:128)
0.315   at org.springframework.boot.loader.zip.ZipContent$Loader.loadContent(ZipContent.java:592)
0.315   at org.springframework.boot.loader.zip.ZipContent$Loader.openAndLoad(ZipContent.java:578)
0.316   at org.springframework.boot.loader.zip.ZipContent$Loader.loadNonNested(ZipContent.java:563)
0.316   at org.springframework.boot.loader.zip.ZipContent$Loader.load(ZipContent.java:549)
0.316   at org.springframework.boot.loader.zip.ZipContent.open(ZipContent.java:384)
0.316   at org.springframework.boot.loader.zip.ZipContent.open(ZipContent.java:361)
0.316   at org.springframework.boot.loader.zip.ZipContent$Loader.load(ZipContent.java:551)
0.316   at org.springframework.boot.loader.zip.ZipContent.open(ZipContent.java:384)
0.316   at org.springframework.boot.loader.zip.ZipContent.open(ZipContent.java:373)
0.316   at org.springframework.boot.loader.jar.NestedJarFileResources.<init>(NestedJarFileResources.java:60)
0.316   at org.springframework.boot.loader.jar.NestedJarFile.<init>(NestedJarFile.java:147)
0.316   at org.springframework.boot.loader.jar.NestedJarFile.<init>(NestedJarFile.java:124)
0.316   at org.springframework.boot.loader.net.protocol.jar.UrlNestedJarFile.<init>(UrlNestedJarFile.java:42)
0.317   at org.springframework.boot.loader.net.protocol.jar.UrlJarFileFactory.createJarFileForNested(UrlJarFileFactory.java:86)
0.317   at org.springframework.boot.loader.net.protocol.jar.UrlJarFileFactory.createJarFile(UrlJarFileFactory.java:55)
0.317   at org.springframework.boot.loader.net.protocol.jar.UrlJarFiles.getOrCreate(UrlJarFiles.java:72)
0.317   at org.springframework.boot.loader.net.protocol.jar.JarUrlConnection.connect(JarUrlConnection.java:289)
0.317   at org.springframework.boot.loader.net.protocol.jar.JarUrlConnection.getJarFile(JarUrlConnection.java:99)
0.317   at org.springframework.boot.loader.net.protocol.jar.JarUrlClassLoader.getJarFile(JarUrlClassLoader.java:184)
0.317   at org.springframework.boot.loader.net.protocol.jar.JarUrlClassLoader.definePackage(JarUrlClassLoader.java:142)
0.317   at org.springframework.boot.loader.net.protocol.jar.JarUrlClassLoader.definePackageIfNecessary(JarUrlClassLoader.java:125)
0.317   ... 11 more

When switching back to the CLASSIC launcher, like this:

bootJar {
    loaderImplementation = org.springframework.boot.loader.tools.LoaderImplementation.CLASSIC

    ...

It started working again, so the jar itself is not corrupt from what I've observed.

[wilkinsona]

@KaurKadakWise the size of the jar could be the problem, but it could also be due to the content of one of the nested jar files. Unfortunately, to get to the bottom of this one, I think it's likely that we'll need to be able to reproduce the problem. Could you perhaps try to create a jar that fails in the same way? It need not be the actual jar for your application, but a "synthetic" approximation of that jar that has the required structure/size to reproduce the problem. Alternatively, if you're happy to do so, perhaps you could share the actual jar file with us privately?

[KaurKadakWise]

Can't share the actual jar, company policy.

I tried to recreate a similar jar on my local computer by adding some large resources, but my local computer can't build it, or I didn't figure out how to get the jar size up, was using gradle. If you have some advice with how could I create a large artifical jar, I'm open to suggestions!

The original jar is a layered jar and most of the size comes from a 3rd party dependency that we are running on-prem.

[philwebb]

@KaurKadakWise You might be able to use ZipOutputStream to create one, but it can be tricky to reproduce these issues. Running your app with -Dloader.debug=true might also help identify the cause of problem. Feel free to email me those logs (phil.webb@broadcom.com) if you don't want to post them publicly.

Does your 2.6GB jar contain lots of small files, or does it contain one or more large files?

[KaurKadakWise]

Does your 2.6GB jar contain lots of small files, or does it contain one or more large files?

It's one large file, and the usual Spring stuff.

I cloned Spring Boot project and placed my failing JAR into it and it looks like the Zip file handling is what is failing. My guess is that the large (~2.5GB) file has the same byte signature at some point as Zip64EndOfCentralDirectoryRecord.SIGNATURE I don't know much about how Zips files work, but I'll dig into it a little bit more, see if I can find some methods on the internet to validate that the EOCD file found from the jar is valid.

[philwebb]

https://en.wikipedia.org/wiki/ZIP_(file_format) is pretty useful. You can also try java -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y -jar target/my.jar if you want to try and debug things.

[philwebb]

@KaurKadakWise I've managed to reproduce the issue. See https://github.com/philwebb/spring-boot-gh-42012