Error build a project with the same module and submodule name

[sergeygalkin]

Plugin can't build project with module structure like this
basic
module/basic
module/submodule

Part of mvn compile -X -pl module/submodule jib:buildTar

> [DEBUG]     Snapshot dependencies:
> [DEBUG]         /usr/home/gals/.m2/repository/io/site/0.0.1-SNAPSHOT/basic-0.0.1-SNAPSHOT.jar
> [DEBUG]         /usr/home/gals/.m2/repository/io/site/module/basic/0.0.1-SNAPSHOT/basic-0.0.1-SNAPSHOT.jar
> [DEBUG]         /usr/home/gals/.m2/repository/io/site/module/submodule/0.0.1-SNAPSHOT/submodule-0.0.1-SNAPSHOT.jar

Trace output is

> [ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:0.9.8:buildTar (default-cli) on project eth: Building image tarball failed: entry 'app/libs//basic-0.0.1-SNAPSHOT.jar' closed at '5340' before the '15583' bytes specified in the header were written -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.google.cloud.tools:jib-maven-plugin:0.9.8:buildTar (default-cli) on project eth: Building image tarball failed
> 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
> 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> 	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
> 	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
> 	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
> 	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
> 	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
> 	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
> 	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
> 	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
> 	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
> 	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> 	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> 	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> 	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.MojoExecutionException: Building image tarball failed
> 	at com.google.cloud.tools.jib.maven.BuildTarMojo.execute(BuildTarMojo.java:148)
> 	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
> 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
> 	... 20 more
> Caused by: java.io.IOException: entry 'app/libs//basic-0.0.1-SNAPSHOT.jar' closed at '5340' before the '15583' bytes specified in the header were written
> 	at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.closeArchiveEntry(TarArchiveOutputStream.java:419)
> 	at com.google.cloud.tools.jib.tar.TarStreamBuilder.writeEntriesAsTarArchive(TarStreamBuilder.java:52)
> 	at com.google.cloud.tools.jib.blob.WriterBlob.writeTo(WriterBlob.java:36)
> 	at com.google.cloud.tools.jib.cache.CacheWriter.writeLayer(CacheWriter.java:73)
> 	at com.google.cloud.tools.jib.builder.steps.BuildAndCacheApplicationLayerStep.call(BuildAndCacheApplicationLayerStep.java:121)
> 	at com.google.cloud.tools.jib.builder.steps.BuildAndCacheApplicationLayerStep.call(BuildAndCacheApplicationLayerStep.java:37)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:127)
> 	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:80)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 	at java.lang.Thread.run(Thread.java:748)
> 	Suppressed: java.io.IOException: This archive contains unclosed entries.
> 		at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.finish(TarArchiveOutputStream.java:289)
> 		at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.close(TarArchiveOutputStream.java:306)
> 		at com.google.cloud.tools.jib.tar.TarStreamBuilder.writeEntriesAsTarArchive(TarStreamBuilder.java:54)
> 		... 10 more

workaround is rename module/basic

[coollog]

Hi @sergeygalkin , thanks for reporting this issue! It looks like this is a consequence of Jib placing all the dependency JARs in the /app/libs directory, so if two dependency JARs had the same name (basic-0.0.1-SNAPSHOT.jar), this failure will occur. I think the solution is to rename dependency JARs to include their group IDs as well when Jib adds the entries for the JARs.

[coollog]

The issue in Gradle is that the dependencies are only resolved with their file path, and not with their groupId like in Maven. Therefore, we cannot easily get this groupId in Gradle without performing logic on the file path.

We should probably keep the logic for Maven/Gradle the same in terms of generating the filename for the dependency jar in the container image. Therefore, there are a few strategies:

• name the JAR with a unique ID - /app/libs/e22k1nckie.jar
• name the JAR with its original name plus a unique ID - /app/libs/basic-0.0.1-SNAPSHOT-gw2nck3.jar
• name only duplicate JAR names with a unique ID - second /app/libs/basic-0.0.1-SNAPSHOT.jar becomes /app/libs/e22k1nckie.jar

Thoughts? @GoogleContainerTools/java-tools

[chanseokoh]

The first option would be the simplest and has a merit, but I think retaining the original name outweighs the benefit of the simple implementation here for debugging purposes when something goes wrong. The option 2 seems to work, but I wonder how long the unique ID would be.

[coollog]

I think with option 3, we could even number further occurrences such that the second one becomes /app/libs/basic-0.0.1-SNAPSHOT-2.jar, for example.

[coollog]

With option 2, the unique ID can also just be a counter, so the length would just be log number of jars.

[chanseokoh]

I've thought about using numbers. I'd still add some special, discernible prefix, like -renamed-2, so that it doesn't look like part of the semver versioning (e.g., -2 in 0.1.0-2 might look like the actual version of the dep). I vote for option 3 with a simple scheme, but if detecting dup seems unnecessarily complex, option 2 might be better.

[coollog]

Yea, detecting dup shouldn't be too complex since all the dependency jars are processed in the same code loop.

[loosebazooka]

I dunno, I mean the final jar artifact and the maven coordinates are two different things. In the end they are filenames that would be included in a jar referencing it's classpath through metadata in the same way.

Class-Path: one.jar two.jar

Or in war as files in WEB-INF/lib.

It really seems like the user should not be reusing names. And if they do it's unexpected behavior. I'm curious what the war plugin in gradle/maven does?

[chanseokoh]

The problem is that, the filename of a dependency JAR is basically the artifact ID of its Maven coordinates. For example, you can have multiple dependency JAR files with the same filename. (BTW, one should not include both javax.servlet.jstl:jstl and jstl:jstl.)

.m2/repository/javax/servlet/jstl/1.2/jstl-1.2.jar
.m2/repository/jstl/jstl/1.2/jstl-1.2.jar

What we currently do is to copy those jstl-1.2.jar into the same location, /app/libs, so one file ends up overwriting the other.

This is OK in WAR, since the group ID is appended.

$ ls WEB-INF/lib/
javax.servlet-jstl-1.2.jar  jstl-jstl-1.2.jar

It really seems like the user should not be reusing names

I think it is very possible that different dependencies may have the same artifact ID, which is OK because they will have different group IDs.

[loosebazooka]

This is OK in WAR, since the group ID is appended.

You're right, I seemed to have missed this. I guess we should solve it. Although I wonder if resolving the jars is deterministic in the build system? Should they be sorted by size?

[coollog]

Hi @sergeygalkin , we've released v1.0.0-rc1 with a fix for this issue.

[sergeygalkin]

@coollog A lot of thanks ! great job