Bug: `java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF` in actor / osgiBundle

[He-Pin]

Error when try to execute:

Jmh/run -i 3 -wi 3 -f1 -t1 .*ZipWithIndexBenchmark.* 

error] (actor / osgiBundle) java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
[error] Total time: 39 s, completed 2024-1-20 1:00:26
[IJ]last actor / osgiBundle
[info] Validating all packages are set private or exported for OSGi explicitly...
[warn] bnd: Unused -privatepackage instructions , no such package(s) on the class path: [org.apache.pekko.osgi.impl]
[warn] bnd: Unused Import-Package instructions: [scala.compat.java8.*]
[error] java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
[error]         at java.util.zip.ZipOutputStream.putNextEntry(ZipOutputStream.java:232)
[error]         at java.util.jar.JarOutputStream.putNextEntry(JarOutputStream.java:109)
[error]         at sbt.io.IO$.addFileEntry$1(IO.scala:717)
[error]         at sbt.io.IO$.$anonfun$writeZip$6(IO.scala:726)
[error]         at sbt.io.IO$.$anonfun$writeZip$6$adapted(IO.scala:726)
[error]         at scala.collection.immutable.Stream.foreach(Stream.scala:533)
[error]         at sbt.io.IO$.writeZip(IO.scala:726)
[error]         at sbt.io.IO$.$anonfun$archive$2(IO.scala:676)
[error]         at sbt.io.IO$.$anonfun$archive$2$adapted(IO.scala:673)
[error]         at sbt.io.IO$.$anonfun$withZipOutput$1(IO.scala:773)
[error]         at sbt.io.IO$.$anonfun$withZipOutput$1$adapted(IO.scala:753)
[error]         at sbt.io.Using.apply(Using.scala:28)
[error]         at sbt.io.IO$.withZipOutput(IO.scala:753)
[error]         at sbt.io.IO$.archive(IO.scala:673)
[error]         at sbt.io.IO$.jar(IO.scala:621)
[error]         at com.typesafe.sbt.osgi.Osgi$.$anonfun$bundleTask$1(Osgi.scala:187)
[error]         at scala.Option.getOrElse(Option.scala:189)
[error]         at com.typesafe.sbt.osgi.Osgi$.withCache(Osgi.scala:112)
[error]         at com.typesafe.sbt.osgi.Osgi$.bundleTask(Osgi.scala:139)
[error]         at com.typesafe.sbt.osgi.SbtOsgi$.$anonfun$defaultOsgiSettings$1(SbtOsgi.scala:57)
[error]         at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error]         at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:63)
[error]         at sbt.std.Transform$$anon$4.work(Transform.scala:69)
[error]         at sbt.Execute.$anonfun$submit$2(Execute.scala:283)
[error]         at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24)
[error]         at sbt.Execute.work(Execute.scala:292)
[error]         at sbt.Execute.$anonfun$submit$1(Execute.scala:283)
[error]         at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error]         at sbt.CompletionService$$anon$2.call(CompletionService.scala:65)
[error]         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error]         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error]         at java.lang.Thread.run(Thread.java:750)
[error] (actor / osgiBundle) java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
[error] (slf4j / osgiBundle) java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
[error] (stream / osgiBundle) java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
[error] (testkit / osgiBundle) java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
[error] (coordination / osgiBundle) java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF

[He-Pin]

@mdedetrich Would you like to take a view?

[Roiocam]

Don't know why, but won't happen on my laptop, can you provider more information?

[He-Pin]

@Roiocam to reproduce:

1) sbt shell
2) project bench-jmh
3) Jmh/run -i 3 -wi 3 -f1 -t1 .*ZipWithIndexBenchmark.* 

[Roiocam]

@Roiocam to reproduce:

1) sbt shell
2) project bench-jmh
3) Jmh/run -i 3 -wi 3 -f1 -t1 .*ZipWithIndexBenchmark.* 

Yep, that's what I did before. Those ERROR don't appear on my laptop.

[He-Pin]

I think it may need assemblyMergeStrategy or something like that. @Roiocam I'm using Windows 11.

[He-Pin]

refs: sbt/sbt-osgi#113

This blockme to run any jmh on windows :(

[mdedetrich]

This seems to be windows related, I am also not experiencing any issues.

[Roiocam]

I think it may need assemblyMergeStrategy or something like that. @Roiocam I'm using Windows 11.

I think this issue related to #920

[Roiocam]

I think it may need assemblyMergeStrategy or something like that. @Roiocam I'm using Windows 11.

I think this issue related to #920

The issue: sbt/sbt-assembly#80 may help with this issue.

[mdedetrich]

The issue: sbt/sbt-assembly#80 may help with this issue.

I have doubts that this is the core issue, its over 11 years old and our issue is unique to windows which to me indicates the problem is elsewhere.

[mdedetrich]

@He-Pin What I would recommend is changing the version in https://github.com/apache/incubator-pekko/blob/main/project/plugins.sbt#L16 until you hit a point where you can no longer replicate the issue.

[He-Pin]

addSbtPlugin("com.typesafe.sbt" % "sbt-osgi" % "0.9.4")
addSbtPlugin("com.typesafe.sbt" % "sbt-osgi" % "0.9.6")
works.

 // Recent versions of BND create corrupted jars so use JDK jar instead, see https://github.com/sbt/sbt-osgi/pull/81
      OsgiKeys.packageWithJVMJar := true)
addSbtPlugin("com.github.sbt" % "sbt-osgi" % "0.9.7")

broken.

      // Recent versions of BND create corrupted jars so use JDK jar instead, see https://github.com/sbt/sbt-osgi/pull/81
      OsgiKeys.packageWithJVMJar := false)
addSbtPlugin("com.github.sbt" % "sbt-osgi" % "0.9.7")

works.

     // Recent versions of BND create corrupted jars so use JDK jar instead, see https://github.com/sbt/sbt-osgi/pull/81
      OsgiKeys.packageWithJVMJar := false)
addSbtPlugin("com.github.sbt" % "sbt-osgi" % "0.9.10")

works.

I think the duplicate file comes from packageWithJVMJar.

[mdedetrich]

So I believe the relevant change is sbt/sbt-osgi#81, i.e. moving from BND to JVM jar tools to re-create the final osgi jar bundle.

@He-Pin Can you change https://github.com/apache/incubator-pekko/blob/main/project/plugins.sbt#L16 to 0.9.7 and https://github.com/apache/incubator-pekko/blob/main/project/OSGi.scala#L46 to false and let me know if you still get problems (or not)?

If you are not getting issues when setting packageWithJVMJar to false with sbt-osgi 0.9.7 its highly likely the logic in https://github.com/sbt/sbt-osgi/pull/81/files#diff-2530a9c505dd4cb34217eb195d5db3cab065049f033db306f485078da72edd5dR81-R93 is not properly portable on windows.