Binary incompatible problem betwen published versinon 1.0.0, 1.0.1 and snapshot

[ohze]

I don't understand this. It seems to me we haven't touched this code.

I'm disinclined to spend my time digging into it. We're still in 0.x releases, and over at #213 we're planning on going 1.0, which means resetting binary compatibility anyway.

I'll just add an exclusion, and if it turns out this was a real problem we shouldn't have ignored, then I apologize to you, problem-experiencer-from-the-future.

Originally posted by @SethTisue in #211 (comment)

The published version 1.0.0 and 1.0.1 (on maven central) is incompatible with both version 0.9.1 and the snapshot version build from current main branch (commit 6074480)

Verify:

1. Add to build.sbt

lazy val tmp = project
  .settings(
    scalaVersion := "2.13.6",
//    libraryDependencies := Seq("org.scala-lang.modules" %% "scala-java8-compat" % "0.9.1"),
    libraryDependencies := Seq("org.scala-lang.modules" %% "scala-java8-compat" % "1.0.0"),
    TaskKey[Unit]("runMe") := {
      import scala.sys.process._
      val myCp = (Compile / products).value
      val compatCp = (scalaJava8Compat / Compile / products).value
      val scalaLib = scalaInstance.value.libraryJars
      val cp = (myCp ++ compatCp ++ scalaLib).mkString(":")
      val cmd = s"java -cp $cp Main"
      println(cmd)
      val out = cmd.!!
      println(out)
    },
  )

2. Add file tmp/src/main/scala/Main.scala

import scala.compat.java8.FunctionConverters._
import java.util.function.IntFunction

object Main {
  def invoke(jfun: IntFunction[String]): String = jfun(2)
  def main(args: Array[String]): Unit = {
    val fun = (i: Int) => s"ret: $i"
    val ret = invoke(fun.asJava)
    println(ret)
  }
}

3. In sbt:

clean
[success] Total time: 0 s, completed Oct 3, 2021 7:39:19 PM
[IJ]tmp/run
[info] compiling 1 Scala source to /Users/thanhbv/ohze/oss/scala-java8-compat/tmp/target/scala-2.13/classes ...
[info] done compiling
[info] running Main 
ret: 2
[success] Total time: 1 s, completed Oct 3, 2021 7:39:25 PM
[IJ]tmp/runMe
[info] running (fork) WrapFnGen /Users/thanhbv/ohze/oss/scala-java8-compat/target/scala-2.13/src_managed/main/FunctionConverters.scala
[info] compiling 13 Scala sources and 51 Java sources to /Users/thanhbv/ohze/oss/scala-java8-compat/target/scala-2.13/classes ...
[warn] /Users/thanhbv/ohze/oss/scala-java8-compat/src/main/scala-2.13+/scala/compat/java8/converterImpl/StepperExtensions.scala:21:10: match may not be exhaustive.
[warn] It would fail on the following input: (x: AnyRef forSome x not in (AnyAccumulator, DoubleAccumulator, IntAccumulator, LongAccumulator))
[warn]     info.companion match {
[warn]          ^
[warn] one warning found
[info] /Users/thanhbv/ohze/oss/scala-java8-compat/src/main/java-2.13+/scala/compat/java8/ScalaStreamSupport.java: Some input files use unchecked or unsafe operations.
[info] /Users/thanhbv/ohze/oss/scala-java8-compat/src/main/java-2.13+/scala/compat/java8/ScalaStreamSupport.java: Recompile with -Xlint:unchecked for details.
[info] done compiling
java -cp /Users/thanhbv/ohze/oss/scala-java8-compat/tmp/target/scala-2.13/classes:/Users/thanhbv/ohze/oss/scala-java8-compat/target/scala-2.13/classes:/Users/thanhbv/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.6/scala-library-2.13.6.jar Main
Exception in thread "main" java.lang.NoSuchMethodError: 'scala.Function1 scala.compat.java8.FunctionConverters.package$.enrichAsJavaIntFunction(scala.Function1)'
	at Main$.main(Main.scala:8)
	at Main.main(Main.scala)
[error] stack trace is suppressed; run last tmp / runMe for the full output
[error] (tmp / runMe) Nonzero exit value: 1
[error] Total time: 10 s, completed Oct 3, 2021 7:39:45 PM
[IJ]set tmp/libraryDependencies := Seq("org.scala-lang.modules" %% "scala-java8-compat" % "0.9.1")
[info] Defining tmp / libraryDependencies
[info] The new value will be used by tmp / allDependencies, tmp / dependencyPositions, tmp / updatePgpSignatures / signaturesModule
[info] Reapplying settings...
[info] set current project to scala-java8-compat (in build file:/Users/thanhbv/ohze/oss/scala-java8-compat/)
[IJ]clean
[success] Total time: 0 s, completed Oct 3, 2021 7:40:04 PM
[IJ]tmp/runMe
[info] compiling 1 Scala source to /Users/thanhbv/ohze/oss/scala-java8-compat/tmp/target/scala-2.13/classes ...
[info] running (fork) WrapFnGen /Users/thanhbv/ohze/oss/scala-java8-compat/target/scala-2.13/src_managed/main/FunctionConverters.scala
[info] done compiling
[info] compiling 13 Scala sources and 51 Java sources to /Users/thanhbv/ohze/oss/scala-java8-compat/target/scala-2.13/classes ...
[warn] /Users/thanhbv/ohze/oss/scala-java8-compat/src/main/scala-2.13+/scala/compat/java8/converterImpl/StepperExtensions.scala:21:10: match may not be exhaustive.
[warn] It would fail on the following input: (x: AnyRef forSome x not in (AnyAccumulator, DoubleAccumulator, IntAccumulator, LongAccumulator))
[warn]     info.companion match {
[warn]          ^
[warn] one warning found
[info] /Users/thanhbv/ohze/oss/scala-java8-compat/src/main/java-2.13+/scala/compat/java8/ScalaStreamSupport.java: Some input files use unchecked or unsafe operations.
[info] /Users/thanhbv/ohze/oss/scala-java8-compat/src/main/java-2.13+/scala/compat/java8/ScalaStreamSupport.java: Recompile with -Xlint:unchecked for details.
[info] done compiling
java -cp /Users/thanhbv/ohze/oss/scala-java8-compat/tmp/target/scala-2.13/classes:/Users/thanhbv/ohze/oss/scala-java8-compat/target/scala-2.13/classes:/Users/thanhbv/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.6/scala-library-2.13.6.jar Main
ret: 2

[success] Total time: 13 s, completed Oct 3, 2021 7:40:19 PM
[IJ]

Analyse

tmp/runMe fail with

libraryDependencies := Seq("org.scala-lang.modules" %% "scala-java8-compat" % "1.0.0") // or 1.0.1

and success with

libraryDependencies := Seq("org.scala-lang.modules" %% "scala-java8-compat" % "0.9.1")

So snapshot code in main branch is:

• compatible with 0.9.1
• but not compatible with 1.0.0 and 1.0.1

[ohze]

Output of javap -v -c Priority1FunctionConverters.class:

• for target/scala-2.13/classes/scala/compat/java8/Priority1FunctionConverters.class
enrichAsJavaIntFunction has 2 params

public default <A0 extends java.lang.Object, R extends java.lang.Object>
  scala.Function1<java.lang.Object, R> enrichAsJavaIntFunction(
    scala.Function1<A0, R>,
    scala.$eq$colon$eq<A0, java.lang.Object>);
descriptor: (Lscala/Function1;Lscala/$eq$colon$eq;)Lscala/Function1;

• for 1.0.1/scala/compat/java8/Priority1FunctionConverters.class
  (extract from scala-java8-compat_2.13-1.0.1.jar from maven central)
  enrichAsJavaIntFunction has ONLY 1 param

public default <R extends java.lang.Object> scala.Function1<java.lang.Object, R>
  enrichAsJavaIntFunction(scala.Function1<java.lang.Object, R>);
descriptor: (Lscala/Function1;)Lscala/Function1;

[ohze]

Hmm. I tried to run Release CI job in my fork and in the generated FunctionConverters.scala, method enrichAsJavaIntFunction have 1 param:

@inline implicit def enrichAsJavaIntFunction[R](sf: scala.Function1[Int, R]): RichFunction1AsIntFunction[R] = new RichFunction1AsIntFunction[R](sf)

But running in local machine, the result is different:

$ CI_COMMIT_TAG=x \
  PGP_PASSPHRASE=x \
  PGP_SECRET=x \
  SONATYPE_PASSWORD=x \
  SONATYPE_USERNAME=x \
  CI_SNAPSHOT_RELEASE=';clean ;set publishTo := Some(Resolver.file("sonatype-local-bundle", sonatypeBundleDirectory.value)) ;+publish' \
  CI_RELEASE=';set publishTo := Some(Resolver.file("sonatype-local-bundle", sonatypeBundleDirectory.value)) ;+publish' \
  CI_SONATYPE_RELEASE=version \
  sbt ci-release

$ grep enrichAsJavaIntFunction target/scala-*/src_managed/main/FunctionConverters.scala                                                                                  
target/scala-2.11/src_managed/main/FunctionConverters.scala:  @inline implicit def enrichAsJavaIntFunction[A0, R](sf: scala.Function1[A0, R])(implicit evA0: =:=[A0, Int]): RichFunction1AsIntFunction[R] = new RichFunction1AsIntFunction[R](sf.asInstanceOf[scala.Function1[Int, R]])
target/scala-2.12/src_managed/main/FunctionConverters.scala:  @inline implicit def enrichAsJavaIntFunction[A0, R](sf: scala.Function1[A0, R])(implicit evA0: =:=[A0, Int]): RichFunction1AsIntFunction[R] = new RichFunction1AsIntFunction[R](sf.asInstanceOf[scala.Function1[Int, R]])
target/scala-2.13/src_managed/main/FunctionConverters.scala:  @inline implicit def enrichAsJavaIntFunction[A0, R](sf: scala.Function1[A0, R])(implicit evA0: =:=[A0, Int]): RichFunction1AsIntFunction[R] = new RichFunction1AsIntFunction[R](sf.asInstanceOf[scala.Function1[Int, R]])
target/scala-3.0.2/src_managed/main/FunctionConverters.scala:  @inline implicit def enrichAsJavaIntFunction[A0, R](sf: scala.Function1[A0, R])(implicit evA0: =:=[A0, Int]): RichFunction1AsIntFunction[R] = new RichFunction1AsIntFunction[R](sf.asInstanceOf[scala.Function1[Int, R]])

[ohze]

Haha
I change release.yml CI file:
from sbt ci-release
to sbt +sources then grep enrichAsJavaIntFunction target/scala-*/src_managed/main/FunctionConverters.scala
grep output:

target/scala-2.11/src_managed/main/FunctionConverters.scala:
@inline implicit def enrichAsJavaIntFunction[R](sf: scala.Function1[Int, R]): RichFunction1AsIntFunction[R] = new RichFunction1AsIntFunction[R](sf)
target/scala-2.12/src_managed/main/FunctionConverters.scala:
@inline implicit def enrichAsJavaIntFunction[R](sf: scala.Function1[Int, R]): RichFunction1AsIntFunction[R] = new RichFunction1AsIntFunction[R](sf)
target/scala-2.13/src_managed/main/FunctionConverters.scala:
@inline implicit def enrichAsJavaIntFunction[R](sf: scala.Function1[Int, R]): RichFunction1AsIntFunction[R] = new RichFunction1AsIntFunction[R](sf)
target/scala-3.0.2/src_managed/main/FunctionConverters.scala:
@inline implicit def enrichAsJavaIntFunction[R](sf: scala.Function1[Int, R]): RichFunction1AsIntFunction[R] = new RichFunction1AsIntFunction[R](sf)

And with just a should-be-no-effect-change in fnGen/WrapFnGen.scala, the grep output is different: (has 2 params instead of 1)

grep enrichAsJavaIntFunction target/scala-*/src_managed/main/FunctionConverters.scala
target/scala-2.11/src_managed/main/FunctionConverters.scala:
@inline implicit def enrichAsJavaIntFunction[A0, R](sf: scala.Function1[A0, R])(implicit evA0: =:=[A0, Int]): RichFunction1AsIntFunction[R] = new RichFunction1AsIntFunction[R](sf.asInstanceOf[scala.Function1[Int, R]])
target/scala-2.12/src_managed/main/FunctionConverters.scala:
@inline implicit def enrichAsJavaIntFunction[A0, R](sf: scala.Function1[A0, R])(implicit evA0: =:=[A0, Int]): RichFunction1AsIntFunction[R] = new RichFunction1AsIntFunction[R](sf.asInstanceOf[scala.Function1[Int, R]])
target/scala-2.13/src_managed/main/FunctionConverters.scala:
@inline implicit def enrichAsJavaIntFunction[A0, R](sf: scala.Function1[A0, R])(implicit evA0: =:=[A0, Int]): RichFunction1AsIntFunction[R] = new RichFunction1AsIntFunction[R](sf.asInstanceOf[scala.Function1[Int, R]])
target/scala-3.0.2/src_managed/main/FunctionConverters.scala:
@inline implicit def enrichAsJavaIntFunction[A0, R](sf: scala.Function1[A0, R])(implicit evA0: =:=[A0, Int]): RichFunction1AsIntFunction[R] = new RichFunction1AsIntFunction[R](sf.asInstanceOf[scala.Function1[Int, R]])

I think fnGen is buggy and too complex.
We should remove that sbt project and add FunctionConverters.scala into git 😄

[SethTisue]

I've added a notice to https://github.com/scala/scala-java8-compat/releases/tag/v1.0.1 about the issue.