JDK 11 - Class sun.reflect.CallerSensitive not found - continuing with a stub.

[Sciss]

I am building a project with "-release", "8", compiling with sbt under OpenJDK 11. For both cross Scala versions 2.12.9 and 2.13.0, this fails at several places with errors like

[error] /home/hhrutz/Documents/devel/SoundProcesses/core/src/main/scala/de/sciss/synth/proc/SynthGraphObj.scala:166:27: Class sun.reflect.CallerSensitive not found - continuing with a stub.
[error]           val companion = Class.forName(s"$className").getField("MODULE$").get(null)
[error]                           ^

or

[error] /home/hhrutz/Documents/devel/SoundProcesses/core/src/main/scala/de/sciss/synth/proc/impl/MemoryClassLoader.scala:20:39: Class sun.reflect.CallerSensitive not found - continuing with a stub.
[error] final class MemoryClassLoader extends ClassLoader {
[error]                                       ^

Unfortunately it's not easy to isolate the code that breaks (I tried simple reductions but they don't crash the compiler). It's on a project that runs on multiple snapshot dependencies, so the heavy way to reproduce is

java -version
openjdk version "11.0.4" 2019-07-16
OpenJDK Runtime Environment (build 11.0.4+11-post-Debian-1deb10u1)
OpenJDK 64-Bit Server VM (build 11.0.4+11-post-Debian-1deb10u1, mixed mode, sharing)

(i.e. use JDK 11)

git clone https://github.com/Sciss/Lucre.git
cd Lucre
git checkout -b debug 53c779f0edfbf6bf0fc8f276d946290c145d85e3
sbt update
sbt publishLocal
cd ..
git clone https://github.com/Sciss/LucreSwing.git
cd LucreSwing
git checkout -b debug e6b50892553e765a3e840ea68c4bf5e5e0879fd9
sbt update
sbt publishLocal
cd ..
git clone https://github.com/Sciss/SoundProcesses.git
cd SoundProcesses
git checkout -b debug f25d534f5f74b80c3cc52abbec5fcbbed3958e9d
sbt update
sbt compile

This should be reproducible and yield

[error] /data/temp/bla/SoundProcesses/core/src/main/scala/de/sciss/synth/proc/SynthGraphObj.scala:166:27: Class sun.reflect.CallerSensitive not found - continuing with a stub.
[error]           val companion = Class.forName(className).getField("MODULE$").get(null)
[error]                           ^

Perhaps related to #10817 . The only place that CallerSensitive appears in seems to be https://github.com/scala/scala/blob/3027652cab884c88f5cc20f710e351a0730c5ba2/src/compiler/scala/tools/nsc/backend/jvm/opt/BytecodeUtils.scala#L132

[SethTisue]

@lrytz this one might interest you

[lrytz]

git clone https://github.com/Sciss/Lucre.git
cd Lucre
git checkout -b debug 53c779f0edfbf6bf0fc8f276d946290c145d85e3
sbt update

failed already for me

[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	::          UNRESOLVED DEPENDENCIES         ::
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	:: com.sleepycat#je;6.4.25: not found
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] 	Note: Unresolved dependencies path:
[warn] 		com.sleepycat:je:6.4.25 (/Users/luc/Downloads/tmp/Lucre/build.sbt#L171)
[warn] 		  +- de.sciss:lucre-bdb6_2.12:3.14.0-SNAPSHOT
[error] sbt.librarymanagement.ResolveException: unresolved dependency: com.sleepycat#je;7.4.5: not found

I don't think it's related to the code in the backend that you refer to, as we're not loading any symbols at this point.

It's probably some other entity that carries the @CallerSensitive annotation causing the symbol to be loaded.

It's quite certainly related to the use of -release. Since CallerSensitive is part of sun.reflect, does it actually exists in the ct.sym file? cc @retronym.

[Sciss]

failed already for me

Oh great, Oracle is doing its best to tear down all their Java related resources (https://download.oracle.com/maven - 404). It will take me a moment to see where that repository went.

[Sciss]

Hmmm, no, download works fine for me (and Travis CI also builds without complaining); perhaps you are behind a strange firewall?

...
[info] downloading http://download.oracle.com/maven/com/sleepycat/je/6.4.25/je-6.4.25.jar ...
[info] 	[SUCCESSFUL ] com.sleepycat#je;6.4.25!je.jar (4266ms)
[info] Done updating.
[info] downloading https://repo1.maven.org/maven2/de/sciss/fileutil_2.12/1.1.3/fileutil_2.12-1.1.3.jar ...
[info] downloading https://repo1.maven.org/maven2/de/sciss/model_2.12/0.3.4/model_2.12-0.3.4.jar ...
[info] downloading https://repo1.maven.org/maven2/de/sciss/span_2.12/1.4.2/span_2.12-1.4.2.jar ...
[info] 	[SUCCESSFUL ] de.sciss#fileutil_2.12;1.1.3!fileutil_2.12.jar (286ms)
[info] 	[SUCCESSFUL ] de.sciss#model_2.12;0.3.4!model_2.12.jar (286ms)
[info] 	[SUCCESSFUL ] de.sciss#span_2.12;1.4.2!span_2.12.jar (463ms)
[info] Done updating.
[info] downloading http://download.oracle.com/maven/com/sleepycat/je/5.0.104/je-5.0.104.jar ...
[info] 	[SUCCESSFUL ] com.sleepycat#je;5.0.104!je.jar (3239ms)
[info] Done updating.
[info] downloading http://download.oracle.com/maven/com/sleepycat/je/7.4.5/je-7.4.5.jar ...
[info] 	[SUCCESSFUL ] com.sleepycat#je;7.4.5!je.jar (4582ms)
[info] Done updating.
[info] downloading https://repo1.maven.org/maven2/de/sciss/fingertree_2.12/1.5.4/fingertree_2.12-1.5.4.jar ...
[info] 	[SUCCESSFUL ] de.sciss#fingertree_2.12;1.5.4!fingertree_2.12.jar (386ms)
[info] Done updating.
[info] Updating ...
[info] Done updating.

[retronym]

Yes, the sun.* classes are absent from the ct.sym based archive of historical Java standard library APIs which {javac,scalac} -release use.

APIPhase, a custom compiler phase in SBT/Zinc that serializes the API of each file to perform change detection in incremental compilation, is attempting to serialize the API of the member @CallerSensitive ClassLoader getParent that de.sciss.synth.proc.impl.MemoryClassLoader inherits from j.l.ClassLoader. Until that point, scalac was doing okay without having a classfile for CallerSensitve -- it just used a StubSymbol in its place.

When APIPhase filters the list of annotations for those that extend scala.StaticAnnotation, the stub symbol gives up the ruse and aborts compilation. Static annotations are part of the API because the are could affect how client code is compiled.

I see two ways to make this work.

• We could update StubSymbol.isStatic to return false, which should exclude that annotation from Zinc's API serialization.
• We could update Zinc with effectively the same change.

/cc @jvican @eed3si9n

[retronym]

An immediate workaround would be to declare an annotation sun.reflect.CallerSensitive in your project. You could structure the build in a way to avoid shipping this in your library.

[retronym]

I tried changing AnnotationInfo.isStatic. That makes the test project compile as I hoped.

$ sbt '++2.12.10-bin-b7c2be1-SNAPSHOT!' soundprocesses-core/compile
...
[info] Done compiling.
[success] Total time: 23 s, completed 27 Aug. 2019, 4:58:39 pm

commit b7c2be1df4fdc3d0664caaa88c82252d86531367 (HEAD -> ticket/11679, retronym/ticket/11679)
Author: Jason Zaugg <jzaugg@gmail.com>
Date:   Tue Aug 27 16:52:40 2019 +1000

    Assume StubSymbols are not StaticAnnotations

diff --git a/src/reflect/scala/reflect/internal/AnnotationInfos.scala b/src/reflect/scala/reflect/internal/AnnotationInfos.scala
index e438b2d677..619d9ac23e 100644
--- a/src/reflect/scala/reflect/internal/AnnotationInfos.scala
+++ b/src/reflect/scala/reflect/internal/AnnotationInfos.scala
@@ -323,7 +323,14 @@ trait AnnotationInfos extends api.Annotations { self: SymbolTable =>
     /** Check whether the type or any of the arguments are erroneous */
     def isErroneous = atp.isErroneous || args.exists(_.isErroneous)

-    def isStatic = symbol isNonBottomSubClass StaticAnnotationClass
+    def isStatic = {
+      symbol match {
+        case _: StubSymbol =>
+          false // See scala/bug#11679
+        case _ =>
+          symbol isNonBottomSubClass StaticAnnotationClass
+      }
+    }

     /** Check whether any of the arguments mention a symbol */
     def refsSymbol(sym: Symbol) = hasArgWhich(_.symbol == sym)

But I fear that this could make scalac too lenient -- when annotations based on stub symbols make it into signatures serialized by the pickler phase, we should emit a hard error.

[retronym]

Extended patch to scalac that would retain the desired fail-fast behaviour in the pickler: retronym/scala#67

[Sciss]

Hello. Is there any workaround, I'm running into this again (cross building with Scala 2.12.10 and 2.13.1)

[error] /home/hhrutz/Documents/devel/dotterweide/core/src/main/scala/dotterweide/compiler/BytecodeInvoker.scala:49:44: Class sun.reflect.CallerSensitive not found - continuing with a stub.
[error]   private class DynamicClassLoader extends ClassLoader {
[error]                                            ^
[error] one error found

Is there a trick I can use? Unfortunately here I really have to compile under JDK 11 and I really need -release, 8, because one project module makes use of JDK 11 while all others should be compatible with JDK 8.

[lrytz]

Maybe you could put the JDK 8 rt.jar on the Scala compiler's classpath while compiling that module (and then not use -release)?