scalac 2.12.9 and 2.13.0 does not work with "-opt:l:method" on JDK13+

[xuwei-k]

Third time 😅 #10717 #11372

$ echo "class A" > A.scala
$ java --version
openjdk 13-ea 2019-09-17
OpenJDK Runtime Environment (build 13-ea+27)
OpenJDK 64-Bit Server VM (build 13-ea+27, mixed mode, sharing)
$ scala -version
Scala code runner version 2.12.9 -- Copyright 2002-2019, LAMP/EPFL and Lightbend, Inc.
$ scalac -opt:l:method A.scala 
error: java.lang.IllegalArgumentException: Unsupported class file major version 57
	at scala.tools.asm.ClassReader.<init>(ClassReader.java:184)
	at scala.tools.asm.ClassReader.<init>(ClassReader.java:166)
	at scala.tools.asm.ClassReader.<init>(ClassReader.java:152)
	at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.$anonfun$parseClass$1(ByteCodeRepository.scala:279)
	at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.parseClass(ByteCodeRepository.scala:277)
	at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.$anonfun$parsedClassNode$1(ByteCodeRepository.scala:71)
	at scala.collection.mutable.MapLike.getOrElseUpdate(MapLike.scala:209)
	at scala.collection.mutable.MapLike.getOrElseUpdate$(MapLike.scala:206)
	at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:82)
	at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.parsedClassNode(ByteCodeRepository.scala:71)
	at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.classNode(ByteCodeRepository.scala:92)
	at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.methodNode(ByteCodeRepository.scala:240)
	at scala.tools.nsc.backend.jvm.opt.CallGraph.$anonfun$addMethod$2(CallGraph.scala:161)
	at scala.tools.nsc.backend.jvm.opt.CallGraph.$anonfun$addMethod$2$adapted(CallGraph.scala:158)
	at scala.collection.Iterator.foreach(Iterator.scala:941)
	at scala.collection.Iterator.foreach$(Iterator.scala:941)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
	at scala.tools.nsc.backend.jvm.opt.CallGraph.addMethod(CallGraph.scala:158)
	at scala.tools.nsc.backend.jvm.opt.CallGraph.$anonfun$addClass$1(CallGraph.scala:117)
	at scala.tools.nsc.backend.jvm.opt.CallGraph.$anonfun$addClass$1$adapted(CallGraph.scala:117)
	at scala.collection.Iterator.foreach(Iterator.scala:941)
	at scala.collection.Iterator.foreach$(Iterator.scala:941)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
	at scala.collection.IterableLike.foreach(IterableLike.scala:74)
	at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
	at scala.tools.nsc.backend.jvm.opt.CallGraph.addClass(CallGraph.scala:117)
	at scala.tools.nsc.backend.jvm.PostProcessor.$anonfun$runGlobalOptimizations$5(PostProcessor.scala:119)
	at scala.tools.nsc.backend.jvm.PostProcessor.$anonfun$runGlobalOptimizations$5$adapted(PostProcessor.scala:117)
	at scala.collection.TraversableLike$WithFilter.$anonfun$foreach$1(TraversableLike.scala:877)
	at scala.collection.immutable.List.foreach(List.scala:392)
	at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:876)
	at scala.tools.nsc.backend.jvm.PostProcessor.$anonfun$runGlobalOptimizations$3(PostProcessor.scala:117)
	at scala.tools.nsc.backend.jvm.PostProcessor.$anonfun$runGlobalOptimizations$3$adapted(PostProcessor.scala:117)
	at scala.collection.immutable.List.foreach(List.scala:392)
	at scala.tools.nsc.backend.jvm.PostProcessor.runGlobalOptimizations(PostProcessor.scala:117)
	at scala.tools.nsc.backend.jvm.GeneratedClassHandler$GlobalOptimisingGeneratedClassHandler.complete(GeneratedClassHandler.scala:92)
	at scala.tools.nsc.backend.jvm.GenBCode$BCodePhase.$anonfun$run$1(GenBCode.scala:81)
	at scala.tools.nsc.backend.jvm.GenBCode$BCodePhase.run(GenBCode.scala:78)
	at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1503)
	at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1487)
	at scala.tools.nsc.Global$Run.compileSources(Global.scala:1480)
	at scala.tools.nsc.Global$Run.compile(Global.scala:1606)
	at scala.tools.nsc.Driver.doCompile(Driver.scala:47)
	at scala.tools.nsc.MainClass.doCompile(Main.scala:32)
	at scala.tools.nsc.Driver.process(Driver.scala:67)
	at scala.tools.nsc.Driver.main(Driver.scala:80)
	at scala.tools.nsc.Main.main(Main.scala)
Exception in thread "main" java.lang.IllegalArgumentException: Unsupported class file major version 57
	at scala.tools.asm.ClassReader.<init>(ClassReader.java:184)
	at scala.tools.asm.ClassReader.<init>(ClassReader.java:166)
	at scala.tools.asm.ClassReader.<init>(ClassReader.java:152)
	at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.$anonfun$parseClass$1(ByteCodeRepository.scala:279)
	at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.parseClass(ByteCodeRepository.scala:277)
	at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.$anonfun$parsedClassNode$1(ByteCodeRepository.scala:71)
	at scala.collection.mutable.MapLike.getOrElseUpdate(MapLike.scala:209)
	at scala.collection.mutable.MapLike.getOrElseUpdate$(MapLike.scala:206)
	at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:82)
	at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.parsedClassNode(ByteCodeRepository.scala:71)
	at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.classNode(ByteCodeRepository.scala:92)
	at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.methodNode(ByteCodeRepository.scala:240)
	at scala.tools.nsc.backend.jvm.opt.CallGraph.$anonfun$addMethod$2(CallGraph.scala:161)
	at scala.tools.nsc.backend.jvm.opt.CallGraph.$anonfun$addMethod$2$adapted(CallGraph.scala:158)
	at scala.collection.Iterator.foreach(Iterator.scala:941)
	at scala.collection.Iterator.foreach$(Iterator.scala:941)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
	at scala.tools.nsc.backend.jvm.opt.CallGraph.addMethod(CallGraph.scala:158)
	at scala.tools.nsc.backend.jvm.opt.CallGraph.$anonfun$addClass$1(CallGraph.scala:117)
	at scala.tools.nsc.backend.jvm.opt.CallGraph.$anonfun$addClass$1$adapted(CallGraph.scala:117)
	at scala.collection.Iterator.foreach(Iterator.scala:941)
	at scala.collection.Iterator.foreach$(Iterator.scala:941)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
	at scala.collection.IterableLike.foreach(IterableLike.scala:74)
	at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
	at scala.tools.nsc.backend.jvm.opt.CallGraph.addClass(CallGraph.scala:117)
	at scala.tools.nsc.backend.jvm.PostProcessor.$anonfun$runGlobalOptimizations$5(PostProcessor.scala:119)
	at scala.tools.nsc.backend.jvm.PostProcessor.$anonfun$runGlobalOptimizations$5$adapted(PostProcessor.scala:117)
	at scala.collection.TraversableLike$WithFilter.$anonfun$foreach$1(TraversableLike.scala:877)
	at scala.collection.immutable.List.foreach(List.scala:392)
	at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:876)
	at scala.tools.nsc.backend.jvm.PostProcessor.$anonfun$runGlobalOptimizations$3(PostProcessor.scala:117)
	at scala.tools.nsc.backend.jvm.PostProcessor.$anonfun$runGlobalOptimizations$3$adapted(PostProcessor.scala:117)
	at scala.collection.immutable.List.foreach(List.scala:392)
	at scala.tools.nsc.backend.jvm.PostProcessor.runGlobalOptimizations(PostProcessor.scala:117)
	at scala.tools.nsc.backend.jvm.GeneratedClassHandler$GlobalOptimisingGeneratedClassHandler.complete(GeneratedClassHandler.scala:92)
	at scala.tools.nsc.backend.jvm.GenBCode$BCodePhase.$anonfun$run$1(GenBCode.scala:81)
	at scala.tools.nsc.backend.jvm.GenBCode$BCodePhase.run(GenBCode.scala:78)
	at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1503)
	at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1487)
	at scala.tools.nsc.Global$Run.compileSources(Global.scala:1480)
	at scala.tools.nsc.Global$Run.compile(Global.scala:1606)
	at scala.tools.nsc.Driver.doCompile(Driver.scala:47)
	at scala.tools.nsc.MainClass.doCompile(Main.scala:32)
	at scala.tools.nsc.Driver.process(Driver.scala:67)
	at scala.tools.nsc.Driver.main(Driver.scala:80)
	at scala.tools.nsc.Main.main(Main.scala)

[SethTisue]

yeah, I should get on scala/community-build#873 :-/

[lrytz]

Need to upgrade asm to 7.1 (and backport that to 2.12).

[smarter]

Keeping up with Java bytecode versions is a never-ending task, given that the Scala compiler itself still produces Java 8 bytecode why is the optimizer reading these classfiles ? Can we get it to stop doing that, or at least gracefully degrade when it encounters an unknown bytecode version ?

[lrytz]

I have to remind myself why -opt:l:method (not inlining) reads classfiles from the classpath. But when you run the compiler on JDK 13, the optimizer comes across version 57 classfiles (from the JDK).

Generally, ASM version X supports classfiles up to version Y, and it would not be safe to just skip the check and use it to parse a classfile verison Y+1. There might be changes in semantics, new bytecodes.

[lrytz]

why -opt:l:method (not inlining) reads classfiles from the classpath

opt:l:method enables closure-invocations (rewrite (new FunctionN(..)).apply(...) to a call of the lambda body method). This optimization uses the call graph.

The optimizer is very eager when building the call graph. For a simple class C, the <init> method invokes java/lang/Object.<init>, which causes the Object classfile to be parsed (using ASM).

[smarter]

OK, but is the result of that parsed file used for anything ? Can we prevent ASM from trying to read it, or catch the exception and keep going ? Would that require patching ASM ?

[lrytz]

I think the best solution would be to make building the call graph more lazy.

[SethTisue]

@lrytz want to try to get a fix into 2.12.10, or shall we slip it to 2.12.11?

[lrytz]

2.12.11 seems fine, I'd say.

[xuwei-k]

asm 7.2 released (support jdk 14)
https://asm.ow2.io/versions.html

[SethTisue]

@lrytz do you intend to tackle this for 2.12.11/2.13.2?

[lrytz]

👍 thanks for the reminder

[mkurz]

ASM 8.0.1 available: https://asm.ow2.io/versions.html