macro regression in 2.12.5 when compiling on Java 9 or 10 ("macro implementation not found")

[SethTisue]

EDIT: Adding bug description here at the top now that we know more about it, and since lots of people are hitting this:

• error only occurs when compiling on Java 9 or 10. running on Java 9 or 10 is fine
• any code that calls macros is affected. doesn't matter what the macro is
• error message is usually "macro implementation not found"
  • but see akka-http ScalatestRouteTest example doesn't compile after update to 2.12.5 bug#10800 (comment) for an example of how a different error message can result instead
• there is no workaround. you must compile on Java 8 or wait for 2.12.6
• 2.12.6 status thread: https://contributors.scala-lang.org/t/scala-2-12-6-coming-soon/1729

====

original report follows:

on Java 9 https://gist.github.com/SethTisue/0681f6aa70662ecde47a4a842ed13fb6 gives

[info] Compiling 1 Scala source to /Users/tisue/tmp/20180320/target/scala-2.12/classes ...
[error] /Users/tisue/tmp/20180320/S.scala:5:4: macro implementation not found: macroTransform
[error] (the most common reason for that is that you cannot use macro implementations in the same compilation run that defines them)
[error]   @newtype case class WidgetId(toInt: Int)
[error]    ^

reported on Gitter by @CremboC

[SethTisue]

another, perhaps related report on Gitter from @plokhotnyuk

compiling https://github.com/Sizmek/fast-string-interpolator on 2.12.5 and Java 9 gives

[error] /Users/tisue/fast-string-interpolator/benchmark/src/main/scala/com/sizmek/fsi/benchmark/StringConcatenationBenchmark.scala:41:5: macro implementation not found: fast
[error] (the most common reason for that is that you cannot use macro implementations in the same compilation run that defines them)
[error]     fast"${int}xxx${long}xxx${float}xxx${double}xxx${char}xxx${boolean}xxx$string".toString
[error]     ^

[lrytz]

I was going to say it's most likely the macro/plugin classloader change (scala/scala#6314 / scala/scala#6400 / scala/scala#6412), but looking at the diffs, the code path with the cache disabled (whch ist the default) seems to be identical.

[hrhino]

It's because the classpath that it's looking at uses jrt:// urls. AbstractFile.getUrl returns null on those.

The salient line is in Macros#findMacroClassLoader:

val locations = classpath.map(u => Path(AbstractFile.getURL(u).file))

[lrytz]

But it seems to me locations is only used if the cache is enabled, which is not the case by default?

[hrhino]

It's only used in that case, but it's computed always.

The cheap fix is probably to make it lazy.

The slightly more generous fix is to intermediately map only AbstractFile.getURL(u) and disable caching if any are null. Maybe with a kind explanation of why we don't have these specific nice things yet.

[lrytz]

Ah, then .file throws an NPE, which is then caught by the macro engine. Thanks..

[jvican]

I have a fix on the way, thanks Harrison for the preliminary investigation. I'm not going to fix the fact that AbstractFile.getURL returns null for URLs starting with jrt. I'll leave that as a future improvement of Java 9 support in scalac.

[yanns]

The same with https://github.com/lightbend/scala-logging:

[error] <...>: macro implementation not found: debug
[error] (the most common reason for that is that you cannot use macro implementations in the same compilation run that defines them)
[error]           logger.debug(s"Fetched ${results.count} on $uri. Stop = $stop")
[error]                       ^

[SethTisue]

The same with https://github.com/lightbend/scala-logging

yeah, I think we can expect any macro to be affected.

[dragos]

Stepped on this as well using scalatest, @dotta can provide more info. getURL returns null on paths starting with file:// as well, it seems.. (on Java 8)

[dotta]

Stepped on this as well using scalatest, @dotta can provide more info. getURL returns null on paths starting with file:// as well, it seems.. (on Java 8)

Indeed. I don't have a reproducible, but if you manage to have a non-existing directory as part of the computed classpath, then AbstractFile.getURL(badDir) will return null.

[huntc]

Just adding that I stumbled upon this problem with utest.

[monksy]

This is causing Scala test to fail

t.scala:183: error: value should is not a member of Int
[ERROR]       actual should be(expected)

[unoexperto]

It still happens for me in 2.12.6 when compiling in IntelliJ IDEA (sbt 1.1.6) with circe json library. It compiles from commandline sbt compile though.

[SethTisue]

@unoexperto whatever problem you're having, I very much doubt it can have anything to do with this ticket besides happening to share an error message