Lombok APT Fail with Gradle v4.9

[aplatypus]

I started to get this the build error:

• lombok.javac.apt.LombokProcessor could not be initialized. Lombok will not run during this compilation: java.lang.ClassCastException

... following an upgrade to Gradle v4.9.

I have this ERROR using Lombok with Gradle on Linux. On the command line. I rolled back my project to Monday (today being Friday). Same resultafter reverting my project to Monday 's check-point.

I moved my Gradle version back to Gradle v4.8 using SDKMAN and ran the same build command.

• It works!!

Something between Gradle 4.8 and Gradle 4.9 is breaking a very straightforward Lombok build.

As the error trace reports issues stemming from the JavacAnnotationHandler ... I begin by reporting this here. Apparently this has occurred before but people were looking at IDE plug-in problems, etc. Running my builds on the command line eliminated 99% of that noise.

The conclusion is that there is a misalignment between gradle-apt-plugin v0.17 and gradle v4.9.

• Gradle v4.8 works.

And that means it is possible that the Gradle v4.9 broke my build.

• There is noting I could find in the Gradle v4.9 Release nodes covering known APT issues.

Even though it is Obviously a Gradle change causing this ERROR, the root cause appears to be a APT processing error/mismatch. For instance, I had a day and a half of fun organising the publish functionality to work with the upgrade.

From my perspective, I feel it is quite important for the Gradle project to Inform plugin providers about these structural changes, BEFORE the release. At least that way you can provide some kind of 'Warning' message.

2nd problem

This message is emitted as a "Warning". This situation is a definite, certified ERROR, None of my source code that used Lombok compiles. It is a project Build / Run / Test show-stopper. No compilation. Not successful build.

To be fair, I thing Lombok is reporting a "Warning" ... imho, when an annotation processor reports that it: "Can Not process the code"; this is a (project build) Fatal Error.

configuration:

O/S name:       Linux
Gradle version: 4.9
Groovy version: 2.4.12
Java version:   1.8.0_172-b11 (vendor: Oracle Corporation)

Root project:   'root project 'foundation''
  JAVA_HOME   = '/usr/lib/java/jre/home'
  target...
    JRE_HOME  = '/usr/lib/java/jre/home'
    JDK_HOME  = '/usr/lib/java/jdk/home'

net.ltgt.apt = v0.17
Lombok       = v18.0

related:

• #1046 -- ClassCastException: Cannot cast HandleFieldDefaults to JavacASTVisitor (IntelliJ)
• #197 -- Cannot cast lombok.javac.handlers.HandleFieldDefaults to lombok.javac.JavacAnnotationHandler

stackdump (see attachment for details):

warning: lombok.javac.apt.LombokProcessor could not be initialized. Lombok will not run during this compilation: java.lang.ClassCastException: org.gradle.api.internal.tasks.compile.processing.IncrementalFiler cannot be cast to com.sun.tools.javac.processing.JavacFiler
public class Rpt
       ^
        at lombok.javac.apt.LombokProcessor.getJavacFiler(LombokProcessor.java:447)
        at lombok.javac.apt.LombokProcessor.init(LombokProcessor.java:90)
        at lombok.core.AnnotationProcessor$JavacDescriptor.want(AnnotationProcessor.java:112)
        at lombok.core.AnnotationProcessor.init(AnnotationProcessor.java:165)
        at lombok.launch.AnnotationProcessorHider$AnnotationProcessor.init(AnnotationProcessor.java:73)
        at org.gradle.api.internal.tasks.compile.processing.DelegatingProcessor.init(DelegatingProcessor.java:57)
        at org.gradle.api.internal.tasks.compile.processing.IsolatingProcessor.init(IsolatingProcessor.java:44)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment$ProcessorState.<init>(JavacProcessingEnvironment.java:500)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment$DiscoveredProcessors$ProcessorStateIterator.next(JavacProcessingEnvironment.java:597)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs(JavacProcessingEnvironment.java:690)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.access$1800(JavacProcessingEnvironment.java:91)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(JavacProcessingEnvironment.java:1035)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1176)
        at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1170)
        at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:856)
        at com.sun.tools.javac.main.Main.compile(Main.java:523)
        at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129)
        at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138)
        at org.gradle.api.internal.tasks.compile.AnnotationProcessingCompileTask.call(AnnotationProcessingCompileTask.java:85)
        at org.gradle.api.internal.tasks.compile.ResourceCleaningCompilationTask.call(ResourceCleaningCompilationTask.java:53)
        at org.gradle.api.internal.tasks.compile.JdkJavaCompiler.execute(JdkJavaCompiler.java:50)
        at org.gradle.api.internal.tasks.compile.JdkJavaCompiler.execute(JdkJavaCompiler.java:36)
        at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.delegateAndHandleErrors(NormalizingJavaCompiler.java:100)
        at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.execute(NormalizingJavaCompiler.java:52)
        at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.execute(NormalizingJavaCompiler.java:38)
        at org.gradle.api.internal.tasks.compile.AnnotationProcessorDiscoveringCompiler.execute(AnnotationProcessorDiscoveringCompiler.java:49)
        at org.gradle.api.internal.tasks.compile.AnnotationProcessorDiscoveringCompiler.execute(AnnotationProcessorDiscoveringCompiler.java:35)
        at org.gradle.api.internal.tasks.compile.CleaningJavaCompilerSupport.execute(CleaningJavaCompilerSupport.java:40)
        at org.gradle.api.internal.tasks.compile.CleaningJavaCompilerSupport.execute(CleaningJavaCompilerSupport.java:27)
        at org.gradle.api.tasks.compile.JavaCompile.performCompilation(JavaCompile.java:156)
        at org.gradle.api.tasks.compile.JavaCompile.compile(JavaCompile.java:141)
        at org.gradle.api.tasks.compile.JavaCompile.compile(JavaCompile.java:113)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        :
     etc.

compileJava-FAILED.txt

[tbroyer]

This is completely unrelated to the net.ltgt.apt plugin: with Gradle 4.9 (and 4.8, back to 4.6), it basically only configures the options.annotationProcessorGeneratedSourcesDirectory of the JavaCompile tasks (it also adds the various DSLs, and the legacy apt configurations), as everything required (annotationProcessor configurations, etc.) is provided by Gradle proper. If you don't use the plugin's DSL, try removing the plugin and run your build again; the only difference would be that no generated source is written to disk.

That said, this exception points at Lombok code itself. Lombok is "hacking" into the compiler using internal and unsupported APIs, and assuming a certain runtime environment. Gradle itself changes that environment for its "incremental annotation processing" feature, but still following the API contracts for annotation processors. Lombok is not an annotation processor though, it just masquerades as one to hook into the mechanism as a way to "inject" itself into the compilation/compiler. So first make sure you're using the very last version of Lombok; I know they already had to make a change for Gradle "incremental annotation processing", and maybe another one was required for Gradle 4.9.

(as a side note, I'd recommend using the Gradle wrapper to "lock" a version of Gradle that's known to be "compatible" with your build rather than using external installations, through SDKMAN, distribution packages, or whatever)

[tbroyer]

Looks like projectlombok/lombok#1782

[aplatypus]

Ay ya!

Side note advised. For me builds shall-be: specific / reproducible / traceable. Where I am.

Anyway personally speaking, imho ...

1. Every bug has a home and
2. Every bug has an owner.

The Lombok v18.0 is the latest build with Maven central.

• also that Lombok version has been the same for over two months.

If the problem is elsewhere, say -- with Gradle v4.9(??) then we should identify that.

[rzwitserloot]

We just released lombok v1.18.2 which is the first release version with the fix from our end in it.

[aplatypus]

Thanks @rzwitserloot ...

Different compile or project problem. There is a different issue. The resulting message is more useful.

warning: Can't initialize javac processor due to (most likely) a class loader problem: java.lang.NoClassDefFoundError: com/sun/tools/javac/processing/JavacProcessingEnvironment
public abstract class LibraryTestContext        extends TestContext
                                     ^
        at lombok.javac.apt.LombokProcessor.getJavacProcessingEnvironment(LombokProcessor.java:413)
        at lombok.javac.apt.LombokProcessor.init(LombokProcessor.java:89)
        at lombok.core.AnnotationProcessor$JavacDescriptor.want(AnnotationProcessor.java:128)
        at lombok.core.AnnotationProcessor.init(AnnotationProcessor.java:181)
        at lombok.launch.AnnotationProcessorHider$AnnotationProcessor.init(AnnotationProcessor.java:73)
        at org.gradle.api.internal.tasks.compile.processing.DelegatingProcessor.init(DelegatingProcessor.java:57)
        at org.gradle.api.internal.tasks.compile.processing.IsolatingProcessor.init(IsolatingProcessor.java:44)
   :
  etc.

Class loading problem -- sounds interesting .... BUT !!!

• I opened the Java class: public abstract class LibraryTestContext file.

@Slf4j
ublic abstract class LibraryTestContext	extends	TestContext
{
    :
}

• It is plain old Java; No Groovy mentioned.
• The parent class is Java

AND but ...

• This class only uses the @slf4j annotation.
• There seems to me to be a project problem or a Gradle problem
  • It builds fine with Gradle 4.8.1.

Other other peer projects use @slf4j and build OK with Gradle 4.9. I must do some digging.

[rzwitserloot]

Looks like gradle now invokes the annotation processors with its own modular classloader system. There's no coming back from that. If team gradle insists this is the best way forward, henceforth gradle and lombok cannot be used together then. If it truly is this, the fix from our side is technically perhaps impossible, and even if it is possible, a huge project (we'd have to basically use reflection for _everything_. We'd probably move forward by writing a tool to reflect-ify an entire directory full of class files, and the performance is just one of many big worries with that kind of drastic move). Roel and I will try to make some time to look at this, reproduce, etc. It'll be a little while I'm afraid.

…

--Reinier Zwitserloot

On Sat, 28 Jul 2018 at 14:45, will ***@***.***> wrote: Thanks @rzwitserloot <https://github.com/rzwitserloot> ... Same outcome, the class itself fails to compile. There is a different issue. The resulting message is more useful. warning: Can't initialize javac processor due to (most likely) a class loader problem: java.lang.NoClassDefFoundError: com/sun/tools/javac/processing/JavacProcessingEnvironment public abstract class LibraryTestContext extends TmacsTestContext ^ at lombok.javac.apt.LombokProcessor.getJavacProcessingEnvironment(LombokProcessor.java:413) at lombok.javac.apt.LombokProcessor.init(LombokProcessor.java:89) at lombok.core.AnnotationProcessor$JavacDescriptor.want(AnnotationProcessor.java:128) at lombok.core.AnnotationProcessor.init(AnnotationProcessor.java:181) at lombok.launch.AnnotationProcessorHider$AnnotationProcessor.init(AnnotationProcessor.java:73) at org.gradle.api.internal.tasks.compile.processing.DelegatingProcessor.init(DelegatingProcessor.java:57) at org.gradle.api.internal.tasks.compile.processing.IsolatingProcessor.init(IsolatingProcessor.java:44) : etc. Class loading problem -- sounds interesting .... *BUT* *!!!* - I opened the Java class: public abstract class *LibraryTestContext* file. - It is plain old Java; No Groovy mentioned. - The build fails because AND *but* ... - This class has *NO* Lombok processing - There seems to *me* to be a Gradle problem - It builds fine with Gradle 4.8.1. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#87 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAFsd1Tl5tGPTD4T9I1YhyZB2cfmRR8Eks5uLFzrgaJpZM4VYJui> .

[tbroyer]

Out of curiosity, could Lombok work as a javac plugin? (the kind loaded with an -Xplugin:Lombok)

[rzwitserloot]

@tbroyer yeah probably; looked at the API and you get basically the same classes as annotation processors get. Because of that I'm not entirely sure what the point of the plugin API is (not sure why it is needed given that annotation processors are a thing), but if you think this would solve some gradle issues that sounds like a good direction to go into for fixing things.

[tbroyer]

Because of that I'm not entirely sure what the point of the plugin API is (not sure why it is needed given that annotation processors are a thing)

@rzwitserloot I'd say that's basically the difference between abusing an unrelated API (annotation processors are supposed to generate new classes, not modifying the internal compiler state or whatever Lombok actually does) and relying on implementation assumptions (exactly what currently breaks Lombok in Gradle: they respect the interface contracts, but you're assuming you can actually cast the instance to a specific javac class) vs. using a supported API designed (AFAIK) specifically to support your use-case 🤷‍♂️

[rzwitserloot]

The plugin API doesn't let you really do anything that an annotation processor can't already do: There's nothing in the interfaces returned there which lets you modify existing classes, either. Hence why I wonder what the point of this is.

Just like with the annotation processor API (Because it's mostly the same types!) you can presume certain implementations, cast to them (or reflect into them) and do things the interfaces won't let you do.

[tbroyer]

Oh 🤔 I stand corrected then (I never actually looked at the plugin API, hadn't noticed it's for analysis only), but I still think a plugin would be a better fit 🤷‍♂️

[rzwitserloot]

Yeah, probably! At the time we wrote lombok it wasn't around, and at least up to java 8, annotation processors on the classpath are just.. run. No need for extra parameters. Trying to tell maven to inject an -X parameter in javac is not easy at all, and given that the APIs are virtually identical, even today I'd say lombok being an AP is the better general mode.

modules change things a bit. The build tools are catching up somewhat. I guess the day where lombok should prefer to be a plugin instead of an AP is coming closer, but it's not here quite yet.