java.lang.NullPointerException on project compilation with IBM J9 java 1.6

[lombokissues]

Migrated from Google Code (issue 554)

[lombokissues]

👤 tom.bujok   🕗 Jul 30, 2013 at 07:59 UTC

What steps will reproduce the problem?

1. mvn compile on a multi-module project
   2.
   3.

What is the expected output? What do you see instead?

What version of the product are you using? On what operating system?
0.12.0

Please provide any additional information below.
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ sr-grp-dto ---
[INFO] Compiling 3 source files to C:\projectX\target\classes
java.lang.NullPointerException
at java.util.IdentityHashMap$IdentityHashMapEntry.setValue(IdentityHashMap.java:131)
at lombok.javac.apt.Processor.process(Processor.java:263)
at lombok.core.AnnotationProcessor$JavacDescriptor.process(AnnotationProcessor.java:117)
at lombok.core.AnnotationProcessor.process(AnnotationProcessor.java:167)
at com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor(JavacProcessingEnvironment.java:639)
at com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs(JavacProcessingEnvironment.java:568)
at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:713)
at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:999)
at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:739)
at com.sun.tools.javac.main.Main.compile(Main.java:365)
at com.sun.tools.javac.main.Main.compile(Main.java:291)
at com.sun.tools.javac.main.Main.compile(Main.java:282)
at com.sun.tools.javac.Main.compile(Main.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at org.codehaus.plexus.compiler.javac.JavacCompiler.compileInProcess(JavacCompiler.java:554)
at org.codehaus.plexus.compiler.javac.JavacCompiler.compile(JavacCompiler.java:161)
at org.apache.maven.plugin.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:605)
at org.apache.maven.plugin.CompilerMojo.execute(CompilerMojo.java:128)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] Failure executing javac, but could not parse the error:

[lombokissues]

👤 r.spilker   🕗 Jul 30, 2013 at 09:34 UTC

Can you please provide a zip-file containing an example demonstrating the problem?

[lombokissues]

👤 r.spilker   🕗 Jul 30, 2013 at 09:36 UTC

Also, is this problem new in the 0.12.0, or was it also present in previous versions? Since what version is it broken? I personally use lombok 0.11.6 in a multi-module maven project without any problems.

[lombokissues]

👤 tom.bujok   🕗 Jul 30, 2013 at 10:49 UTC

Sorry, cannot attach zip, it's a big commercial project - it would be extremely difficult to externalize stuff.

I tested version 0.12.0, 0.11.8 and 0.11.6 and it fails.
It works on 0.11.4

BTW, I am using IBM SDK version javac 1.6.0_14

[lombokissues]

👤 tom.bujok   🕗 Jul 30, 2013 at 11:23 UTC

0.11.4 does not work either.
0.11.2 is the latest version that works for me...
Does it have sth to do with java version 5 vs. 6?

[lombokissues]

👤 tom.bujok   🕗 Jul 31, 2013 at 08:42 UTC

Any hints?

[lombokissues]

👤 reinierz   🕗 Aug 12, 2013 at 20:05 UTC

Java 6 is EOL or about to; I checked out the code in the java 7 version of IdentityHashMap.Entry's setValue method and it couldn't possibly throw an NPE. Also, line 131 is the javadoc of the class itself.

So, either (A) IdentityHashMap has been completely revamped between java 6 and java 7, or (B) you're using a very weird version of the runtime classes which has a bug in IdentityHashMap.Entry.setValue.

What happens if you update to java 7? You're going to have to sooner rather than later.

NB: I'm on a modern mac; I can't even run java6, so this is impossible for me to test. Roel doesn't have it on his PC either (though he can at least in theory install it).

[lombokissues]

👤 tom.bujok   🕗 Aug 12, 2013 at 20:52 UTC

Thanks for the answer. I understand your point of view. The question is more whether you want to support non-standard (non-oracle) or old environments. In the corporate world it's pretty common to use java 5 nowadays. These EJB 2.x are still running there.
I would not necessarily exclude all these people from the lombok community. I know of at least couple of projects running on Websphere 6.5 (IBM SDK 1.5) that use lombok.

That problem occurred on IBM SDK 1.5 (and 1.6). You can download it here to have a look, of course if you are willing to play with the non-oracle world...

[lombokissues]

👤 reinierz   🕗 Aug 12, 2013 at 22:36 UTC

Oh, sure, JRE5 and JRE6 are still extremely widespread, no argument here.

However, lombok is irrelevant at the deployment side of any application; lombok modifies the way javac (or jikes, I guess, if that's still a thing in IBM SDK, though, isn't that C code? I'm 100% positive lombok doesn't work at all in jikes) compiles, that is it.

You can compile with javac7, running in a JRE8, producing class files that you end up running on a JRE6. That's fine.

I'm saying: You should upgrade to javac7, and run javac7 on JRE7, producing class files which as far as we care are targeted at 1.4, let alone 1.5 or 1.6. So, on your development machine, have a JDK7 installed, and use its javac, using "-source 1.5 -target 1.5" to make sure it produces class files compatible with JRE5, to compile.

The issue is this: During a lombok operation, a call in java.util.IdentityHashMap fails. Now, as far as I know, unless harmony is involved, j.u.IHM is NOT AN IBM THING - it's just the same plain old sun/oracle-written class. Except it's screwed up somehow. Also, it's from the oracle JRE6's RT, which is very much EOL, and extremely hard for both me and Roel to test.

Note for example that if you were to use the javac that ships with JDK5, lombok doesn't work AT ALL, and never has. proper annotation processing wasn't a feature until javac 6.

So, in short:

• Run your compiler on JDK7 or above.
• Use the 'javac' that ships with JDK7 or above.
• Use -source 1.5 -target 1.5 or whatever you prefer. Heck, config the bootclasspath too just so that you know you aren't accidentally calling core library methods that didn't exist yet in 1.5, if you want.
• Run the produced class files on JRE5.

[lombokissues]

👤 tom.bujok   🕗 Aug 15, 2013 at 14:09 UTC

Fair enough. So I've got JDK 1.7 and the bug is still there. I attach a screenshot where you can see the message.

I am using sth like this:

org.apache.maven.plugins
maven-compiler-plugin
2.3.2

1.7
1.7

What's more, the compilation in the console hangs forever.

$ java -version
java version "1.7.0"
Java(TM) SE Runtime Environment (build pwa6470sr2-20120901_01(SR2))
IBM J9 VM (build 2.6, JRE 1.7.0 Windows 7 amd64-64 20120809_118929 (JIT enabled, AOT enabled)
J9VM - R26_Java726_SR2_20120809_0948_B118929
JIT - r11.b01_20120808_24925
GC - R26_Java726_SR2_20120809_0948_B118929
J9CL - 20120809_118929)
JCL - 20120831_02 based on Oracle 7u3-b05

[lombokissues]

👤 tom.bujok   🕗 Aug 15, 2013 at 14:09 UTC

[lombokissues]

👤 tom.bujok   🕗 Aug 15, 2013 at 14:20 UTC

Do you test Lombok at IBM's SKDs at all?

[lombokissues]

👤 setanta.mathews   🕗 Aug 28, 2013 at 10:54 UTC

For what it's worth, I'm having the same problem with J9.

[lombokissues]

👤 r.spilker   🕗 Aug 28, 2013 at 18:39 UTC

To answer the question: no we never tested with IBM's SDK. From the stacktrace I do see that something is not as expected. We'll have a go soon to try this out, but currently it is still holiday season over here. And we're also working hard on java8 support since the new Netbeans uses the java8 compiler internally, and that has higher priority.

[lombokissues]

👤 tom.bujok   🕗 Aug 29, 2013 at 09:01 UTC

Tomorrow we are organising a Hackergarten meeting with @ aalmiray and other geeks.
We will contribute some patches to selected open-source projects.

We could have a look at this issue. Do you think it's feasible to solve it in couple of hours?
If yes, could you write a bit more what to look at, how to investigate it, etc. to kick start us?

[lombokissues]

👤 setanta.mathews   🕗 Aug 29, 2013 at 11:50 UTC

It might be as simple as dealing with nulls instead of expecting empty arrays. See the following bug against spring.

https://jira.springsource.org/browse/SPR-10862

[lombokissues]

👤 r.spilker   🕗 Aug 29, 2013 at 12:08 UTC

That might be the case. In ecj we expects nulls, in javac empty lists.

What would be my strategy for tracking this down:

• Create a different jar file that has a simple normal annotation processor registered in META-INF/services and from it print the list of compilationunits. Put this class on the classpath for the compiler. See it that also triggers the NPE. (excludes annotation processors that encouter this problem)
• See if a manual compile (without maven) with lombok also triggers this NPE. (excludes maven-specific problems)
• Create a simple java file without any lombok, but compile a maven project with lombok. (excludes specific lombok transformations)
• Make System.getProperty("lombok.disable") return null, and test with and without maven.
• Try to put tools.jar from OpenJDK or Oracle on the boorclasspath

[lombokissues]

👤 r.spilker   🕗 Aug 29, 2013 at 12:16 UTC

Looking at the code, it might be as simple as inserting a check on lombok.javac.apt.Processor.java:213 to test if unwrapped == null and if it is return from the method.

[lombokissues]

👤 tom.bujok   🕗 Aug 29, 2013 at 12:19 UTC

Perfect. Thanks for the hints. Would be nice to have it done. We will try to tackle it... Do you have twitter? To ping you if we are stuck or sth?

[lombokissues]

👤 askoning   🕗 Oct 07, 2013 at 18:44 UTC

Issue #618 has been merged into this issue.

[lombokissues]

👤 askoning   🕗 Oct 07, 2013 at 18:46 UTC

From issue #618: maven isn't the problem. One down, N to go?

[lombokissues]

👤 r.spilker   🕗 Jan 20, 2014 at 02:45 UTC

I think this has been fixed already in f0cdc27. Can anyone confirm? We will be closing this issue unless someone speaks up within two weeks.

[lombokissues]

👤 r.spilker   🕗 Feb 28, 2014 at 01:01 UTC

[lombokissues]

👤 r.spilker   🕗 Jun 03, 2014 at 23:38 UTC

According to http://stackoverflow.com/questions/23812105/annotating-a-list-of-fields-instead-of-a-single-field-with-lombok-v-0-9-2 this bug is still open. A possible workaround is to replace the whole IdentityHashMap by a different map implementation or a ReferenceFieldAugment.

[lombokissues]

👤 r.spilker   🕗 Jun 03, 2014 at 23:46 UTC

Alternatively, we can use a regular HashMap with a key that holds a reference to the JCCompilationUnit and implements equals and hashcode by doing identity operations on it.

[lombokissues]

👤 Maaartinus   🕗 Jun 05, 2014 at 04:34 UTC

You might be able to use normal HashMap without any wrappers, since JCCompilationUnit doesn't override equals. As there's no explanation and I have no clue, I can offer some wild guesses: 1. The IdentityHashMap is broken and chokes on the nulls you're using as markers for no level; using some non-null placeholder value might help. 2. There's some multithreading and map is seen as null in a code like
http://grepcode.com/file/repo1.maven.org/maven2/org.robovm/robovm-rt/0.0.9/java/util/IdentityHashMap.java﹟124

[lombokissues]

👤 r.spilker   🕗 Jun 11, 2014 at 09:32 UTC

In the 1.14.2 we've changed the code a bit to at least no longer use IdentityHashMapEntry﹟setValue() but since we cannot reproduce the problem I have no idea if this fixes the problem.

[lombokissues]

👤 vzorge   🕗 Jun 11, 2014 at 13:53 UTC

Thanks you!
I am running with 1.14.2 now and all seems to be working good so far.

Running with IBM jdk 6 that comes with Websphere 7

Cheers,
Vincent

[lombokissues]

👤 r.spilker   🕗 Feb 06, 2015 at 12:50 UTC

Thanks for the feedback

[lombokissues]

End of migration