MethodHandles.Lookup.defineClass for CGLIB class definition purposes [SPR-15859] #20414
Comments
|
Alan Bateman commented Look up java.lang.invoke.MethodHandles.Lookup.defineClass, it's the supported way to inject classes into the same runtime package as another class. It is new in Java SE 9 so it means using a MR JAR or using reflective if you are compiling to an older release. Also note that Unsafe.defineClass is deprecated (forRemoval=true) so it will likely be removed at some point. |
|
Juergen Hoeller commented AlanBateman, the From my perspective, the However, for Spring Framework 5.0+, we could enforce a different default policy. In particular, we could ship a JDK 8/9 oriented CGLIB fork that uses |
|
Rafael Winterhalter commented The problem that we face in cglib is not the requirement to use cglib has existed since Java 1.1 and has a large use-base. At the same time, it is no longer under active development what led to people finding work-arounds for bugs. As a consequence, it is almost impossible to change anything without breaking things for many users which is why I am hesistant to apply such changes. In Byte Buddy, users can supply a Foo foo = mock(Foo.class, MethodHandles.lookup());for any mock class what would destroy the API's expressiveness and make field injection impossible. Therefore, we still rely on the class definition via As for now: I could add a system property to cglib that allows always using sun.misc.Unsafe which Spring could set before loading its first cglib-related class. Would this solve your problem? If you shade cglib, you automatically shade the property's name space such that this should be invisible to the outside. |
|
Alan Bateman commented If Spring could use Lookup.defineClass then it would avoid cglib needing to know about Lookup objects. |
|
Rafael Winterhalter commented Typically, Spring defines a proxy in the same package as a user class to access package-private methods. This would require getting hold of a lookup object provided by the user which would require an API change on Spring's side to get hold of such a lookup with the right privileges. |
|
Alan Bateman commented If the user module opens the package to Spring then Spring should be able obtain a full power Lookup with MethodHandles.privateLookupIn. |
|
Rafael Winterhalter commented That is true for bean classes that are defined and controlled by the user but not for classes that are provided by third-parties that do not neccessarily open their packages to Spring. This would require a command-line opening of those modules. That said, using a method handle will most likely work out for many cases but it is not a drop-in replacement. |
|
Alan Bateman commented If the developer of a library chooses to encapsulate its internals then that should be respected. If Spring is dependent on breaking into that library then it should work with the maintainer of the library to come up with a solution. |
|
Juergen Hoeller commented To be clear, what Rafael is referring to here is not that Spring is trying to break into any particular library but rather that Spring's generic AOP facilities can be used to create class-based proxies for arbitrary target classes... So some users may have chosen to apply AOP to third-party classes that are not fully opened, with the framework just doing its usual generic stuff about it. That said, I don't see this as a major problem. It's the user's responsibility to make sure that everything is properly opened in a module arrangement, according to the purposes of that particular application setup. And in most cases, all that really matters in AOP setup is the public API surface of that particular class, so even a public lookup handle should be sufficient. Beyond AOP, we're using |
|
Juergen Hoeller commented Hmm, I see that |
|
Mark Mielke commented
From my perspective... The difficulty here is that modules didn't exist prior to Java 9, and the concept of "developer of a library chooses to encapsulate its internals" was only a recommendation as it could not be enforced. We don't actually know who intended for the modules to be usable with Spring in the ways they might be used at present, and who didn't. All we really know, is that over several years certain the class and method access qualifiers, and the Spring implementation have evolved to a current state. Then, Java 9 came along and made modules restricted by default. This effectively broke compatibility, and requires the use of the "-add-opens" and other flags to restore elements of the previous long-standing behaviour. I don't disagree with modules themselves, but I think we should be looking at this problem as "now that compatibility has been broken, and things must be more explicit... which libraries need to change to enable their use in Java 9?" Spring is one of them. But, all these third party libraries are also a concern. And getting them all to open themselves up to Spring, or other AOP frameworks, is either going to be a chore, or a severe frustration which will result in "-add-opens" and other such flags being long-standing new best practices, or the module system relaxed in some way. It's a good opportunity to re-evaluate. However, the practical element may mean that even if everybody agrees that something should be done, it is still very difficult or impossible to do it (for example, third party libraries where the original authors are not willing to support Java 9 yet or ever?). But... that all said... when I read the Spring documentation around these topics, and the ability to use it against third party code, I remember having spider sensations on the back of my neck and thinking this didn't sound wise, and was likely to bite us. So, I believe in my case, we have never done this. I can't think of a single place in our code where we couldn't enable Spring to have access. I think this makes us lucky, though, and we have to consider those who read the same documentation, and thought it sounded like a great idea to take advantage of, and design their architecture around... |
|
Alan Bateman commented I don't understand the last comment. If existing libraries are used as modules then they work as automatic modules, no encapsulation. In any case, this issue is about Spring+CGLIB using the protected ClassLoader.defineClass method from the wrong context and seeing if the the Lookup.defineClass method can be used instead. Juergen - yes, it needs PACKAGE access because the resulting class will get package access. If the application has opened the package to spring then you can use privateLookupIn to get a full-power Lookup that includes PACKAGE access. |
|
Juergen Hoeller commented AlanBateman, Rafael Winterhalter, this sounds almost like the CGLIB |
|
Mark Mielke commented
I'm trying to say that it isn't like people have spent years deciding exactly where the module boundaries should be and they are definitely where they should be. The module boundaries have been approximate up until now, because there was no way to enforce them prior. With Java 9, they are now enforced, and this causes a requirement for re-evaluation on both consumer and producer side, and it will take some time to reach a stable state again where everybody agrees on where the boundaries should be. Back to your statement:
I don't think most people have carefully and meticulous chose with full awareness of impact. Java 9 is a surprise for most people still - both producer and consumer. And even if they did spend 5 minutes thinking about it when they published their module info, I don't think we're at a final resting point here where what the producer says is what goes. We're still at evaluating impact phase. :-( Anyways... I don't mean to distract from the CGLIB discussion. I just want to make sure we're not presuming some deeper well-developed thought on the part of the producers here. We all have a problem to solve here, and that is understanding how Java modules affect us now that they are finally here. |
|
Rafael Winterhalter commented I experienced the same problem with Mockito and the prototype I came up with requires users to register a lookup in a global mockito registry or falls back to a public lookup if no such instance is available. This is of course rather unsatisfactory as it requires more work to do less and we decidedto retain use of Unsafe, also since we need Objenesis for instantiation. Another issue comes with load time weaving where one often needs to define helper classes what can be done with lookups only when the classes are defined by the instrumented class as the Java agent is not provided a lookup for the instrumented class's package what I could image is also a problem for Spring when being used with AspectJ. |
|
Philippe Marschall commented Oracle just removed |
|
Juergen Hoeller commented Indeed, this escalates the issue. While things will still work on the classpath (as long as illegal reflection access is still permitted there - either by default or by a command-line flag), this totally prevents use on the module path, so we'll definitely do something about it in the Spring Framework 5.1 timeframe (GA before JDK 11). |
|
Juergen Hoeller commented I've patched our CGLIB fork accordingly, passing a This seems to work fine so far, in particular getting rid of the warnings on the classpath. It has some limitations in that it can't enforce creation of the proxy class in a sub |
|
Rafael Winterhalter commented On a side note, this solution does not really work for cglib as a library if the library is not shaded. Using this loading strategy in a modularized would require the user to explicitly privilege cglib (https://mydailyjava.blogspot.no/2018/04/jdk-11-and-proxies-in-world-past.html) what breaks module encapsulation if the use of the library is an implementation detail. I do not think that developing cglib any further makes sense at this point, especially with some rather fundamental changes in the near future to adjust to the new byte codes for value types etc. Cglib has been dormant for a while and this has been communicated and I think that this is a good moment to discontinue development. If you want to continue using it, I understand that but you would need to maintain the library by yourself from here. |
|
Steven Pearce commented I'm unsure whether I should log this here or under #20937, but I'm still seeing a variety of this error in 5.1.0.BUILD-SNAPSHOT taken on the 16th August, against JDK 11. Stack trace with --illegal-access=debug is below
|
|
Juergen Hoeller commented That's a class from Spring Data triggering this, so please report it there. They got DATACMNS-1373 already, not sure it's meant to cover this case as well. |
|
Steven Pearce commented Thanks, have logged it there now. |
|
Josh Long commented I'm still seeing this issue in my Java 11 application (openjdk version "11" 2018-09-25; OpenJDK Runtime Environment 18.9 (build 11+28); OpenJDK 64-Bit Server VM 18.9 (build 11+28, mixed mode)) and I don't have Spring Data on the classpath. Here's the error: [INFO] Scanning for projects... Downloaded from central: https://cloudnativejava.artifactoryonline.com/cloudnativejava/plugins-release/org/jooq/jooq-codegen-maven/3.11.5/jooq-codegen-maven-3.11.5.pom (3.5 kB at 6.1 kB/s) Downloaded from central: https://cloudnativejava.artifactoryonline.com/cloudnativejava/plugins-release/org/jooq/jooq-parent/3.11.5/jooq-parent-3.11.5.pom (11 kB at 61 kB/s) Downloaded from central: https://cloudnativejava.artifactoryonline.com/cloudnativejava/plugins-release/org/jooq/jooq-codegen-maven/3.11.5/jooq-codegen-maven-3.11.5.jar (16 kB at 90 kB/s) |
|
Philippe Marschall commented @Josh Long Unfortunately the stack trace is missing but from
|
|
Juergen Hoeller commented Josh Long, indeed, this one doesn't seem related to CGLIB. It rather seems to be some other code trying to reflectively call a non-accessible |
|
Rafael Winterhalter commented Josh Long, you can run your JVM while setting --illegal-access=debug to get the stack trace that would reveal the caller of the method. |
|
Larkin Lowrey commented Here's my stack trace: |
|
Oliver Drotbohm commented Larkin Lowrey – This one in particular is being tracked in DATACMNS-1401. |
|
Marcel commented I still get this error: WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.springframework.util.ReflectionUtils (file:/some/path/WEB-INF/lib/spring-core-5.1.2.RELEASE.jar) to field java.sql.Timestamp.nanos
WARNING: Please consider reporting this to the maintainers of org.springframework.util.ReflectionUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
|
|
Juergen Hoeller commented L., your particular warning is not related to the ticket here. Some code uses Spring's |
|
Alan Bateman commented
I can't think why anything would want to access that field directly as the setNanos/getNanos methods are public. |
spring-projects-issues
added
type: enhancement
|
I am using JDK 12. Spring Boot 2.1.3.RELEASE is ok, but Sprin Boot 2.1.4.RELEASE and newer cause error
This is my dependencies tree https://gist.github.com/donhuvy/54d64b3abc9d29d392bbd22018ea3c66 |
|
I am seeing this on spring boot 2.1.6.RELEASE as well in a JDK11 environment |
|
@spring-issuemaster I'm seeing this on spring boot 2.1.7.RELEASE as well with openjdk 11.0.1 |
|
@valentin-nasta, the stack trace you have provided shows that this is coming from Apache Wicket's direct use of CGLIB, not from Spring. So please open an issue with the Apache Wicket team. |
|
@sbrannen That's correct, I'll open an issue with the Apache Wicket team, thank you! |
On OpenJDK 10.0.2 using |
|
I hope this is the right place to add my comment even if the issue is closed. I have the same thing on Java 11 with Spring 5.2.3 What can I do about it? |
|
When With Then |
Juergen Hoeller opened SPR-15859 and commented
As discussed in cglib/cglib@d6fe1d8 and in #20245 comments, there is currently a
defineClasswarning triggered by CGLIB when running on JDK 9 in classpath mode. While there are workarounds for it ("illegal-access=deny" or "add-opens java.base/java.lang=ALL-UNNAMED"), suppressing that warning at runtime, it'd be nice to avoid the warning completely when running on JDK 9, possibly through a specific check for JDK 9 which skips theClassLoader.defineClassaccess attempt completely, always going with theUnsafe.defineClassfallback right away in such a scenario. We have yet to see whether this can be patched in CGLIB itself or just in Spring's CGLIB fork.UPDATE: Since JDK 11 won't have
Unsafe.defineClassat all anymore, we need to useMethodHandles.Lookup.defineClassas our primary mechanism, avoiding aClassLoader.defineClasswarning on the classpath and providing compatibility with the module path on JDK 11.Affects: 5.0 RC3
Reference URL: cglib/cglib@d6fe1d8
Issue Links:
@Configurationusing Java 9 prints ugly illegal access warnings ("is duplicated by")Referenced from: commits 6a34ca2, 61c3db0
17 votes, 40 watchers
The text was updated successfully, but these errors were encountered: