VerifyErrors when using SpEL compilation with Thymeleaf [SPR-12271]

[spring-projects-issues]

Joris Kuipers opened SPR-12271 and commented

I tried to enable SpEL compilation support in my project using -Dspring.expression.compiler.mode=immediate. This project uses Thymeleaf for views, and thus relies rather heavily on SpEL expressions.
When running the application I can see that the compiler kicks in, but I'm seeing VerifyErrors on the generated SpEL classes:

Caused by: java.lang.VerifyError: (class: spel/Ex7, method: getValue signature: (Ljava/lang/Object;Lorg/springframework/expression/EvaluationContext;)Ljava/lang/Object;) Expecting to find integer on stack
	at java.lang.Class.getDeclaredConstructors0(Native Method)
	at java.lang.Class.privateGetDeclaredConstructors(Class.java:2532)
	at java.lang.Class.getConstructor0(Class.java:2842)
	at java.lang.Class.newInstance(Class.java:345)
	at org.springframework.expression.spel.standard.SpelCompiler.compile(SpelCompiler.java:106)
	at org.springframework.expression.spel.standard.SpelExpression.compileExpression(SpelExpression.java:464)
	at org.springframework.expression.spel.standard.SpelExpression.checkCompile(SpelExpression.java:434)
	at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:266)
	at org.thymeleaf.spring4.expression.SpelVariableExpressionEvaluator.evaluate(SpelVariableExpressionEvaluator.java:139)
	at org.thymeleaf.standard.expression.VariableExpression.executeVariable(VariableExpression.java:149)
	at org.thymeleaf.standard.expression.SimpleExpression.executeSimple(SimpleExpression.java:59)
	at org.thymeleaf.standard.expression.Expression.execute(Expression.java:103)
	at org.thymeleaf.standard.expression.Expression.execute(Expression.java:133)
	at org.thymeleaf.standard.expression.Expression.execute(Expression.java:120)
	at org.thymeleaf.standard.processor.attr.AbstractStandardSingleAttributeModifierAttrProcessor.getTargetAttributeValue(AbstractStandardSingleAttributeModifierAttrProcessor.java:67)
	at org.thymeleaf.processor.attr.AbstractSingleAttributeModifierAttrProcessor.getModifiedAttributeValues(AbstractSingleAttributeModifierAttrProcessor.java:59)
	at org.thymeleaf.processor.attr.AbstractAttributeModifierAttrProcessor.processAttribute(AbstractAttributeModifierAttrProcessor.java:61)
	at org.thymeleaf.processor.attr.AbstractAttrProcessor.doProcess(AbstractAttrProcessor.java:87)
	at org.thymeleaf.processor.AbstractProcessor.process(AbstractProcessor.java:212)
	at org.thymeleaf.dom.Node.applyNextProcessor(Node.java:1016)
	at org.thymeleaf.dom.Node.processNode(Node.java:971)
	at org.thymeleaf.dom.NestableNode.computeNextChild(NestableNode.java:672)
	at org.thymeleaf.dom.NestableNode.doAdditionalProcess(NestableNode.java:655)
	at org.thymeleaf.dom.Node.processNode(Node.java:990)
	at org.thymeleaf.dom.NestableNode.computeNextChild(NestableNode.java:672)
	at org.thymeleaf.dom.NestableNode.doAdditionalProcess(NestableNode.java:655)
	at org.thymeleaf.dom.Node.processNode(Node.java:990)
	at org.thymeleaf.dom.NestableNode.computeNextChild(NestableNode.java:672)
	at org.thymeleaf.dom.NestableNode.doAdditionalProcess(NestableNode.java:655)
	at org.thymeleaf.dom.Node.processNode(Node.java:990)
	at org.thymeleaf.dom.NestableNode.computeNextChild(NestableNode.java:672)
	at org.thymeleaf.dom.NestableNode.doAdditionalProcess(NestableNode.java:655)
	at org.thymeleaf.dom.Node.processNode(Node.java:990)
	at org.thymeleaf.dom.NestableNode.computeNextChild(NestableNode.java:672)
	at org.thymeleaf.dom.NestableNode.doAdditionalProcess(NestableNode.java:655)
	at org.thymeleaf.dom.Node.processNode(Node.java:990)
	at org.thymeleaf.dom.NestableNode.computeNextChild(NestableNode.java:672)
	at org.thymeleaf.dom.NestableNode.doAdditionalProcess(NestableNode.java:655)
	at org.thymeleaf.dom.Node.processNode(Node.java:990)
	at org.thymeleaf.dom.NestableNode.computeNextChild(NestableNode.java:672)
	at org.thymeleaf.dom.NestableNode.doAdditionalProcess(NestableNode.java:655)
	at org.thymeleaf.dom.Node.processNode(Node.java:990)
	at org.thymeleaf.dom.NestableNode.computeNextChild(NestableNode.java:672)
	at org.thymeleaf.dom.NestableNode.doAdditionalProcess(NestableNode.java:655)
	at org.thymeleaf.dom.Node.processNode(Node.java:990)
	at org.thymeleaf.dom.NestableNode.computeNextChild(NestableNode.java:672)
	at org.thymeleaf.dom.NestableNode.doAdditionalProcess(NestableNode.java:655)
	at org.thymeleaf.dom.Node.processNode(Node.java:990)
	at org.thymeleaf.dom.Document.process(Document.java:93)
	at org.thymeleaf.TemplateEngine.process(TemplateEngine.java:1155)
	at org.thymeleaf.TemplateEngine.process(TemplateEngine.java:1060)
	at org.thymeleaf.TemplateEngine.process(TemplateEngine.java:1011)
	at org.thymeleaf.spring4.view.ThymeleafView.renderFragment(ThymeleafView.java:335)
	at org.thymeleaf.spring4.view.ThymeleafView.render(ThymeleafView.java:190)
	at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1228)
	at org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1011)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:955)
	... 68 more

I haven't tried to dig any deeper into the root cause, as byte code generation isn't really my cup of tea.

Thymeleaf has a lot of runtime overhead compared to JPSs or FreeMarker: if Spring could support compiled SpEL expressions in Thymeleaf views I think it could have a very positive impact on performance, so I'm eager to get this to work.

---

Affects: 4.1 GA

Attachments:

• SPR-12271.zip (5.03 kB)

Issue Links:

• VerifyError when trying to compile constructor invocation with SpEL [SPR-12326] #16931 VerifyError when trying to compile constructor invocation with SpEL

Referenced from: commits bd7d56a

[spring-projects-issues]

Joris Kuipers commented

Updated stack trace after switching to thymleaf-spring4 to rule that out as a possible cause

[spring-projects-issues]

Juergen Hoeller commented

Andy Clement, any idea for the root cause of that VerifyError?

Juergen

[spring-projects-issues]

Andy Clement commented

I probably need to see the particular expression. Can we get a quick demo project together that contains the failure, then I'll take a look? If it was a representative example of thymeleaf in a project i can also fix up any other compile issues that exist in it at the same time (if there are any).

[spring-projects-issues]

Joris Kuipers commented

That sounds like a good plan: I'll try to whip up a small project (can probably use Spring Boot for that) that uses Thymeleaf with some representative SpEL expressions and will make sure that I can reproduce the error with that.
Thanks for the quick response!

[spring-projects-issues]

Joris Kuipers commented

I've attached a Spring Boot app that uses a Thymeleaf view with several expressions (it's secured with username 'user' and password 'secret', so that I could test a Security SpEL expression as well).
However, I haven't been able to reproduce the VerifyError yet. I do notice in the logging output that most SpEL expressions aren't actually compiled at all, even though they're just performing simple property access. Could this have something to do with the way that Thymeleaf integrates with SpEL?

[spring-projects-issues]

Joris Kuipers commented

OK, did a little more investigation using the problematic app and found the expression that's causing the problem: it's reproducable.
If you add this line to the home.html file in my attached application you can reproduce the problem:

This one causes a VerifyError: <span th:text="${#strings.contains(#httpServletRequest.servletPath, 'organisation') ? 'active' : ''}">boom</span>.

[spring-projects-issues]

Andy Clement commented

Not all expressions are compilable right now. When they aren't it should just silently not compile them - if it makes a mistake in that decision process and produces a class anyway it can lead to a verify error. Initial effort has been spent on compiling specific kinds of expression that benefited some Spring Integration/XD environments, but the framework is there to add more over time. Let me find out what the verify error is - thanks for the test case. I'll also take a look at what is or is not compilable of the set in this sample.

[spring-projects-issues]

Joris Kuipers commented

Thanks. BTW, looking at the code that determines if an expression is compilable I got the impression that Thymeleaf should change their code to ensure that something like simple property access (pretty common in a web view) would be compilable. Any tips on doing that (in a Spring 4.0-compatible manner) would be much appreciated as well, I can file some issues in their JIRA for that.

[spring-projects-issues]

Andy Clement commented

What an exciting bunch of expressions in there :) With a variety of reasons for not compiling. I breakpointed on expression compilation:

hasAnyRole('ROLE_USER')
-> not compilable because hasAnyRole is a varargs method and the compiler does not yet support this (because there is a bunch of byte code we have to generate to generate the array from the parameters that are actually passed). This is something we can address in Spring.

@testService.someProperty
-> bean references are not compilable. No performance use case has needed them so far. Probably not to difficult to support, we can address this in Spring.

@testService.findSomeInteger()
-> bean references are not compilable.

person.name.firstName
-> VariablesMapPropertyAccessor is not compilable, it is a custom accessor. If it implemented CompilablePropertyAccessor rather than just PropertyAccessor we would call it to generate byte code. I've been wondering if there is a more general (though probably slower performing) solution to this we could do in Spring. Probably worth putting a bit of thought into this before forcing them to implement CompilablePropertyAccessor.

person.getName().lastName
-> VariablesMapPropertyAccessor is not compilable, it is a custom accessor.

person.gender
-> VariablesMapPropertyAccessor is not compilable, it is a custom accessor.

#strings.contains(#httpServletRequest.servletPath,'organisation')?'active':''
-> VerifyError because we aren't handling the situation where the condition of the ternary returns a boxed Boolean and not a primitive boolean. The 'isCompilable()' check allows it through (it copes with 'Boolean' or 'boolean') but the generateCode stuff doesn't handle it. I'll fix it in Spring now.

[spring-projects-issues]

Andy Clement commented

To turn our regular map accessor into a CompilablePropertyAccessor we added these methods:

public boolean isCompilable() {
  return true; 
}

public Class<?> getPropertyType() {
  return Object.class; // the type of the objects coming out of the map
}

public void generateCode(String propertyName, MethodVisitor mv, CodeFlow codeflow) {
  String descriptor = codeflow.lastDescriptor();
  if (descriptor == null) { // If there is no descriptor the map access is the first thing in this expression so load the context object
    codeflow.loadTarget(mv);
  }
  mv.visitLdcInsn(propertyName);
  mv.visitMethodInsn(INVOKEINTERFACE, "java/util/Map", "get","(Ljava/lang/Object;)Ljava/lang/Object;",true);
}

[spring-projects-issues]

Andy Clement commented

I just pushed the fixes to address the problem expression:

#strings.contains(#httpServletRequest.servletPath,'organisation')?'active':''

In fact there were two problems:

• the ternary condition resulting in a Boolean. We now generate the necessary unboxing code
• an IllegalAccessError because the result of accessing #httpServletRequest is actually an instance of a non public class (although getServletPath() is not on that non public type). If we generated code to do a class cast to the non public type we get an IllegalAccessError. I've made the code smarter on variable accessing to avoid that.

With these changes in place that expression compiles.

[spring-projects-issues]

Juergen Hoeller commented

Thanks for the thorough analysis and quick turnaround, Andy! I'll mark this as resolved for 4.1.2 then. We still got a few weeks for further testing before the release...

Juergen

[spring-projects-issues]

Joris Kuipers commented

Nice, glad that the issues could be fixed. I guess for now enabling this support for Thymeleaf views will probably not result in much of a speed improvement given that most expressions will not be compiled, but at least now I can further investigate if this is indeed the case :)

[spring-projects-issues]

Andy Clement commented

Hey Joris - do we have any measurements to verify that compiling these expressions will make a difference? Or are we just hoping it will? Just asking before I dive into spending time compiling these kinds of expression, I'd hate to do loads of work and it to have no appreciable effect.

[spring-projects-issues]

Joris Kuipers commented

Hi Andy,

Thymeleaf view rendering in general is quite slow compared to something like JSPs or Freemarker. Some initial testing using the new Java Flight Recorder suggested that expression evaluation was contributing to this, but I haven't done proper profiling yet. I'll see if I can get you some numbers today.