Support Snakeyaml 1.20+

[twicksell]

When running Spring Boot 2.0.2 we see the following error when Snakeyaml 1.20 is on the classpath. This occurs when we do not use the dependency management plugin, and have Spring Framework 5.0.4+ on the classpath which introduced Snakeyaml 1.20.

java.lang.NoSuchMethodError: org.yaml.snakeyaml.nodes.ScalarNode.getStyle()Ljava/lang/Character;
	at org.springframework.boot.env.OriginTrackedYamlLoader$KeyScalarNode.<init>(OriginTrackedYamlLoader.java:127)
	at org.springframework.boot.env.OriginTrackedYamlLoader$KeyScalarNode.get(OriginTrackedYamlLoader.java:138)
	at org.springframework.boot.env.OriginTrackedYamlLoader$KeyScalarNode.get(OriginTrackedYamlLoader.java:133)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
	at org.springframework.boot.env.OriginTrackedYamlLoader$OriginTrackingConstructor.replaceMappingNodeKeys(OriginTrackedYamlLoader.java:99)
	at org.springframework.boot.env.OriginTrackedYamlLoader$OriginTrackingConstructor.constructObject(OriginTrackedYamlLoader.java:92)
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructDocument(BaseConstructor.java:161)
	at org.yaml.snakeyaml.constructor.BaseConstructor.getData(BaseConstructor.java:127)
	at org.yaml.snakeyaml.Yaml$1.next(Yaml.java:547)
	at org.springframework.beans.factory.config.YamlProcessor.process(YamlProcessor.java:168)
	at org.springframework.beans.factory.config.YamlProcessor.process(YamlProcessor.java:141)
	at org.springframework.boot.env.OriginTrackedYamlLoader.load(OriginTrackedYamlLoader.java:75)
	at org.springframework.boot.env.YamlPropertySourceLoader.load(YamlPropertySourceLoader.java:50)
	at org.springframework.boot.context.config.ConfigFileApplicationListener$Loader.loadDocuments(ConfigFileApplicationListener.java:547)
	at org.springframework.boot.context.config.ConfigFileApplicationListener$Loader.load(ConfigFileApplicationListener.java:517)
	at org.springframework.boot.context.config.ConfigFileApplicationListener$Loader.loadForFileExtension(ConfigFileApplicationListener.java:496)
	at org.springframework.boot.context.config.ConfigFileApplicationListener$Loader.load(ConfigFileApplicationListener.java:464)
	at org.springframework.boot.context.config.ConfigFileApplicationListener$Loader.lambda$null$6(ConfigFileApplicationListener.java:446)
	at java.lang.Iterable.forEach(Iterable.java:75)
	at org.springframework.boot.context.config.ConfigFileApplicationListener$Loader.lambda$load$7(ConfigFileApplicationListener.java:445)
	at java.lang.Iterable.forEach(Iterable.java:75)
	at org.springframework.boot.context.config.ConfigFileApplicationListener$Loader.load(ConfigFileApplicationListener.java:442)
	at org.springframework.boot.context.config.ConfigFileApplicationListener$Loader.load(ConfigFileApplicationListener.java:330)
	at org.springframework.boot.context.config.ConfigFileApplicationListener.addPropertySources(ConfigFileApplicationListener.java:212)
	at org.springframework.boot.context.config.ConfigFileApplicationListener.postProcessEnvironment(ConfigFileApplicationListener.java:195)
	at org.springframework.boot.context.config.ConfigFileApplicationListener.onApplicationEnvironmentPreparedEvent(ConfigFileApplicationListener.java:182)
	at org.springframework.boot.context.config.ConfigFileApplicationListener.onApplicationEvent(ConfigFileApplicationListener.java:168)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:127)
	at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:74)
	at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:54)
	at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:358)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:317)
	at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:139)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:117)
	at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:108)
	at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:117)
	at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:83)
	at org.springframework.boot.test.autoconfigure.SpringBootDependencyInjectionTestExecutionListener.prepareTestInstance(SpringBootDependencyInjectionTestExecutionListener.java:44)
	at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:246)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:227)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:289)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:291)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:246)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

[philwebb]

Looks like there's a method signature change from 1.19 -> 1.20. I guess officially we only support 1.19 in Boot 2.0 but I'll look to see if we can do some reflection hackery to also support 1.20.

[wilkinsona]

I've just been looking and it's nasty because part of the breaking API change is in a constructor of ScalarNode which we subclass. I don't think there's any easy reflective workaround.

[wilkinsona]

Just in case it's useful, the failure can be reproduced with this test:

@RunWith(ModifiedClassPathRunner.class)
@ClassPathOverrides("org.yaml:snakeyaml:1.20")
public class OriginTrackedYamlLoaderSnakeYamlCompatibilityTests
		extends OriginTrackedYamlLoaderTests {

}

[twicksell]

That'd be greatly appreciated if its doable.

I think beyond that, the big thing to ask for here is alignment between Spring core and Spring boot deps like this. I personally have no opinions on snakeyaml versions, but just having the latest of Boot and Core and normal conflict resolution (for gradle at least) gets me into a bad state. The dependency management plugin hides this by forcing versions from Boot, but then causes other problems when what I wanted was conflict resolution. So if Spring Framework and Spring Boot can stay aligned on dependencies like this within their respective major releases, it would just help prevent things like this from happening.

[wilkinsona]

We should probably raise this with the Framework team as I'm not sure that they're aware of how Gradle is now interpreting versions in a pom's optional dependencies and the effect it can have on dependency resolution. We should also raise it with the Gradle team as well as I'm not sure they're fully aware of the implications.

While Framework compiles against 1.20, it's also compatible with earlier versions. As the dependency is declared optional in its pom, it's not expected that the version Framework compiles against will have any effect on dependency resolution in a consuming build. That's Maven's behaviour and has, I believe, been Gradle's behaviour until recently. Building on this expectation, Framework has moved from 1.19 to 1.20 in a 5.0.x maintenance release. Due to Gradle interpreting Maven metadata in a different way to how Maven does (and how library publishers expect), this change causes problems as upgrading to a new Framework 5.0.x build inadvertently upgrades SnakeYAML from 1.19 to 1.20.

[wilkinsona]

I've opened gradle/gradle#5434.

[asomov]

Hi all, do you need any support in SnakeYAML ?
I mean: what can be done in the coming next release to help to resolve the issue ?

[wilkinsona]

@asomov Thanks very much for asking.

The problem with supporting 1.20 alongside 1.19 that we already support, is this backwards incompatible change. While we can use reflection to cope with the change in the signature of ScalarNode.getStyle(), I don't think we can do so to cope with the change to the ScalarNode constructor that we call from our subclass.

Perhaps there's an alternative approach that would work with both 1.19 and 1.20?

[asomov]

Well, I do not see a solution to support both.

I updated the changes to mention backwards-incompatible contributions. By the way, they were made to line up SnakeYAML to its YAML 1.2 brother.
When Spring Boot decides to support the latest YAML spec (which is a superset of JSON) we will need to migrate Sprint Boot to yet another API (which is not finalized yet)