testFrameworkProperty cannot be changed any further

[marcingrzejszczak]

After upgrading standalone sample to Gradle 8.1.1 (I've upgraded 3.1.x, 4.0.x and main branches) I've started seeing exceptions in the dsl/http-server project

* Where:
Build file '/home/marcin/repo/spring-cloud-contract/samples/standalone/dsl/http-server/build.gradle' line: 49

* What went wrong:
A problem occurred evaluating root project 'http-server-dsl-gradle'.
> The value for task ':contractTest' property 'testFrameworkProperty' cannot be changed any further.

@shanman190 can you check it out in your spare time 🙏 ?

[shanman190]

@marcingrzejszczak sure, I'll try to take a look. Based on the output, it immediately points me toward something about the Test task setup likely with useJUnitPlatform(). I vaguely recall an issue similar to this on a 7.x patch release for Gradle. I'm sure we'll figure something out for it.

[shanman190]

So the issue here is related to the finalization of testFramework on read within Gradle proper specifically for jvm-test-suites. The good news is that the finalization has been removed as part of gradle/gradle#24331. The bad news is that this change is only available in the latest snapshots or 8.2 RC1 build and will be available once 8.2 is actually released.

As a workaround that we can do right now, would simply be to drop the useJUnitPlatform() invocation on line 49 of the sample as that is exactly the same as the default we apply to the contractTest suite in the plugin itself. For other cases where include or exclude tags are being used, those would be blocked until 8.2 is released...

[shanman190]

Found a workaround for include/exclude tags. It involves moving from configuring the options on the task itself from the top level to modifying the options from inside of the test suite.

So this:

contractTest {
    useJUnitPlatform {
        includeTags "fast"
    }
}

Should work as (untested):

testing.suites.named("contractTest").configure {
    useJUnitPlatform {
        includeTags "fast"
    }
}

[marcingrzejszczak]

Hmm I tried to fix it through this 169a39d but it didn't help. Most likely I misunderstood you 😬

[shanman190]

Sorry, I really should have just included a permalink. Haha

Here's a link of the one line to comment out:
https://github.com/spring-cloud/spring-cloud-contract/blob/main/samples/standalone/dsl/http-server/build.gradle#L49

- useJUnitPlatform()
+ //useJUnitPlatform()

[marcingrzejszczak]

When I tried that one the tests never ran and the build faild :(

[shanman190]

Hmm. That doesn't sound right given we have it configured explicitly inside of the plugin.

https://github.com/spring-cloud/spring-cloud-contract/blob/main/spring-cloud-contract-tools/spring-cloud-contract-gradle-plugin/src/main/java/org/springframework/cloud/contract/verifier/plugin/SpringCloudContractVerifierGradlePlugin.java#L137

Prior to the change to explicitly configure JUnit Platform, I would have expected the tests to fail as the JUnit 4 engine couldn't run JUnit 5 tests, but now we've explicitly told Gradle to by default use the JUnit 5 engine unless someone changes it later. I'll pull it down here in a bit and try a few things, but I would honestly have expected that to work presently.

[shanman190]

So I've found the following to be true:

• 3.1.x works with Gradle 8.1.1 with the snippet that @marcingrzejszczak commented out uncommented (ie. no changes).
• 4.0.x works with Gradle 8.1.1 with the snippet that @marcingrzejszczak commented out uncommented and useJUnitPlatform() commented out. However, there are some failing contracts at least on Windows.
• main fails for me due to what I'm suspecting is lacking a JDK 21 EA installation on my test machine. I'm assuming that commenting out useJUnitPlatform() only would also succeed like it did in the 4.0.x branch test.

The reason for this difference across the versions is due to the jvm-test-suite plugin being introduced in the 4.0.x releases. So for that one and onward, useJUnitPlatform() presently cannot be specified on the contractTest task. For 3.1.x, that predates the introduction of the jvm-test-suite plugin, so this needs to have useJUnitPlatform() still in order to properly execute tests.

[marcingrzejszczak]

Ehh so why is the Github Actions build broken for all of those branches? 😬

[marcingrzejszczak]

🤦 I moved the useJUnitPlatform() to doFirst {} and now it's working fine. This is ridiculous

[shanman190]

@marcingrzejszczak, that will only be valid for the 3.1.x branch and will still be invalid for 4.0.x and main. If the Maven build has stabilized a bit (before I wasn't even seeing it get past shaded dependencies), then I can push the fixes for this too.

[marcingrzejszczak]

Actually it worked fine for 3.1.x and 4.0.x - the build is green (which doesn't mean that it's how it should be done). In other words the build is green so feel free to fix it however you think it should be done ;)