I'm conflicted to announce Cucumber-JVM v6.0.0. While v5.0.0 was released about six months ago, it seems like a lifetime.
Upgrading from v5 should relatively straightforward. Prior to upgrading to v6.0.0 upgrade to v5.7.0 and stop using all deprecated features. Some features will log a deprecation warning.
Below we'll discuss some notable changes in v6.0.0 As always the full change log can be found in the usual place.
Several years ago Matt Wayne came up with example mapping. Last year Cucumber Ruby introduced the Rule Keyword and today it comes to Cucumber JVM.
With the introduction of a new keyword in Gherkin we ran into the problem that current json formatter was essentially a direct representation of the Gherkin syntax tree; test results added to it like Christmas decorations.
Additionally, the json formatter did not have a schema, extremely high memory consumption, and lacked a consistent output between Cucumber implementations.
To resolve these issues Cucumber now has a message based output format.
To use the message formatter use:
@CucumberOptions(plugin = "message:target/cucumber-report.ndjson")The output looks like:
{"source":{"uri":"features/minimal/minimal.feature","data":"Feature: minim ...
{"gherkinDocument":{"uri":"features/minimal/minimal.feature","feature": ...
{"pickle":{"id":"4","uri":"features/minimal/minimal.feature", ... }}
{"stepDefinition":{"id":"0","pattern":{"source":"I have {int} cukes in my ...
{"testRunStarted":{"timestamp":{"seconds":"0","nanos":0}}}
{"testCase":{"id":"6","pickleId":"4","testSteps":[{"id":"5","pickleStepId": ...
{"testCaseStarted":{"timestamp":{"seconds":"0","nanos":1000000},"attempt":0, ...
{"testStepStarted":{"timestamp":{"seconds":"0","nanos":2000000}, ...
{"testStepFinished":{"testStepResult":{"status":"PASSED","duration": ...
{"testCaseFinished":{"timestamp":{"seconds":"0","nanos":6000000}, ...
{"testRunFinished":{"timestamp":{"seconds":"0","nanos":7000000}}}
This formatter will eventfully replace the existing json formatter. If you are using the json formatter to generate reports, please request support for this new format.
The old html formatter has been replaced with an improved html formatter that outputs the entire report as single file rather than a collection of files.
@CucumberOptions(plugin = "html:target/cucumber-report.html")Note: If you are already using the html formatter. Do not forget to append
.html.
The cucumber.options property could be used to pass commandline arguments to
cucumber-junit and cucumber-testng For example:
mvn clean test -Dcucumber.options='--monochrome --tags "not @ignored"'This is rather complicated. Especially so when intermediates (e.g. maven, teamcity) also interpret the arguments. A better way to do this is to provide each option as an individual property.
mvn clean test \
-Dcucumber.ansi-colors.disabled=true \
-Dcucumber.filter.tags="not @ignored"A complete and accurate list of supported properties can be obtained by running
mvn exec:java \
-Dexec.classpathScope=test \
-Dexec.mainClass=io.cucumber.core.cli.Main \
-Dexec.args="--help"Cucumber Spring had a complicated way to configure the application context.
The configuration could be provided by either:
- A context configuration annotation on a class which also happens to have step definitions
- A magic file named
cucumber.xml - An empty application context if the previous options could not be discovered
This made it hard to explain how to use Cucumber Spring and the fallback strategy to the empty application context hides errors.
From now on the preferred way to use cucumber-spring is to annotate a class
with both @CucumberContextConfiguration and a Spring context configuration
annotation such as @ContextConfiguration, @SpringBootTest, ect.
import com.example.app;
import org.springframework.boot.test.context.SpringBootTest;
import io.cucumber.spring.CucumberContextConfiguration;
@CucumberContextConfiguration
@SpringBootTest(classes = TestConfig.class)
public class CucumberSpringConfiguration {
}The alternatives, cucumber.xml and annotating step definitions with a
@ContextConfiguration are no longer supported.
After execution a scenario in Cucumber has five possible end states. JUnit and
TestNG tests only use three states and build tools only support two outcomes.
When integrating with these tools Cucumber has to map its internal states to the
states used by the tooling. Depending on the strictness Cucumber will map
PENDING and UNDEFINED to different states.
When using --strict Cucumber maps PENDING and UNDEFINED to test or build
failure. With --non-strict they mapped test skipped or build success. So
missing and unimplemented steps will either fail or get skipped. The idea
behind --non-strict is that it allows work in progress not to fail the build.
Working in progress is not a good practice, so we don't see a lot of reason to support it.
On top of that, while the strict mode is available to plugins it is not
available to consumers of a plugins output. This makes it harder for reporting
tools to correctly interpret the outcome of a Cucumber test. To make this even
more complicated, different Cucumber implementations default to different values
of strict. So by removing --non-strict and defaulting to --strict we will
eventually make it possible to remove this complexity for consumers of Cucumbers
output.
As an alternative, tags and tag filters can be used to mark features and scenarios that are not yet implemented. This has the advantage that unimplemented features clearly marked as not yet implemented.
Previously when using Cucumber JUnit or TestNG a progress indicator and summary of all tests would be printed. While helpful the output would interleave with the progress reporting from other tools such as Maven or Gradle. Since these tools already do some progress reporting, Cucumber should not duplicate it by default.
To enable the original helpful output use:
@CucumberOptions(plugins={"progress", "summary"})
public class RunCucumberTest {
}