JUnit runner does not distinguish between pending and ignored via Assume

[RichardBradley]

JUnit tests can be marked as "ignored" via the Assume API. If an assumption fails, then JUnit will mark the test as "ignored". Tests in this state do not count as failures.

Cucumber tests can be in "pending" state if they have undefined steps. Tests in this state should count as a failure if "strict" is set.

The cucumber.api.junit.Cucumber JUnit test runner doies not distinguish between these two cases; both are either failures or not according to the "strict" setting.

I believe that the "strict setting should make "pending" tests a failure, but not make "ignored" tests a failure.

[RichardBradley]

See #359 -- I think it's wrong to treat "Assume" the same as "pending": the former means "ignore without failing" whereas the latter means "fail if strict or ignore if not strict".

[mpkorstanje]

This will be partially resolved in version 2.0.0 once #1142 merged.

This won't cause tests to be marked as SKIPPED in Cucumber. They'll be marked as PENDING but when non-strict is enabled but these count as skipped in JUnit terms.

I think it's wrong to treat "Assume" the same as "pending": the former means "ignore without failing" whereas the latter means "fail if strict or ignore if not strict".

@brasmusson how do you feel about mapping AssumptionViolatedException to SKIPPED?

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.