AssertionFailedError interrupts test tree in Console and Gradle

[smoyer64]

Overview

• (X) Bug report. When assertions fail during a test run and provide a multi-line message (as is common with the Hamcrest assertions and matchers, the test tree is interrupted when running via the console or Gradle. This problem is also seen when assertAll() throws a MultipleFailuresError. The example below demonstrates both these failure conditions:

@Test
public void assertAllExample() {
	assertAll(
		() -> { throw new AssertionError(new Throwable("first thing went wrong")); },
		() -> { throw new AssertionError(new Throwable("second thing went wrong")); },
		() -> assertThat("ERROR! - something is broke", not(containsString("ERROR!")))
	);
}

@Test
public void assertOneExample() {
	assertTrue(false, "Wasn't expecting that!\n\nNow I've broken the tree");
}

The output from when these tests are run via Gradle is as follows:

│  ├─ example.testrule.ErrorCollectorJupiterDemo
│  │  ├─ assertAllExample ✘ Multiple Failures (3 failures)
        java.lang.Throwable: first thing went wrong
        java.lang.Throwable: second thing went wrong
        
Expected: not a string containing "ERROR!"
     but: was "ERROR! - something is broke"
│  │  ├─ assertOneExample ✘ Wasn't expecting that!

Now I've broken the tree

Deliverables

• Continue the tree structure across line breaks similar to the following:

│  ├─ example.testrule.ErrorCollectorJupiterDemo
│  │  ├─ assertAllExample ✘ Multiple Failures (3 failures)
│  │  │    java.lang.Throwable: first thing went wrong
│  │  │    java.lang.Throwable: second thing went wrong
│  │  │    
│  │  │    Expected: not a string containing "ERROR!"
│  │  │    but: was "ERROR! - something is broke"
│  │  ├─ assertOneExample ✘ Wasn't expecting that!
│  │  │    
│  │  │    Now I've broken the tree

[marcphilipp]

Thanks for opening the issue! I'm putting it up-for-grabs for now. Do you want to give it a go?

[smoyer64]

@marcphilipp ... I'm working on #343 at the moment (could you review what I've proposed) and had also started on #144. Since #343 is still slated for the M4 release I'd like to finish that work before I commit to any further fixes. After that I'd be happy to pick off some of the up-for-grabs (I may need to add the fail() methods in #656 to finish #343 though).

[sormuras]

As I'm still mentally inside the tree-rendering topic, I assigned this issue to myself.

[sormuras]

The VerboseTreePrintingListener already handles the multiline detail message. Will move some of that logic up the hierachy.

[sormuras]

@smoyer64 This should be fixed with #667 - can you please verify? Thanks in advance.

[smoyer64]

CONFIRMED

I commented out line 30 of the ErrorCollectorJupiterDemo (@extensions.ExpectToFail) and now see the following output when the assertAll() method is executed:

│  ├─ ErrorCollectorJupiterDemo
│  │  ├─ example() ✘ Multiple Failures (3 failures)
│  │  │            java.lang.Throwable: first thing went wrong
│  │  │            java.lang.Throwable: second thing went wrong
│  │  │  
│  │  │      Expected: not a string containing "ERROR!"
│  │  │           but: was "ERROR! - something is broke"