Suite configuration is not applied to tests selected by unique ID

[mpkorstanje]

Steps to reproduce

Given a suite with some configuration:

@Suite
@ConfigurationParameter(key = Constants.DEFAULT_TEST_INSTANCE_LIFECYCLE_PROPERTY_NAME, value = "per_class")
@SelectClasses(ExampleTest.class)
public class SuiteTest {
}

And a test that is sensitive to that configuration

class ExampleTest {

    boolean shared = false;

    @Test
    void test1(){
        if(!shared) {
            System.out.println("Hello world 1");
        } else {
            System.out.println("World was already greeted");
        }
        shared = true;

    }

    @Test
    void test2(){
        if(!shared) {
            System.out.println("Hello world 2");
        } else {
            System.out.println("World was already greeted");
        }
        shared = true;
    }
}

Then executing the tests either by their uniqueId or the SuiteTest class should not make a difference in execution:

public class TestRunner {

    public static void main(String[] args) {
        System.out.println("By suite");

        LauncherDiscoveryRequest requestA = request()
                .selectors(
                        DiscoverySelectors.selectClass(SuiteTest.class)
                )
                .build();

        LauncherFactory.create().execute(requestA);

        System.out.println("By unique id");

        LauncherDiscoveryRequest requestB = request()
                .selectors(
                        DiscoverySelectors.selectUniqueId(
                                UniqueId.forEngine("junit-platform-suite")
                                        .append("suite", "com.example.SuiteTest")
                                        .appendEngine("junit-jupiter")
                                        .append("class", "com.example.ExampleTest" )
                                        .append("method", "test1()")
                        ),
                        DiscoverySelectors.selectUniqueId(
                                UniqueId.forEngine("junit-platform-suite")
                                        .append("suite", "com.example.SuiteTest")
                                        .appendEngine("junit-jupiter")
                                        .append("class", "com.example.ExampleTest" )
                                        .append("method", "test2()")
                        )
                )
                .build();

        LauncherFactory.create().execute(requestB);
    }
}

But when executed we see that this is the case.

By suite
Hello world 1
World was already greeted

By unique id
Hello world 1
Hello world 2

Context

• JUnit 5.10.1.

The above example is somewhat contrived for the sake of reproducibility. The actual problems showed up when using Surefire to rerun Cucumber tests in a Suite. Surefire uses unique ids to rerun failures and Cucumber Suites tend to come with more configuration than JUnit.

[sbrannen]

Thanks for bringing this to our attention and for providing a detailed example.

I've confirmed that this is a bug, and I've assigned it to the 5.11 M1 milestone.

We don't currently have any plans to release 5.10.3; however, if more bugs accumulate we may choose to backport the fix for this to 5.10.x as well.