jdk19 Jep425Tests_testVirtualThread test_VirtualthreadYieldResume Virtual Thread 1: incorrect result of 1

[pshipton]

https://openj9-jenkins.osuosl.org/job/Test_openjdk19_j9_sanity.functional_x86-64_windows_OpenJDK19_testList_1/9
Jep425Tests_testVirtualThread_0

13:17:15  FAILED: test_VirtualthreadYieldResume
13:17:15  java.lang.AssertionError: Virtual Thread 1: incorrect result of 1
13:17:15  	at org.testng.AssertJUnit.fail(AssertJUnit.java:59)
13:17:15  	at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:24)
13:17:15  	at org.openj9.test.jep425.VirtualThreadTests.test_VirtualthreadYieldResume(VirtualThreadTests.java:86)

@tajila fyi

[tajila]

@JasonFengJ9 We'll need t exclude these as well

[tajila]

@fengxue-IS

[babsingh]

Removing the test excluded label since this issue has been resolved.

[pshipton]

https://openj9-jenkins.osuosl.org/job/Test_openjdk19_j9_sanity.functional_ppc64_aix_Nightly_testList_1/89
Jep425Tests_testVirtualThread_2 --enable-preview -Xgcpolicy:balanced

01:17:03  FAILED: test_VirtualthreadYieldResume
01:17:03  java.lang.AssertionError: Virtual Thread 0: incorrect result of 2
01:17:03  	at org.testng.AssertJUnit.fail(AssertJUnit.java:59)
01:17:03  	at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:24)
01:17:03  	at org.openj9.test.jep425.VirtualThreadTests.test_VirtualthreadYieldResume(VirtualThreadTests.java:87)

[tajila]

@babsingh can you please take a look at this?

[babsingh]

re #15651 (comment):

Unable to reproduce the failure

Ran two grinders:

• 100x (20x on 5 different AIX machines): https://openj9-jenkins.osuosl.org/job/Grinder/1827/; and
• 300x (on the AIX machine where the recent failure was seen): https://openj9-jenkins.osuosl.org/job/Grinder/1828/.

The failure couldn't be reproduced.

Failing test's source code

openj9/test/functional/Java19andUp/src/org/openj9/test/jep425/VirtualThreadTests.java

Lines 72 to 92 in b975e82

	public void test_VirtualthreadYieldResume() {
	try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
	int[] results = new int[6];
	IntStream.range(0, 6).forEach(i -> {
	executor.submit(() -> {
	results[i] = 1;
	Thread.sleep(Duration.ofSeconds(1));
	results[i] += 1;
	Thread.sleep(Duration.ofSeconds(1));
	results[i] += 1;
	return i;
	});
	});
	executor.awaitTermination(5L, TimeUnit.SECONDS);
	for (int i = 0; i < 6; i++) {
	AssertJUnit.assertTrue("Virtual Thread " + i + ": incorrect result of " + results[i], (results[i] == 3));
	}
	} catch (Exception e) {
	Assert.fail("Unexpected exception occured : " + e.getMessage() , e);
	}
	}

Probable cause

I believe that the problem lies in the below line of the test's source code:

executor.awaitTermination(5L, TimeUnit.SECONDS);

5 seconds is not sufficient in the worst case scenario.

Why? 6 virtual threads are being launched in the test-case. Each virtual thread sleeps twice for 1 second (2 seconds in total) and performs three arithmetic operations. In addition, there is cost associated to schedule, sleep and wake the thread. Assumption: each virtual thread may take 3-4 seconds to execute in the worst case scenario. If the machine has scarce resources i.e. each virtual thread is executed sequentially (worst-case), the test-case may take 4 seconds x 6 threads = 24 seconds to execute. Overall, awaitTermination should wait for 24 seconds or preferably more for the virtual threads to finish. Next week, I will create a PR to reflect this change in the test code.

[pshipton]

Removing from milestone plan since this won't block the release.

[pshipton]

https://openj9-jenkins.osuosl.org/job/Test_openjdk19_j9_sanity.functional_ppc64_aix_Nightly_testList_1/109
Jep425Tests_testVirtualThread_1

02:10:08  FAILED: test_YieldedVirtualThreadGetStackTrace
02:10:08  java.lang.AssertionError: Expected 11 frames, got 7
02:10:08  	at org.testng.AssertJUnit.fail(AssertJUnit.java:59)
02:10:08  	at org.testng.AssertJUnit.assertTrue(AssertJUnit.java:24)
02:10:08  	at org.openj9.test.jep425.VirtualThreadTests.test_YieldedVirtualThreadGetStackTrace(VirtualThreadTests.java:182)

[babsingh]

@fengxue-IS A similar failure from #16693 is seen above for different reasons. Can you review the test case for other weaknesses?

[babsingh]

This issue only tracks failures related to test_VirtualthreadYieldResume. The test_YieldedVirtualThreadGetStackTrace failure is seen for the first time. @pshipton Should we track it in a new issue?

[pshipton]

Yes, sorry. Created #16722