Flaky-test: BrokerRegistryTest.testRegisterAndLookup

[lhotari]

Search before asking

• I searched in the issues and found nothing similar.

Example failure

https://github.com/apache/pulsar/actions/runs/4073474341/jobs/7019289431#step:11:1397

Exception stacktrace

  Error:  Tests run: 12, Failures: 1, Errors: 0, Skipped: 8, Time elapsed: 18.168 s <<< FAILURE! - in org.apache.pulsar.broker.loadbalance.extensions.BrokerRegistryTest
  Error:  testRegisterAndLookup(org.apache.pulsar.broker.loadbalance.extensions.BrokerRegistryTest)  Time elapsed: 5.181 s  <<< FAILURE!
  java.lang.AssertionError: expected [2] but found [3]
  	at org.testng.Assert.fail(Assert.java:110)
  	at org.testng.Assert.failNotEquals(Assert.java:1413)
  	at org.testng.Assert.assertEqualsImpl(Assert.java:149)
  	at org.testng.Assert.assertEquals(Assert.java:131)
  	at org.testng.Assert.assertEquals(Assert.java:1240)
  	at org.testng.Assert.assertEquals(Assert.java:1274)
  	at org.apache.pulsar.broker.loadbalance.extensions.BrokerRegistryTest.testRegisterAndLookup(BrokerRegistryTest.java:277)
  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
  	at org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
  	at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
  	at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
  	at java.base/java.lang.Thread.run(Thread.java:833)

Logs: https://gist.github.com/lhotari/4e4a840de15e19774213fe6d034db3f2

Are you willing to submit a PR?

• I'm willing to submit a PR!

[lhotari]

@Demogorgon314 @heesung-sn Would you be able to fix this flaky test? It looks like it was introduced in #18810

[lhotari]

@Demogorgon314 Another problem is that the test leaves the test Pulsar brokers running.
This code is not valid for tests:

pulsar/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/extensions/BrokerRegistryTest.java

Lines 214 to 216 in 8a4a8a6

	if (pulsarService.isRunning()) {
	pulsarService.shutdownNow();
	}

Please replace it with pulsarService.close(); . It should be called in all cases, so remove the isRunning too.

[lhotari]

@Demogorgon314 I added the full logs for the failed test run to the description of the issue.

[lhotari]

@Demogorgon314 Yet another failure: https://github.com/apache/pulsar/actions/runs/4073367805/jobs/7019320477#step:11:1395 . It seems that this is very flaky. Logs: https://gist.github.com/lhotari/cb90c0eeb0320920be512c03f0258434

[Demogorgon314]

Thanks! I have found the root cause, and I'll push a PR to fix it soon.