Flaky-test: org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection

[lhotari]

org.apache.pulsar.broker.loadbalance.LoadBalancerTest is flaky. The testLeaderElection test method fails sporadically.

java.util.NoSuchElementException: No value present
	at java.util.Optional.get(Optional.java:135)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.loopUntilLeaderChanges(LoadBalancerTest.java:185)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:743)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-02-01T13:40:12.3522919Z
example failure 2021-02-01T11:54:58.8604105Z
example failure 2021-02-01T11:23:42.3794621Z
example failure 2021-02-01T11:06:52.7534377Z
example failure 2021-02-01T03:33:28.1828410Z
example failure 2021-01-31T13:45:59.5870974Z
example failure 2021-01-30T18:37:49.6338778Z
example failure 2021-01-30T18:29:54.3189511Z
example failure 2021-01-30T10:02:27.9882474Z
example failure 2021-01-30T09:44:03.3715367Z
example failure 2021-01-30T09:11:24.3449948Z
example failure 2021-01-30T08:53:01.0720954Z
example failure 2021-01-29T00:11:58.6667897Z
example failure 2021-01-28T15:32:42.7100802Z
example failure 2021-01-28T15:23:10.6157274Z
example failure 2021-01-28T15:12:14.1950625Z
example failure 2021-01-28T07:23:32.7249440Z
example failure 2021-01-28T07:15:33.8671272Z
example failure 2021-01-28T07:07:30.7523687Z
example failure 2021-01-28T01:59:46.6547905Z
example failure 2021-01-28T01:52:11.4000197Z
example failure 2021-01-27T21:58:30.2044042Z
example failure 2021-01-27T12:06:15.0002925Z
example failure 2021-01-27T11:58:41.8274891Z

Full exception stacktrace

java.util.NoSuchElementException: No value present
	at java.util.Optional.get(Optional.java:135)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.loopUntilLeaderChanges(LoadBalancerTest.java:185)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:743)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:46725)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-02-01T10:49:19.1817633Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:46725)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:35563)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-02-01T13:27:59.9104169Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:35563)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost3:38237)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-01-31T13:27:37.9313406Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost3:38237)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:36589)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-01-31T13:37:02.0885058Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:36589)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:36253)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-01-27T23:15:52.1427955Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:36253)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:46437)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-01-27T13:20:28.5577038Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:46437)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:34401)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-01-26T21:22:27.4025488Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:34401)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost1:45573)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-02-01T11:45:44.5009257Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost1:45573)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost1:43183)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-02-01T04:07:41.3221135Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost1:43183)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost3:32955)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-01-26T21:38:08.1357619Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost3:32955)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost3:37271)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-01-27T01:25:39.9608804Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost3:37271)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:38263)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

example failure 2021-01-28T02:07:26.8743907Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:38263)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost1:39203)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-01-26T21:30:13.3435396Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost1:39203)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost2:36493)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-02-01T11:36:57.1174828Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost2:36493)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost1:46259)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-01-27T06:49:07.9611925Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost1:46259)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:37953)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-01-30T09:02:07.6453109Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:37953)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost3:39865)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-01-27T01:41:04.8537837Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost3:39865)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost2:34025)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

example failure 2021-02-01T03:57:37.0419371Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost2:34025)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:38887)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

Usage tip: To enable automatic navigation to failure message, open the links with CTRL/CMD-click.
example failure 2021-01-27T20:47:03.8399173Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:38887)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:46513)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

example failure 2021-02-01T13:18:35.4373409Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:46513)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:40269)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

example failure 2021-01-29T00:19:52.4940690Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:40269)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:41803)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

example failure 2021-01-27T23:30:22.3669629Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:41803)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost1:42221)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

example failure 2021-01-27T06:40:00.5182280Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost1:42221)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.util.NoSuchElementException: No value present
	at java.util.Optional.get(Optional.java:135)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.loopUntilLeaderChanges(LoadBalancerTest.java:186)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:744)

example failure 2021-01-28T18:29:27.4273404Z

Full exception stacktrace

java.util.NoSuchElementException: No value present
	at java.util.Optional.get(Optional.java:135)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.loopUntilLeaderChanges(LoadBalancerTest.java:186)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:744)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost1:41757)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

example failure 2021-01-27T06:29:47.3585099Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost1:41757)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:37805)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

example failure 2021-01-27T08:05:03.5029714Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:37805)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:37205)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)

example failure 2021-01-27T21:50:29.0310289Z

Full exception stacktrace

java.lang.AssertionError: expected [Optional[LeaderBroker(serviceUrl=http://localhost4:37205)]] but found [Optional.empty]
	at org.testng.Assert.fail(Assert.java:99)
	at org.testng.Assert.failNotEquals(Assert.java:1037)
	at org.testng.Assert.assertEqualsImpl(Assert.java:140)
	at org.testng.Assert.assertEquals(Assert.java:122)
	at org.testng.Assert.assertEquals(Assert.java:617)
	at org.apache.pulsar.broker.loadbalance.LoadBalancerTest.testLeaderElection(LoadBalancerTest.java:737)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

[lhotari]

I have started work on fixing this flaky test.

[lhotari]

I was able to reproduce the flakiness and I made some changes in master...lhotari:lh-fix-flaky-load-balancer-test . However the flakiness doesn't go away.
The logs are in https://gist.github.com/lhotari/0653dc5d9ca418a1bca2d01a632fd655 .

The problem is that the leader doesn't get assigned to one of the follower brokers. The value of the current leaders stays Optional.empty. This can be found in the logs around these lines:

10:38:34.605 [TestNG-method=testLeaderElection-1] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - New leader is : http://localhost4:32957
10:38:34.605 [TestNG-method=testLeaderElection-1] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Old leader is : http://localhost4:32957
10:38:34.705 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost4:32957 is : Optional[LeaderBroker(serviceUrl=http://localhost4:32957)]
10:38:34.806 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost2:35915 is : Optional[LeaderBroker(serviceUrl=http://localhost4:32957)]
10:38:34.906 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:35.008 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:35.108 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:35.208 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:35.308 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:35.408 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:35.509 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:35.609 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:35.709 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:35.809 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:35.910 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:36.010 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:36.110 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:36.210 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:36.310 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:36.411 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:36.511 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:36.611 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:36.711 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:36.811 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
10:38:36.912 [awaitility-thread] INFO  org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty
... (continue until it times out)

[lhotari]

This was the stacktrace for the NPE that I happened to come across. I wonder if this is a corner case at startup time. The NPE happened in a log statement. I just wonder why the stack is so deep.

09:43:34.650 [metadata-store-1722-1] WARN  org.apache.pulsar.metadata.coordination.impl.LeaderElectionImpl - Exception in state change listener
java.lang.NullPointerException: null
	at org.apache.pulsar.broker.PulsarService.lambda$startLeaderElectionService$3(PulsarService.java:731) ~[classes/:?]
	at org.apache.pulsar.metadata.coordination.impl.LeaderElectionImpl.lambda$accept$12(LeaderElectionImpl.java:257) ~[pulsar-metadata-2.8.0-SNAPSHOT.jar:2.8.0-SNAPSHOT]
	at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:884) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:866) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_275]
	at org.apache.pulsar.metadata.coordination.impl.LeaderElectionImpl.lambda$null$9(LeaderElectionImpl.java:177) ~[pulsar-metadata-2.8.0-SNAPSHOT.jar:2.8.0-SNAPSHOT]
	at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:884) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:866) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_275]
	at org.apache.pulsar.metadata.coordination.impl.LeaderElectionImpl.lambda$null$9(LeaderElectionImpl.java:177) ~[pulsar-metadata-2.8.0-SNAPSHOT.jar:2.8.0-SNAPSHOT]
	at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:884) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:866) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_275]
	at org.apache.pulsar.metadata.coordination.impl.LeaderElectionImpl.lambda$null$9(LeaderElectionImpl.java:177) ~[pulsar-metadata-2.8.0-SNAPSHOT.jar:2.8.0-SNAPSHOT]
	at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:884) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:866) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_275]
	at org.apache.pulsar.metadata.coordination.impl.LeaderElectionImpl.lambda$null$9(LeaderElectionImpl.java:177) ~[pulsar-metadata-2.8.0-SNAPSHOT.jar:2.8.0-SNAPSHOT]
	at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:884) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:866) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_275]
	at org.apache.pulsar.metadata.coordination.impl.LeaderElectionImpl.lambda$null$9(LeaderElectionImpl.java:177) ~[pulsar-metadata-2.8.0-SNAPSHOT.jar:2.8.0-SNAPSHOT]
	at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:884) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:866) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_275]
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_275]
.
.
.
THIS KEEPS GOING ON and ON until 1024 stackframes
.
.
.

full stacktrace which seems to be truncated to 1024 stack frames: https://gist.github.com/lhotari/11cbf31cfad7d82185f42e8f06e2b5a2

[lhotari]

@merlimat Would you mind taking a look at the comments in this issue? It seems that the flakiness might be related to recent PIP-45 changes made in #9240.

The stack trace with 1024 stack frames seems concerning. The same lines seem to repeat there, looking like a very deep recursion. Is there an explanation for this?

[merlimat]

@lhotari Submitted a fix at #9460

[lhotari]

Submitted a fix at #9460

Thank you @merlimat