HBASE-27720 TestClusterRestartFailover is flakey due to metrics asser…

…tion (#5110)

Signed-off-by: Duo Zhang <zhangduo@apache.org>
(cherry picked from commit f36587c)

hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestClusterRestartFailover.java

@@ -152,6 +152,12 @@ private void setupCluster() throws Exception {
     LOG.info("Setup cluster");
     UTIL.startMiniCluster(StartMiniClusterOption.builder().masterClass(HMasterForTest.class)
       .numMasters(1).numRegionServers(3).build());
+    // this test has been flaky. When it is rerun by surefire, the underlying minicluster isn't
+    // completely cleaned. specifically, the metrics system isn't reset. The result is an otherwise
+    // successful re-run is failed because there's 8 or 12 SCPcounts instead of the 4 that a
+    // single run of the test would otherwise produce. Thus, explicitly reset the metrics source
+    // each time we setup the cluster.
+    UTIL.getMiniHBaseCluster().getMaster().getMasterMetrics().getMetricsSource().init();
     LOG.info("Cluster is up");
     UTIL.waitFor(60000, () -> UTIL.getMiniHBaseCluster().getMaster().isInitialized());
     LOG.info("Master is up");