Tests for failover watcher

[abhishekj720]

What changed?
Added tests for failover watcher: Includes continous trying to fetch the watcher status and behave accordingly

Why?
To make the code more reliable and as part of code coverage week.

How did you test it?
Unit tests

Potential risks
No risk. It is written to avoid the risk.

[codecov]

Codecov Report

Merging #5864 (de0df4c) into master (8f3248c) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

❗ Current head de0df4c differs from pull request most recent head eb1ed59. Consider uploading reports for the commit eb1ed59 to get more accurate results

Additional details and impacted files

Files	Coverage Δ
common/domain/failover_watcher.go	84.94% <100.00%> (+29.03%)	⬆️

... and 10 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8f3248c...eb1ed59. Read the comment docs.

[coveralls]

Pull Request Test Coverage Report for Build 018eab4a-8a37-4faa-814b-36299eb1501f

Details

• 2 of 2 (100.0%) changed or added relevant lines in 1 file are covered.
• 41 unchanged lines in 14 files lost coverage.
• Overall coverage increased (+0.02%) to 66.327%

---

Files with Coverage Reduction	New Missed Lines	%
service/matching/taskReader.go	2	84.88%
common/task/weighted_round_robin_task_scheduler.go	2	89.05%
common/persistence/sql/sqlplugin/mysql/task.go	2	73.68%
service/history/task/transfer_active_task_executor.go	2	72.7%
service/matching/taskListManager.go	2	80.46%
service/history/execution/mutable_state_util.go	2	78.13%
service/history/task/fetcher.go	2	85.05%
common/persistence/sql/sqlplugin/mysql/db.go	2	79.49%
common/task/fifo_task_scheduler.go	3	84.54%
service/history/queue/timer_gate.go	3	95.83%

Totals
Change from base Build 018eab3b-515a-4b06-8fd8-099db2baf188:	0.02%
Covered Lines:	97069
Relevant Lines:	146349

---

💛 - Coveralls