Some metrics are not up to date in some scenarios

[amyangfei]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. What did you do? If possible, provide a recipe for reproducing the error.

   1. start two cdc servers
   2. create a new changefeed, with only one table replicated
   3. change cdc owner via http api
   4. pause changefeed
   5. resume changefeed
   6. query the ticdc_processor_num_of_tables metric of both cdc servers

2. What did you expect to see?

   There exists only one table count in ticdc_processor_num_of_tables of two cdc servers

3. What did you see instead?

➜  curl -s http://127.0.0.1:8300/metrics |grep num_of_tables
# HELP ticdc_processor_num_of_tables number of synchronized table of processor
# TYPE ticdc_processor_num_of_tables gauge
ticdc_processor_num_of_tables{capture="127.0.0.1:8300",changefeed="test-cf"} 1

➜  curl -s http://127.0.0.1:8302/metrics |grep num_of_tables
# HELP ticdc_processor_num_of_tables number of synchronized table of processor
# TYPE ticdc_processor_num_of_tables gauge
ticdc_processor_num_of_tables{capture="127.0.0.1:8302",changefeed="test-cf"} 1

4. Versions of the cluster

   • Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

     v4.0.10
     

   • TiCDC version (execute cdc version):

     v4.0.10
     

5. Known abnormal metrics

   • ticdc_processor_num_of_tables
   • ticdc_processor_table_resolved_ts