owner(cdc): fix two metrics problems #4703

liuzix · 2022-02-25T07:51:28Z

What problem does this PR solve?

Issue Number: close #4714

What is changed and how it works?

Fixed checkpoint reporting to Prometheus when the checkpoint cannot be advanced.
Fixed "Table count maintained by owner" when the new scheduler is enabled.
Optimized the data structures in util.TableSet so that reporting the table count metrics will not hinder performance.

Check List

Tests

Unit test
Integration test
Manual test

The image below shows that this PR fix both problems. Before this PR, during scheduling (i.e. there are WIP tables), the checkpoint metrics should lack data.

Related changes

Need to cherry-pick to the release branch

Release note

Fix checkpoint metrics when tables are being scheduled.

ti-chi-bot · 2022-02-25T07:51:30Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

amyangfei
overvenus

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

liuzix · 2022-02-25T07:51:42Z

/run-integration-tests

3AceShowHand · 2022-02-25T14:08:18Z

/run-all-tests

liuzix · 2022-02-28T05:36:05Z

/run-integration-tests

…ated

…o fix-metrics-not-updated

liuzix · 2022-02-28T07:25:23Z

/run-integration-tests

liuzix · 2022-02-28T07:36:59Z

/run-integration-tests

liuzix · 2022-02-28T08:23:59Z

/run-integration-tests

overvenus · 2022-02-28T08:45:14Z

cdc/scheduler/util/table_set.go

+	if !ok {
+		log.Panic("unreachable", zap.Int64("tableID", tableID))
+	}
+	statusCountEntry[record.Status]--


Will it underflow?

Other states maintained in TableSet is sufficient to guarantee that there is no underflow.

cdc/owner/changefeed.go

cdc/owner/owner.go

liuzix · 2022-02-28T09:08:11Z

/run-integration-tests
/run-leak-tests

liuzix · 2022-02-28T09:40:09Z

/run-check-issue-triage-complete

codecov-commenter · 2022-02-28T10:25:55Z

Codecov Report

Merging #4703 (eb6a077) into master (9607554) will decrease coverage by 0.3656%.
The diff coverage is 52.9739%.

Flag	Coverage Δ
cdc	`59.9211% <52.9739%> (-0.0012%)`	⬇️
dm	`51.3893% <ø> (-0.6396%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master      #4703        +/-   ##
================================================
- Coverage   55.6402%   55.2746%   -0.3657%     
================================================
  Files           494        518        +24     
  Lines         61283      64033      +2750     
================================================
+ Hits          34098      35394      +1296     
- Misses        23750      25146      +1396     
- Partials       3435       3493        +58

overvenus · 2022-02-28T12:02:44Z

/merge

ti-chi-bot · 2022-02-28T12:02:46Z

This pull request has been accepted and is ready to merge.

Commit hash: eb6a077

liuzix · 2022-02-28T15:51:12Z

/run-kafka-integration-test

liuzix · 2022-03-01T00:45:41Z

/run-kafka-integration-test

liuzix · 2022-03-01T01:17:27Z

/run-kafka-integration-test

liuzix · 2022-03-01T04:42:48Z

/run-all-tests

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot · 2022-03-01T06:24:22Z

In response to a cherrypick label: new pull request created: #4729.

close #4714

owner(cdc): fix metrics not updated when dispatching tables

02d109c

ti-chi-bot added do-not-merge/needs-linked-issue release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Feb 25, 2022

ti-chi-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Feb 25, 2022

Merge branch 'master' into fix-metrics-not-updated

6f67c38

liuzix added 3 commits February 28, 2022 15:22

fix owner maintained table counts

4f7de11

Merge remote-tracking branch 'origin/master' into fix-metrics-not-upd…

69104e8

…ated

Merge branch 'fix-metrics-not-updated' of github.com:liuzix/ticdc int…

10784b6

…o fix-metrics-not-updated

ti-chi-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Feb 28, 2022

ti-chi-bot added do-not-merge/needs-triage-completed and removed do-not-merge/needs-linked-issue labels Feb 28, 2022

liuzix changed the title ~~[WIP]owner(cdc): fix metrics not updated when dispatching tables~~ owner(cdc): fix two metrics problems Feb 28, 2022

ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 28, 2022

fix import format

34eab25

liuzix requested review from overvenus, amyangfei and asddongmen and removed request for amyangfei February 28, 2022 07:46

liuzix added 2 commits February 28, 2022 15:46

fix

e0188e5

fix panic

07a24be

liuzix requested a review from amyangfei February 28, 2022 08:36

overvenus reviewed Feb 28, 2022

View reviewed changes

amyangfei reviewed Feb 28, 2022

View reviewed changes

cdc/owner/owner.go Show resolved Hide resolved

liuzix added 2 commits February 28, 2022 17:04

improve PDClient

5fbb93a

update changefeedStatusGauge

524c7a9

liuzix added the type/bugfix This PR fixes a bug. label Feb 28, 2022

ti-chi-bot added needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. and removed do-not-merge/needs-triage-completed labels Feb 28, 2022

Merge branch 'master' into fix-metrics-not-updated

eb6a077

amyangfei approved these changes Feb 28, 2022

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Feb 28, 2022

overvenus approved these changes Feb 28, 2022

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Feb 28, 2022

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Feb 28, 2022

Merge branch 'master' into fix-metrics-not-updated

a77eaff

Merge branch 'master' into fix-metrics-not-updated

8014c8f

ti-chi-bot merged commit d141ee6 into pingcap:master Mar 1, 2022

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Mar 1, 2022

This is an automated cherry-pick of pingcap#4703

7f63220

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Mar 1, 2022

owner(cdc): fix two metrics problems (#4703) #4729

Merged

ti-chi-bot added a commit that referenced this pull request Apr 24, 2022

owner(cdc): fix two metrics problems (#4703) (#4729)

1a35190

close #4714

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

owner(cdc): fix two metrics problems #4703

owner(cdc): fix two metrics problems #4703

liuzix commented Feb 25, 2022 •

edited

Loading

ti-chi-bot commented Feb 25, 2022 •

edited

Loading

liuzix commented Feb 25, 2022

3AceShowHand commented Feb 25, 2022

liuzix commented Feb 28, 2022

liuzix commented Feb 28, 2022

liuzix commented Feb 28, 2022

liuzix commented Feb 28, 2022

overvenus Feb 28, 2022

liuzix Feb 28, 2022

liuzix commented Feb 28, 2022

liuzix commented Feb 28, 2022

codecov-commenter commented Feb 28, 2022 •

edited

Loading

overvenus commented Feb 28, 2022

ti-chi-bot commented Feb 28, 2022

liuzix commented Feb 28, 2022

liuzix commented Mar 1, 2022

liuzix commented Mar 1, 2022

liuzix commented Mar 1, 2022

ti-chi-bot commented Mar 1, 2022

owner(cdc): fix two metrics problems #4703

owner(cdc): fix two metrics problems #4703

Conversation

liuzix commented Feb 25, 2022 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Feb 25, 2022 • edited Loading

liuzix commented Feb 25, 2022

3AceShowHand commented Feb 25, 2022

liuzix commented Feb 28, 2022

liuzix commented Feb 28, 2022

liuzix commented Feb 28, 2022

liuzix commented Feb 28, 2022

overvenus Feb 28, 2022

Choose a reason for hiding this comment

liuzix Feb 28, 2022

Choose a reason for hiding this comment

liuzix commented Feb 28, 2022

liuzix commented Feb 28, 2022

codecov-commenter commented Feb 28, 2022 • edited Loading

Codecov Report

overvenus commented Feb 28, 2022

ti-chi-bot commented Feb 28, 2022

liuzix commented Feb 28, 2022

liuzix commented Mar 1, 2022

liuzix commented Mar 1, 2022

liuzix commented Mar 1, 2022

ti-chi-bot commented Mar 1, 2022

liuzix commented Feb 25, 2022 •

edited

Loading

ti-chi-bot commented Feb 25, 2022 •

edited

Loading

codecov-commenter commented Feb 28, 2022 •

edited

Loading