Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple dm-master become leader at the same time #11602

Closed
GMHDBJD opened this issue Sep 14, 2024 · 0 comments · Fixed by #11604
Closed

Multiple dm-master become leader at the same time #11602

GMHDBJD opened this issue Sep 14, 2024 · 0 comments · Fixed by #11604

Comments

@GMHDBJD
Copy link
Contributor

GMHDBJD commented Sep 14, 2024

What did you do?

kill dm leader

What did you expect to see?

one dm-master becomes leader

What did you see instead?

two dm-master become leader at the same time
dm-master1:

[2024/08/14 23:16:47.974 +00:00] [INFO] [election.go:273] ["successful manually clean election key"] [component=election] ["campaign error"="rpc error: code = Unknown desc = malformed header: missing HTTP content-type"]
[2024/08/14 23:16:47.975 +00:00] [INFO] [election.go:278] ["fail to campaign"] [component=election] ["current member"="{\"id\":\"xxx\",\"addr\":\"xxx-dm-master-1.xxx-dm-master-peer:8261\"}"] [error="rpc error: code = Unknown desc = malformed header: missing HTTP content-type"]
[2024/08/14 23:16:47.975 +00:00] [INFO] [election.go:300] ["get response from election observe"] [component=election] [key=/dm-master/leader/76e2914ca9fdb304] [value="{\"id\":\"xxx-dm-master-1\",\"addr\":\"xxx-dm-master-1.xxx-dm-master-peer:8261\"}"]
[2024/08/14 23:16:47.975 +00:00] [INFO] [election.go:331] ["become leader"] [component=election] ["current member"="{\"id\":\"xxx-dm-master-1\",\"addr\":\"xxx-dm-master-1.xxx-dm-master-peer:8261\"}"]
[2024/08/14 23:16:47.975 +00:00] [INFO] [election.go:331] ["become leader"] [component=election] ["current member"="{\"id\":\"xxx\",\"addr\":\"xxx-dm-master-1.xxx-dm-master-peer:8261\"}"]
dm-master-0: [2024/08/14 23:16:48.045 +00:00] [INFO] [election.go:331] ["become leader"] [component=election] ["current member"="{\"id\":\"xxx-dm-master-0\",\"addr\":\"xxx-dm-master-0.xxx-dm-master-peer:8261\"}"]

Versions of the cluster

DM version (run dmctl -V or dm-worker -V or dm-master -V):

(paste DM version here, and you must ensure versions of dmctl, DM-worker and DM-master are same)

Upstream MySQL/MariaDB server version:

(paste upstream MySQL/MariaDB server version here)

Downstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

(paste TiDB cluster version here)

How did you deploy DM: tiup or manually?

(leave TiUP or manually here)

Other interesting information (system version, hardware config, etc):

>
>

current status of DM cluster (execute query-status <task-name> in dmctl)

(paste current status of DM cluster here)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant