Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

load unit will panic if query-status too often #3457

Closed
lance6716 opened this issue Nov 15, 2021 · 0 comments · Fixed by #3459
Closed

load unit will panic if query-status too often #3457

lance6716 opened this issue Nov 15, 2021 · 0 comments · Fixed by #3459
Labels
area/dm Issues or PRs related to DM. type/bug The issue is confirmed as a bug.

Comments

@lance6716
Copy link
Contributor

What did you do?

in CI

https://ci2.pingcap.net/blue/organizations/jenkins/dm_ghpr_integration_test/detail/dm_ghpr_integration_test/626/pipeline

What did you expect to see?

success

What did you see instead?

[2021-11-15T07:53:35.989Z] WARNING: DATA RACE
[2021-11-15T07:53:35.989Z] Read at 0x00c00061f6e0 by goroutine 291:
[2021-11-15T07:53:35.989Z]   runtime.mapaccess1_faststr()
[2021-11-15T07:53:35.989Z]       /usr/local/go/src/runtime/map_faststr.go:12 +0x0
[2021-11-15T07:53:35.989Z]   github.com/pingcap/ticdc/dm/loader.(*Loader).printStatus()
[2021-11-15T07:53:35.989Z]       /home/jenkins/agent/workspace/dm_ghpr_integration_test/go/src/github.com/pingcap/ticdc/dm/loader/status.go:51 +0x304
[2021-11-15T07:53:35.989Z] 
[2021-11-15T07:53:35.989Z] Previous write at 0x00c00061f6e0 by goroutine 368:
[2021-11-15T07:53:35.989Z]   runtime.mapassign_faststr()
[2021-11-15T07:53:35.989Z]       /usr/local/go/src/runtime/map_faststr.go:202 +0x0
[2021-11-15T07:53:35.989Z]   github.com/pingcap/ticdc/dm/loader.(*Loader).printStatus()
[2021-11-15T07:53:35.989Z]       /home/jenkins/agent/workspace/dm_ghpr_integration_test/go/src/github.com/pingcap/ticdc/dm/loader/status.go:52 +0x3c6
[2021-11-15T07:53:35.989Z] 
[2021-11-15T07:53:35.989Z] Goroutine 291 (running) created at:
[2021-11-15T07:53:35.989Z]   github.com/pingcap/ticdc/dm/loader.(*Loader).Status()
[2021-11-15T07:53:35.989Z]       /home/jenkins/agent/workspace/dm_ghpr_integration_test/go/src/github.com/pingcap/ticdc/dm/loader/status.go:37 +0x2c9
[2021-11-15T07:53:35.989Z]   github.com/pingcap/ticdc/dm/dm/worker.(*SubTask).unitTransWaitCondition()
[2021-11-15T07:53:35.989Z]       /home/jenkins/agent/workspace/dm_ghpr_integration_test/go/src/github.com/pingcap/ticdc/dm/dm/worker/subtask.go:651 +0x57e
[2021-11-15T07:53:35.989Z]   github.com/pingcap/ticdc/dm/dm/worker.(*SubTask).run()
[2021-11-15T07:53:35.989Z]       /home/jenkins/agent/workspace/dm_ghpr_integration_test/go/src/github.com/pingcap/ticdc/dm/dm/worker/subtask.go:240 +0x124
[2021-11-15T07:53:35.989Z]   github.com/pingcap/ticdc/dm/dm/worker.(*SubTask).fetchResultAndUpdateStage()
[2021-11-15T07:53:35.989Z]       /home/jenkins/agent/workspace/dm_ghpr_integration_test/go/src/github.com/pingcap/ticdc/dm/dm/worker/subtask.go:329 +0x1184
[2021-11-15T07:53:35.989Z] 
[2021-11-15T07:53:35.989Z] Goroutine 368 (running) created at:
[2021-11-15T07:53:35.989Z]   github.com/pingcap/ticdc/dm/loader.(*Loader).Status()
[2021-11-15T07:53:35.989Z]       /home/jenkins/agent/workspace/dm_ghpr_integration_test/go/src/github.com/pingcap/ticdc/dm/loader/status.go:37 +0x2c9
[2021-11-15T07:53:35.989Z]   github.com/pingcap/ticdc/dm/dm/worker.(*SubTask).Status()
[2021-11-15T07:53:35.989Z]       /home/jenkins/agent/workspace/dm_ghpr_integration_test/go/src/github.com/pingcap/ticdc/dm/dm/worker/status.go:31 +0xa3
[2021-11-15T07:53:35.989Z]   github.com/pingcap/ticdc/dm/dm/worker.(*SubTask).StatusJSON()
[2021-11-15T07:53:35.989Z]       /home/jenkins/agent/workspace/dm_ghpr_integration_test/go/src/github.com/pingcap/ticdc/dm/dm/worker/status.go:38 +0x6d
[2021-11-15T07:53:35.989Z]   github.com/pingcap/ticdc/dm/dm/worker.(*SubTask).fetchResultAndUpdateStage()
[2021-11-15T07:53:35.989Z]       /home/jenkins/agent/workspace/dm_ghpr_integration_test/go/src/github.com/pingcap/ticdc/dm/dm/worker/subtask.go:315 +0x6d7
[2021-11-15T07:53:35.989Z] ==================

and finally

[2021-11-15T07:53:35.989Z] fatal error: concurrent map writes

Versions of the cluster

DM version (run dmctl -V or dm-worker -V or dm-master -V):

5.3.0

Upstream MySQL/MariaDB server version:

(paste upstream MySQL/MariaDB server version here)

Downstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

(paste TiDB cluster version here)

How did you deploy DM: tiup or manually?

(leave TiUP or manually here)

Other interesting information (system version, hardware config, etc):

>
>

current status of DM cluster (execute query-status <task-name> in dmctl)

(paste current status of DM cluster here)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/dm Issues or PRs related to DM. type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant