Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

statements_summary has concurrent map read and write #35340

Closed
crazycs520 opened this issue Jun 13, 2022 · 1 comment · Fixed by #35367
Closed

statements_summary has concurrent map read and write #35340

crazycs520 opened this issue Jun 13, 2022 · 1 comment · Fixed by #35367
Assignees
Labels
affects-4.0 This bug affects 4.0.x versions. affects-5.0 This bug affects 5.0.x versions. affects-5.1 This bug affects 5.1.x versions. affects-5.2 This bug affects 5.2.x versions. affects-5.3 This bug affects 5.3.x versions. affects-5.4 This bug affects 5.4.x versions. affects-6.0 affects-6.1 severity/critical sig/diagnosis SIG: Diagnosis type/bug The issue is confirmed as a bug.

Comments

@crazycs520
Copy link
Contributor

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

TiDB version: v4.0.10
image

TiDB master also has this problem, which is casue by stmtSummaryByDigestElement.authUsers map.
image

image

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

master: 738f681

@crazycs520 crazycs520 added the type/bug The issue is confirmed as a bug. label Jun 13, 2022
@ti-chi-bot ti-chi-bot added may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.0 may-affects-6.1 labels Jun 14, 2022
@jebter jebter added sig/sql-infra SIG: SQL Infra affects-4.0 This bug affects 4.0.x versions. affects-5.0 This bug affects 5.0.x versions. affects-5.1 This bug affects 5.1.x versions. affects-5.2 This bug affects 5.2.x versions. affects-5.3 This bug affects 5.3.x versions. affects-5.4 This bug affects 5.4.x versions. affects-6.0 affects-6.1 labels Jun 14, 2022
@ti-chi-bot ti-chi-bot removed may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.0 may-affects-6.1 labels Jun 14, 2022
@bb7133 bb7133 added sig/diagnosis SIG: Diagnosis and removed sig/sql-infra SIG: SQL Infra labels Jun 14, 2022
@jackysp
Copy link
Member

jackysp commented Nov 18, 2022

fatal error: concurrent map read and map write

goroutine 12646676248 [running]:
runtime.throw(0x337d7ab, 0x21)
	/usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc01283d408 sp=0xc01283d3d8 pc=0x11d8032
runtime.mapaccess2_faststr(0x2f419a0, 0xc3b7432870, 0xc21f4c8fa0, 0x7, 0x54ff100, 0x0)
	/usr/local/go/src/runtime/map_faststr.go:116 +0x48f fp=0xc01283d478 sp=0xc01283d408 pc=0x11bbb1f
github.com/pingcap/tidb/util/stmtsummary.(*stmtSummaryByDigest).toCurrentDatum(0xc0b0167b80, 0x6375aab8, 0xc37e037270, 0xc038fd3300, 0x0, 0x0, 0x0)
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/src/github.com/pingcap/tidb/util/stmtsummary/statement_summary.go:569 +0x133 fp=0xc01283d4c0 sp=0xc01283d478 pc=0x1c35f53
github.com/pingcap/tidb/util/stmtsummary.(*stmtSummaryByDigestMap).ToCurrentDatum(0xc000379d00, 0xc37e037270, 0x0, 0x1, 0x0, 0x0)
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/src/github.com/pingcap/tidb/util/stmtsummary/statement_summary.go:325 +0x152 fp=0xc01283d568 sp=0xc01283d4c0 pc=0x1c34d22
github.com/pingcap/tidb/executor.(*memtableRetriever).setDataForStatementsSummary(0xc37e037310, 0x390aa80, 0xc2abb3be60, 0x3367a1e, 0x1a, 0x0, 0x0)
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/src/github.com/pingcap/tidb/executor/infoschema_reader.go:1647 +0x245 fp=0xc01283d5d8 sp=0xc01283d568 pc=0x2a6ea85
github.com/pingcap/tidb/executor.(*memtableRetriever).retrieve(0xc37e037310, 0x38c40a0, 0xc31f134cf0, 0x390aa80, 0xc2abb3be60, 0x0, 0x0, 0xc01283d6d8, 0x123ae98, 0xc31f134c30)
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/src/github.com/pingcap/tidb/executor/infoschema_reader.go:133 +0x102f fp=0xc01283d668 sp=0xc01283d5d8 pc=0x2a5ec4f
github.com/pingcap/tidb/executor.(*MemTableReaderExec).Next(0xc0440dd3b0, 0x38c40a0, 0xc31f134cf0, 0xc37e037400, 0x11b5568, 0x70)
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/src/github.com/pingcap/tidb/executor/memtable_reader.go:112 +0x485 fp=0xc01283d760 sp=0xc01283d668 pc=0x2aaf6d5
github.com/pingcap/tidb/executor.Next(0x38c40a0, 0xc31f134cf0, 0x38ce0a0, 0xc0440dd3b0, 0xc37e037400, 0x0, 0x0)
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/src/github.com/pingcap/tidb/executor/executor.go:262 +0x225 fp=0xc01283d8f8 sp=0xc01283d760 pc=0x2a33ec5
github.com/pingcap/tidb/executor.(*CoprocessorDAGHandler).HandleRequest(0xc01283da38, 0x38c40a0, 0xc31f134cf0, 0xc133ff5980, 0xc2abb3be60)
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/src/github.com/pingcap/tidb/executor/coprocessor.go:66 +0x30b fp=0xc01283d9c0 sp=0xc01283d8f8 pc=0x2a2105b
github.com/pingcap/tidb/server.(*rpcServer).handleCopRequest(0xc002d60150, 0x38c40a0, 0xc31f134cf0, 0xc133ff5980, 0x0)
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/src/github.com/pingcap/tidb/server/rpc_server.go:184 +0x199 fp=0xc01283da60 sp=0xc01283d9c0 pc=0x2cbb579
github.com/pingcap/tidb/server.(*rpcServer).Coprocessor(0xc002d60150, 0x38c40a0, 0xc31f134cf0, 0xc133ff5980, 0xc0440dd290, 0x0, 0x0)
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/src/github.com/pingcap/tidb/server/rpc_server.go:83 +0xba fp=0xc01283dad0 sp=0xc01283da60 pc=0x2cba33a
github.com/pingcap/kvproto/pkg/tikvpb._Tikv_Coprocessor_Handler(0x3301180, 0xc002d60150, 0x38c40a0, 0xc31f134cf0, 0xc2ab81a180, 0x0, 0x38c40a0, 0xc31f134cf0, 0xc044630000, 0x983)
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/pkg/mod/github.com/pingcap/kvproto@v0.0.0-20210308075244-560097d1309b/pkg/tikvpb/tikvpb.pb.go:3360 +0x217 fp=0xc01283db40 sp=0xc01283dad0 pc=0x20777e7
google.golang.org/grpc.(*Server).processUnaryRPC(0xc002cdf500, 0x38efca0, 0xc03844f980, 0xc026651400, 0xc002d60270, 0x4f4e970, 0x0, 0x0, 0x0)
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1024 +0x4f4 fp=0xc01283de18 sp=0xc01283db40 pc=0x171ba34
google.golang.org/grpc.(*Server).handleStream(0xc002cdf500, 0x38efca0, 0xc03844f980, 0xc026651400, 0x0)
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1313 +0xd97 fp=0xc01283df48 sp=0xc01283de18 pc=0x171f757
google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc2a499a250, 0xc002cdf500, 0x38efca0, 0xc03844f980, 0xc026651400)
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:722 +0xbb fp=0xc01283dfb8 sp=0xc01283df48 pc=0x172d01b
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc01283dfc0 sp=0xc01283dfb8 pc=0x1208fc1
created by google.golang.org/grpc.(*Server).serveStreams.func1
	/home/jenkins/agent/workspace/db_multi_branch_v4.0.12-20210713/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:720 +0xa1

similar panic happens.

image

will it #35367 fix it? @crazycs520

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-4.0 This bug affects 4.0.x versions. affects-5.0 This bug affects 5.0.x versions. affects-5.1 This bug affects 5.1.x versions. affects-5.2 This bug affects 5.2.x versions. affects-5.3 This bug affects 5.3.x versions. affects-5.4 This bug affects 5.4.x versions. affects-6.0 affects-6.1 severity/critical sig/diagnosis SIG: Diagnosis type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants