Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

index_hash_join hang when query exceed quota #54055

Closed
wshwsh12 opened this issue Jun 17, 2024 · 0 comments · Fixed by #57383
Closed

index_hash_join hang when query exceed quota #54055

wshwsh12 opened this issue Jun 17, 2024 · 0 comments · Fixed by #57383
Labels
affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.5 This bug affects the 8.5.x(LTS) versions. impact/panic severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.

Comments

@wshwsh12
Copy link
Contributor

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

source
one_case.txt

you can query this file some times.

2. What did you expect to see? (Required)

get error.

3. What did you see instead (Required)

hang

goroutine 18447 [select, 1 minutes]:
github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).getResultFromChannel(0xc003e40e00, {0x67bde40, 0xc0049f9360}, 0xc004a3dc20)
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:284 +0x85
github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).runInOrder(0xc003e40e00, {0x67bde40, 0xc0049f9360}, 0xc0049f9310)
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:242 +0x7c
github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).Next(0x67bde08?, {0x67bde08?, 0xc0049c53e0?}, 0x1?)
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:229 +0x92
github.com/pingcap/tidb/pkg/executor/internal/exec.Next({0x67bde08, 0xc0049c53e0}, {0x67e6ac0, 0xc003e40e00}, 0xc0049f9310)
	/workspace/source/tidb/pkg/executor/internal/exec/executor.go:403 +0x2c3
github.com/pingcap/tidb/pkg/executor.(*ExecStmt).next(0xc00f8cfe00, {0x67bde08, 0xc0049c53e0}, {0x67e6ac0, 0xc003e40e00}, 0x50b5dfd?)
	/workspace/source/tidb/pkg/executor/adapter.go:1211 +0x6e
github.com/pingcap/tidb/pkg/executor.(*recordSet).Next(0xc0049d3ec0, {0x67bde08?, 0xc0049c53e0?}, 0xc0049f9310)
	/workspace/source/tidb/pkg/executor/adapter.go:160 +0xbb
github.com/pingcap/tidb/pkg/server/internal/resultset.(*tidbResultSet).Next(0xc002624000?, {0x67bde08?, 0xc0049c53e0?}, 0xc00fa93440?)
	/workspace/source/tidb/pkg/server/internal/resultset/resultset.go:64 +0x25
github.com/pingcap/tidb/pkg/server.(*clientConn).writeChunks(0xc003d82d00, {0x67bde08, 0xc0049c53e0}, {0x67d6638, 0xc0049e4c80}, 0x0, 0x1d1?)
	/workspace/source/tidb/pkg/server/conn.go:2316 +0x18a
github.com/pingcap/tidb/pkg/server.(*clientConn).writeResultSet(0xc003d82d00, {0x67bde08, 0xc0049c53e0}, {0x67d6638, 0xc0049e4c80}, 0xf8?, 0x2, 0xc001c8eb00?)
	/workspace/source/tidb/pkg/server/conn.go:2259 +0x2f0
github.com/pingcap/tidb/pkg/server.(*clientConn).handleStmt(0xc003d82d00, {0x67bde40, 0xc0049d7310}, {0x67d37a8?, 0xc0049af0e0}, {0x0, 0x0, 0x0}, 0x1)
	/workspace/source/tidb/pkg/server/conn.go:2052 +0x52a
github.com/pingcap/tidb/pkg/server.(*clientConn).handleQuery(0xc003d82d00, {0x67bde40, 0xc0049d7310}, {0xc00e41a9b1, 0x46})
	/workspace/source/tidb/pkg/server/conn.go:1774 +0xb4c
github.com/pingcap/tidb/pkg/server.(*clientConn).dispatch(0xc003d82d00, {0x67bde08, 0xc0028b8780}, {0xc00e41a9b0, 0x47, 0x47})
	/workspace/source/tidb/pkg/server/conn.go:1348 +0xf2b
github.com/pingcap/tidb/pkg/server.(*clientConn).Run(0xc003d82d00, {0x67bde08, 0xc0028b8780})
	/workspace/source/tidb/pkg/server/conn.go:1114 +0x545
github.com/pingcap/tidb/pkg/server.(*Server).onConn(0xc00088cc00?, 0xc003d82d00)
	/workspace/source/tidb/pkg/server/server.go:739 +0x89d
created by github.com/pingcap/tidb/pkg/server.(*Server).startNetworkListener in goroutine 17151
	/workspace/source/tidb/pkg/server/server.go:557 +0x78d

goroutine 26756 [chan send, 1 minutes]:
github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).finishJoinWorkers(0xc003e40e00, {0x58c2120, 0xc00d5d6990})
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:201 +0x12c
github.com/pingcap/tidb/pkg/util.WithRecovery.func1()
	/workspace/source/tidb/pkg/util/misc.go:89 +0x4c
panic({0x58c2120?, 0xc00d5d6990?})
	/usr/local/go/src/runtime/panic.go:914 +0x21f
github.com/pingcap/tidb/pkg/util/memory.(*PanicOnExceed).Action(0xc0049c7dc0, 0xc003c5b4d0)
	/workspace/source/tidb/pkg/util/memory/action.go:173 +0x4b2
github.com/pingcap/tidb/pkg/util/memory.(*Tracker).Consume.func1(0xc003c5b4d8, 0xc0007e41b0?)
	/workspace/source/tidb/pkg/util/memory/tracker.go:437 +0xfc
github.com/pingcap/tidb/pkg/util/memory.(*Tracker).Consume(0x76?, 0x4a)
	/workspace/source/tidb/pkg/util/memory/tracker.go:464 +0x242
github.com/pingcap/tidb/pkg/distsql.indexRangesToKVWithoutSplit(0x7935e06375e0?, {0xc0101075e8, 0x1, 0xc0002c3400?}, 0xc004b7e600?, {0xc00e260c68, 0x1, 0x1d62269?}, 0xc0049fa750, 0xc00fa92ef0)
	/workspace/source/tidb/pkg/distsql/request_builder.go:749 +0x1db
github.com/pingcap/tidb/pkg/distsql.indexRangesToKVRangesForTablesWithInterruptSignal(...)
	/workspace/source/tidb/pkg/distsql/request_builder.go:663
github.com/pingcap/tidb/pkg/distsql.IndexRangesToKVRangesWithInterruptSignal(0xc0049e9260?, 0xc00e41aa00?, 0x46?, {0xc00e260c68?, 0x124?, 0xc0101cd200?}, 0x67a87f0?, 0xc0049c52f0?)
	/workspace/source/tidb/pkg/distsql/request_builder.go:647 +0x4d
github.com/pingcap/tidb/pkg/executor.buildKvRangesForIndexJoin(0xc0101cd8b8?, 0x4e54245?, 0x76, 0x1, {0xc004b7e400, 0x40, 0xc004bba140?}, {0xc00e260c68, 0x1, 0x1}, ...)
	/workspace/source/tidb/pkg/executor/builder.go:4636 +0x2b6
github.com/pingcap/tidb/pkg/executor.(*dataReaderBuilder).buildIndexReaderForIndexJoin(0xc0049d3a40, {0x67bde40, 0xc0049f9360}, 0xc0049f0840, {0xc004b7e400, 0x40, 0x40}, {0xc00e260c68, 0x1, 0x1}, ...)
	/workspace/source/tidb/pkg/executor/builder.go:4384 +0x8a9
github.com/pingcap/tidb/pkg/executor.(*dataReaderBuilder).buildExecutorForIndexJoinInternal(0xc004bba190?, {0x67bde40?, 0xc0049f9360?}, {0x67e16a8?, 0xc0049f0840?}, {0xc004b7e400?, 0x0?, 0x0?}, {0xc00e260c68, 0x1, ...}, ...)
	/workspace/source/tidb/pkg/executor/builder.go:4047 +0x170
github.com/pingcap/tidb/pkg/executor.(*dataReaderBuilder).buildExecutorForIndexJoin(...)
	/workspace/source/tidb/pkg/executor/builder.go:4038
github.com/pingcap/tidb/pkg/executor.(*innerWorker).fetchInnerResults(0xc008fe2240, {0x67bde40, 0xc0049f9360}, 0xc004a5cf80, {0xc004b7e400?, 0x40?, 0x40?})
	/workspace/source/tidb/pkg/executor/index_lookup_join.go:703 +0x205
github.com/pingcap/tidb/pkg/executor.(*indexHashJoinInnerWorker).fetchInnerResults(0xc008fe2240, {0x67bde40, 0xc0049f9360}, 0xc0101cde10?)
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:620 +0x5fgoroutine 26755 [select, 1 minutes]:
github.com/pingcap/tidb/pkg/executor.(*indexHashJoinOuterWorker).pushToChan(...)
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:406
github.com/pingcap/tidb/pkg/executor.(*indexHashJoinOuterWorker).run(0xc0049fc1c0, {0x67bde40, 0xc0049f9360})
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:355 +0x485
github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).startWorkers.func1()
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:168 +0x29
github.com/pingcap/tidb/pkg/util.WithRecovery(0x67bde08?, 0xc0049316b0?)
	/workspace/source/tidb/pkg/util/misc.go:97 +0x48
created by github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).startWorkers in goroutine 18447
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:168 +0x2fd

goroutine 26757 [semacquire, 1 minutes]:
sync.runtime_Semacquire(0x2e6e897?)
	/usr/local/go/src/runtime/sema.go:62 +0x25
sync.(*WaitGroup).Wait(0xc003b18fd0?)
	/usr/local/go/src/sync/waitgroup.go:116 +0x48
github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).wait4JoinWorkers(0xc003e40e00)
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:212 +0x25
created by github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).startWorkers in goroutine 18447
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:187 +0x5c7
github.com/pingcap/tidb/pkg/executor.(*indexHashJoinInnerWorker).handleTask(0xc008fe2240, {0x67bde40, 0xc0049f9360}, 0xc004a6e7e0, 0xc00aa3d320, {0x67c4fb8?, 0xc00f6001d0?}, 0xc004a3db60)
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:676 +0x36d
github.com/pingcap/tidb/pkg/executor.(*indexHashJoinInnerWorker).run(0xc008fe2240, {0x67bde40, 0xc0049f9360}, 0xc00fa932f0)
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:514 +0x258
github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).startWorkers.func2()
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:185 +0x4a
github.com/pingcap/tidb/pkg/util.WithRecovery(0x4e48e05?, 0xc0040ab880?)
	/workspace/source/tidb/pkg/util/misc.go:97 +0x48
created by github.com/pingcap/tidb/pkg/executor.(*IndexNestedLoopHashJoin).startWorkers in goroutine 18447
	/workspace/source/tidb/pkg/executor/index_lookup_hash_join.go:185 +0x44f


4. What is your TiDB version? (Required)

master, v8.1.0

@wshwsh12 wshwsh12 added the type/bug The issue is confirmed as a bug. label Jun 17, 2024
@ti-chi-bot ti-chi-bot bot added the affects-8.5 This bug affects the 8.5.x(LTS) versions. label Nov 1, 2024
@wshwsh12 wshwsh12 added affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. and removed may-affects-6.5 may-affects-7.1 may-affects-7.5 may-affects-8.1 may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.1 labels Nov 14, 2024
@ti-chi-bot ti-chi-bot bot closed this as completed in 140525d Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.5 This bug affects the 8.5.x(LTS) versions. impact/panic severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants