TiDB panic when run with tidb_enable_collect_execution_info=0 in TiKV env

[Defined2014]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

Run integration test in TiKV env many times, it will panic forever in some rounds. And we will get log like

[2023/11/02 02:13:07.738 +08:00] [ERROR] [coprocessor.go:1117] ["copIteratorWork meet panic"] [r="runtime error: invalid memory address or nil pointer dereference"] ["stack trace"="github.com/pingcap/tidb/pkg/store/copr.(*copIteratorWorker).handleTask.func1
	/home/jenkins/agent/workspace/pingcap/tidb/periodics_integration_test/tidb/pkg/store/copr/coprocessor.go:1119
runtime.gopanic
	/usr/local/go/src/runtime/panic.go:914
runtime.panicmem
	/usr/local/go/src/runtime/panic.go:261
runtime.sigpanic
	/usr/local/go/src/runtime/signal_unix.go:861
github.com/pingcap/tidb/pkg/store/copr.(*copIteratorWorker).handleCopCache
	/home/jenkins/agent/workspace/pingcap/tidb/periodics_integration_test/tidb/pkg/store/copr/coprocessor.go:1688
github.com/pingcap/tidb/pkg/store/copr.(*copIteratorWorker).handleCopResponse
	/home/jenkins/agent/workspace/pingcap/tidb/periodics_integration_test/tidb/pkg/store/copr/coprocessor.go:1437
github.com/pingcap/tidb/pkg/store/copr.(*copIteratorWorker).handleTaskOnce
	/home/jenkins/agent/workspace/pingcap/tidb/periodics_integration_test/tidb/pkg/store/copr/coprocessor.go:1278
github.com/pingcap/tidb/pkg/store/copr.(*copIteratorWorker).handleTask
	/home/jenkins/agent/workspace/pingcap/tidb/periodics_integration_test/tidb/pkg/store/copr/coprocessor.go:1130
github.com/pingcap/tidb/pkg/store/copr.(*copIteratorWorker).run
	/home/jenkins/agent/workspace/pingcap/tidb/periodics_integration_test/tidb/pkg/store/copr/coprocessor.go:817"]

2. What did you expect to see? (Required)

No panic

3. What did you see instead (Required)

panic forever.

4. What is your TiDB version? (Required)

master

Related log link

https://do.pingcap.net/jenkins/blue/organizations/jenkins/pingcap%2Ftidb%2Fperiodics_integration_test/detail/periodics_integration_test/171/artifacts/

Log file
integration-test.txt.zip

[you06]

Seems it only affects the internal DDL tasks.

tidb/pkg/ddl/reorg.go

Line 542 in 286e852

result, err := distsql.Select(ctx.ddlJobCtx, sctx, kvReq, getColumnsTypes(handleCols))

Whether tidb_enable_collect_execution_info is enabled or not, the cache keys are same for the tasks. So after tidb_enable_collect_execution_info is turned off, if the copr cache it hit, TiDB will panic.

But the panic will be catched, so it's not such serious IMO 🤔.

[you06]

@jebter can we change it to sig/execution, since this is just a coprocessor issue, and it's not related to transaction modules?

[you06]

I checked the code and it shows this bug also exists in 6.1, 6.5 and 7.1, but the fix is not necassary since turnning tidb_enable_collect_execution_info on is a workaround. Actually only few users turnoff it.

[you06]

Since option.EnableCollectExecutionInfo is loaded from instance variable after dag is generated, this panic may happens in any statements.

tidb/pkg/distsql/distsql.go

Line 82 in 286e852

EnableCollectExecutionInfo: config.GetGlobalConfig().Instance.EnableCollectExecutionInfo.Load(),