Slice bounds out of range after inserting many rows in a transaction

[YangKeao]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

create table t1 (id int auto_increment key, b int, index(b));
begin;
insert into t1 (b) values (1),(2),(3),(4),(5),(6),(7),(8);
insert into t1 (b) select /*+ use_index(t1, b) */ id from t1;
insert into t1 (b) select /*+ use_index(t1, b) */ id from t1;
insert into t1 (b) select /*+ use_index(t1, b) */ id from t1;
insert into t1 (b) select /*+ use_index(t1, b) */ id from t1;
insert into t1 (b) select /*+ use_index(t1, b) */ id from t1;
insert into t1 (b) select /*+ use_index(t1, b) */ id from t1;
insert into t1 (b) select /*+ use_index(t1, b) */ id from t1;
insert into t1 (b) select /*+ use_index(t1, b) */ id from t1;

2. What did you expect to see? (Required)

Insert successfully.

3. What did you see instead (Required)

mysql> insert into t1 (b) select id from t1;
ERROR 1105 (HY000): runtime error: slice bounds out of range [-118:]

4. What is your TiDB version? (Required)

4fded5f

[YangKeao]

The backtrace:

runtime error: slice bounds out of range [-118:]
github.com/pingcap/errors.AddStack
        /home/yangkeao/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20240318064555-6bd07397691f/errors.go:178
github.com/pingcap/errors.Trace
        /home/yangkeao/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20240318064555-6bd07397691f/juju_adaptor.go:15
github.com/pingcap/tidb/pkg/util.GetRecoverError
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/util/util.go:288
github.com/pingcap/tidb/pkg/executor/internal/exec.Next.func1
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/internal/exec/executor.go:440
runtime.gopanic
        /usr/lib/go/src/runtime/panic.go:785
runtime.goPanicSliceB
        /usr/lib/go/src/runtime/panic.go:155
github.com/pingcap/tidb/pkg/tablecodec.splitIndexValueForIndexValueVersion0
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/tablecodec/tablecodec.go:1800
github.com/pingcap/tidb/pkg/tablecodec.decodeIndexKvGeneral
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/tablecodec/tablecodec.go:1897
github.com/pingcap/tidb/pkg/tablecodec.DecodeIndexKVEx
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/tablecodec/tablecodec.go:983
github.com/pingcap/tidb/pkg/executor.(*memIndexReader).decodeIndexKeyValue
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/mem_reader.go:205
github.com/pingcap/tidb/pkg/executor.(*memRowsIterForIndex).Next
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/mem_reader.go:1002
github.com/pingcap/tidb/pkg/executor.(*UnionScanExec).getAddedRow
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/union_scan.go:283
github.com/pingcap/tidb/pkg/executor.(*UnionScanExec).getOneRow
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/union_scan.go:203
github.com/pingcap/tidb/pkg/executor.(*UnionScanExec).Next
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/union_scan.go:149
github.com/pingcap/tidb/pkg/executor/internal/exec.Next
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/internal/exec/executor.go:456
github.com/pingcap/tidb/pkg/executor.insertRowsFromSelect
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/insert_common.go:470
github.com/pingcap/tidb/pkg/executor.(*InsertExec).Next
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/insert.go:357
github.com/pingcap/tidb/pkg/executor/internal/exec.Next
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/internal/exec/executor.go:456
github.com/pingcap/tidb/pkg/executor.(*ExecStmt).next
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/adapter.go:1266
github.com/pingcap/tidb/pkg/executor.(*ExecStmt).handleNoDelayExecutor
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/adapter.go:1015
github.com/pingcap/tidb/pkg/executor.(*ExecStmt).handlePessimisticDML
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/adapter.go:1073
github.com/pingcap/tidb/pkg/executor.(*ExecStmt).handleNoDelay
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/adapter.go:845
github.com/pingcap/tidb/pkg/executor.(*ExecStmt).Exec
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/executor/adapter.go:611
github.com/pingcap/tidb/pkg/session.runStmt
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/session/session.go:2288
github.com/pingcap/tidb/pkg/session.(*session).ExecuteStmt
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/session/session.go:2150
github.com/pingcap/tidb/pkg/server.(*TiDBContext).ExecuteStmt
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/server/driver_tidb.go:291
github.com/pingcap/tidb/pkg/server.(*clientConn).handleStmt
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/server/conn.go:2026
github.com/pingcap/tidb/pkg/server.(*clientConn).handleQuery
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/server/conn.go:1779
github.com/pingcap/tidb/pkg/server.(*clientConn).dispatch
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/server/conn.go:1378
github.com/pingcap/tidb/pkg/server.(*clientConn).Run
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/server/conn.go:1147
github.com/pingcap/tidb/pkg/server.(*Server).onConn
        /home/yangkeao/Projects/github.com/YangKeao/tidb/master/pkg/server/server.go:741
runtime.goexit
        /usr/lib/go/src/runtime/asm_amd64.s:1700

[YangKeao]

Direct bisect gives me #56203. However, I don't think they have any connections 🤔 .

[YangKeao]

Direct bisect gives me #56203. However, I don't think they have any connections 🤔 .

This bug will happen only when the select id from t1; is running with IndexFullScan. It's understandable the result of bisection is #56203. Now I'll add /*+ use_index(t1, b) */ and bisect again.

[YangKeao]

Oops. This issue is not related to the foreign key. I'll modify the description.

[YangKeao]

It can be reproduced by

func TestTxnInsertIndex(t *testing.T) {
	store := realtikvtest.CreateMockStoreAndSetup(t)
	tk := testkit.NewTestKit(t, store)
	tk.MustExec("use test")
	tk.MustExec("create table t1 (id int auto_increment key, b int, index(b));")

	for i := 0; i < 20; i++ {
		tk.MustExec("begin")
		tk.MustExec("insert into t1 (b) values (1),(2),(3),(4),(5),(6),(7),(8);")
		for j := 0; j < 8; j++ {
			tk.MustExec("insert into t1 (b) select /*+ use_index(t1, b) */ id from t1;")
		}
	}
}

[YangKeao]

/cc @cfzjywxk

[cfzjywxk]

@YangKeao Is it verified to be reproducable on old versions like 7.1 and 7.5?

[YangKeao]

@YangKeao Is it verified to be reproducable on old versions like 7.1 and 7.5?

No. My mistake. It's not reproducible in 7.1, 7.5 and 8.1.

[cfzjywxk]

Related to the new memdb, @you06 PTAL

[you06]

Related to the new memdb, @you06 PTAL

Yes, the test can pass when I switch back to red-black tree memdb. It's related to the iterator.

[you06]

This may be caused by reading from iterator and writing the membuffer concurrently. When the iterator runs halfway, writing to the membuffer changes the structure of the tree.

The RBT will not return keys out of the given range in this situation, but ART will.