Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

index out of range [1] with length 1 in executor.(*PipelinedWindowExec).getRow #53600

Closed
ycybfhb opened this issue May 28, 2024 · 3 comments · Fixed by #54709
Closed

index out of range [1] with length 1 in executor.(*PipelinedWindowExec).getRow #53600

ycybfhb opened this issue May 28, 2024 · 3 comments · Fixed by #54709
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. component/executor impact/panic severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.

Comments

@ycybfhb
Copy link

ycybfhb commented May 28, 2024

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

First execute the following valid.sql
valid.txt
Then a crash occurs when executing the error.sql below
error.txt

2. What did you expect to see? (Required)

Expect no crashes

3. What did you see instead (Required)

runtime error: index out of range [1] with length 1

tidb.log:

[2024/05/27 12:54:12.442 +00:00] [INFO] [conn.go:1151] ["command dispatched failed"] [conn=1776287798] [session_alias=] [connInfo="id:1776287798, addr:10.1.2.1:57088 status:10, collation:utf8mb4_0900_ai_ci, user:tester"] [command=Query] [status="inTxn:0, autocommit:1"]  [txn_mode=PESSIMISTIC] [timestamp=450052607790022658] [err="runtime error: index out of range [1] with length 1
github.com/pingcap/errors.AddStack
	/root/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20240318064555-6bd07397691f/errors.go:178
github.com/pingcap/errors.Trace
	/root/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20240318064555-6bd07397691f/juju_adaptor.go:15
github.com/pingcap/tidb/pkg/util.GetRecoverError
	/workspace/source/tidb/pkg/util/util.go:304
github.com/pingcap/tidb/pkg/executor/internal/exec.Next.func1
	/workspace/source/tidb/pkg/executor/internal/exec/executor.go:394
runtime.gopanic
	/usr/local/go/src/runtime/panic.go:914
runtime.goPanicIndexU
	/usr/local/go/src/runtime/panic.go:120
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).getRow
	/workspace/source/tidb/pkg/executor/pipelined_window.go:244
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).getEnd
	/workspace/source/tidb/pkg/executor/pipelined_window.go:306
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).enoughToProduce
	/workspace/source/tidb/pkg/executor/pipelined_window.go:437
github.com/pingcap/tidb/pkg/executor.(*PipelinedWindowExec).Next
	/workspace/source/tidb/pkg/executor/pipelined_window.go:120
github.com/pingcap/tidb/pkg/executor/internal/exec.Next
	/workspace/source/tidb/pkg/executor/internal/exec/executor.go:410
github.com/pingcap/tidb/pkg/executor.(*shuffleWorker).run
	/workspace/source/tidb/pkg/executor/shuffle.go:409
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1650"]

4. What is your TiDB version? (Required)

+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tidb_version()                                                                                                                                                                                                                                                   |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Release Version: v8.2.0-alpha-216-gfe5858b
Edition: Community
Git Commit Hash: fe5858b00cd63808ac414c6e102a353778b0aaa7
Git Branch: HEAD
UTC Build Time: 2024-05-23 01:44:42
GoVersion: go1.21.10
Race Enabled: false
Check Table Before Drop: false
Store: tikv |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

We are the BASS team from the School of Cyber Science and Technology at Beihang University. Our main focus is on system software security, operating systems, and program analysis research, as well as the development of automated program testing frameworks for detecting software defects. Using our self-developed database vulnerability testing tool, we have identified the above-mentioned vulnerabilities in TiDB that may lead to database crashes.

@XuHuaiyu
Copy link
Contributor

XuHuaiyu commented Jun 7, 2024

@ycybfhb

Never mind.

I tried to reproduce this issue on my local machine, and this SQL query ran for over 2 hours without finishing. It kept running calculations without any errors reported to the client.
How long did it take for the error to appear on your end?
Here is the execution plan for my SQL query. Does it match the execution plan you had when you encountered this issue?

+---------------------------------------------------------------+---------------------+--------------+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                                                            | estRows             | task         | access object | operator info                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+---------------------------------------------------------------+---------------------+--------------+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Projection_234                                                | 50.00               | root         |               | -862238766->Column#196, test1.t_tjov.c_irfjyds3, test1.t_tjov.c_kq8lwb, <nil>->Column#201                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| └─TopN_237                                                    | 50.00               | root         |               | test1.t_tjov.c_irfjyds3, test1.t_tjov.c_kq8lwb, offset:0, count:50                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|   └─Apply_243                                                 | 10000.00            | root         |               | CARTESIAN inner join                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|     ├─TableReader_248(Build)                                  | 10000.00            | root         |               | MppVersion: 2, data:ExchangeSender_247                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
|     │ └─ExchangeSender_247                                    | 10000.00            | mpp[tiflash] |               | ExchangeType: PassThrough                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
|     │   └─TableFullScan_246                                   | 10000.00            | mpp[tiflash] | table:ref_7   | keep order:false, stats:pseudo                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
|     └─Selection_249(Probe)                                    | 8000.00             | root         |               | lt(NULL, Column#195)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|       └─Projection_250                                        | 10000.00            | root         |               | test1.t_tjov.c_tutmh1->Column#195                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
|         └─Limit_253                                           | 10000.00            | root         |               | offset:0, count:1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
|           └─Selection_254                                     | 10000.00            | root         |               | or(not(like(test1.t_tjov.c_mzh, "%gw", 92)), 0)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
|             └─Projection_255                                  | 10000.00            | root         |               | 1->Column#202                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
|               └─Shuffle_278                                   | 10000.00            | root         |               | execution info: concurrency:5, data sources:[Projection_258]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
|                 └─Window_256                                  | 10000.00            | root         |               | nth_value(Column#185, 32)->Column#186 over(partition by test1.t_n.c_tf_us39fv, test1.t_n.c_tf_us39fv, test1.t_db8a.c_jr2kq78hqt, test1.t_n.c_v order by test1.t_db8a.c_t9, test1.t_db8a.c_d, test1.t_db8a.c_jr2kq78hqt, test1.t_db8a.c_yh_3, test1.t_n.c_tf_us39fv, test1.t_n.c_pf6z4p, test1.t_n.c_v13rl, test1.t_n.c_fr3uo, test1.t_n.c_v, test1.t_n.c_mm8jnf, test1.t_n.c__57bl_eu6, test1.t_n.c_tf_us39fv, test1.t_n.c_pf6z4p, test1.t_n.c_v13rl, test1.t_n.c_fr3uo, test1.t_n.c_v, test1.t_n.c_mm8jnf, test1.t_n.c__57bl_eu6, test1.t_n.c_tf_us39fv, test1.t_n.c_pf6z4p, test1.t_n.c_v13rl, test1.t_n.c_fr3uo, test1.t_n.c_v, test1.t_n.c_mm8jnf, test1.t_n.c__57bl_eu6 range between unbounded preceding and current row) |
|                   └─Sort_277                                  | 9990000000000000.00 | root         |               | test1.t_n.c_tf_us39fv, test1.t_n.c_tf_us39fv, test1.t_db8a.c_jr2kq78hqt, test1.t_n.c_v, test1.t_db8a.c_t9, test1.t_db8a.c_d, test1.t_db8a.c_jr2kq78hqt, test1.t_db8a.c_yh_3, test1.t_n.c_tf_us39fv, test1.t_n.c_pf6z4p, test1.t_n.c_v13rl, test1.t_n.c_fr3uo, test1.t_n.c_v, test1.t_n.c_mm8jnf, test1.t_n.c__57bl_eu6, test1.t_n.c_tf_us39fv, test1.t_n.c_pf6z4p, test1.t_n.c_v13rl, test1.t_n.c_fr3uo, test1.t_n.c_v, test1.t_n.c_mm8jnf, test1.t_n.c__57bl_eu6, test1.t_n.c_tf_us39fv, test1.t_n.c_pf6z4p, test1.t_n.c_v13rl, test1.t_n.c_fr3uo, test1.t_n.c_v, test1.t_n.c_mm8jnf, test1.t_n.c__57bl_eu6                                                                                                                    |
|                     └─ShuffleReceiver_281                     | 999000000000.00     | root         |               |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
|                       └─Projection_258                        | 9990000000000000.00 | root         |               | test1.t_n.c_pf6z4p, test1.t_n.c__57bl_eu6, test1.t_n.c_fr3uo, test1.t_db8a.c_yh_3, test1.t_n.c_v, test1.t_n.c_tf_us39fv, test1.t_n.c_tf_us39fv, test1.t_n.c_tf_us39fv, test1.t_db8a.c_jr2kq78hqt, test1.t_n.c_v, test1.t_db8a.c_t9, test1.t_db8a.c_d, test1.t_n.c_v13rl, test1.t_n.c_mm8jnf, test1.t_n.c__57bl_eu6, test1.t_n.c_pf6z4p, test1.t_n.c_v13rl, test1.t_n.c_fr3uo, test1.t_n.c_v, test1.t_n.c_mm8jnf, test1.t_n.c_pf6z4p, test1.t_n.c_v13rl, test1.t_n.c_fr3uo, test1.t_n.c_mm8jnf, test1.t_n.c__57bl_eu6, cast(test1.t_n.c_tf_us39fv, bigint(22) BINARY)->Column#185                                                                                                                                                |
|                         └─HashJoin_259                        | 9990000000000000.00 | root         |               | CARTESIAN left outer join                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
|                           ├─TableReader_276(Build)            | 80000000.00         | root         |               | data:Selection_275                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|                           │ └─Selection_275                   | 80000000.00         | cop[tikv]    |               | istrue_with_null(cast(lt(cast(test1.t_n.c_v, double BINARY), cast(cast(test1.t_n.c_tf_us39fv, bigint(22) BINARY), double BINARY)), bigint(22) UNSIGNED BINARY))                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
|                           │   └─TableFullScan_274             | 100000000.00        | cop[tikv]    | table:ref_6   | keep order:false, stats:pseudo                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
|                           └─Projection_261(Probe)             | 1248750000000.00    | root         |               | test1.t_db8a.c_t9, test1.t_db8a.c_d, test1.t_db8a.c_jr2kq78hqt, test1.t_db8a.c_yh_3, test1.t_n.c_tf_us39fv, test1.t_n.c_pf6z4p, test1.t_n.c_v13rl, test1.t_n.c_fr3uo, test1.t_n.c_v, test1.t_n.c_mm8jnf, test1.t_n.c__57bl_eu6, test1.t_n.c_tf_us39fv, test1.t_n.c_pf6z4p, test1.t_n.c_v13rl, test1.t_n.c_fr3uo, test1.t_n.c_v, test1.t_n.c_mm8jnf, test1.t_n.c__57bl_eu6                                                                                                                                                                                                                                                                                                                                                       |
|                             └─HashJoin_262                    | 1248750000000.00    | root         |               | CARTESIAN inner join                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                               ├─TableReader_273(Build)        | 100000000.00        | root         |               | data:TableFullScan_272                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
|                               │ └─TableFullScan_272           | 100000000.00        | cop[tikv]    | table:ref_3   | keep order:false, stats:pseudo                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
|                               └─HashJoin_264(Probe)           | 124875000.00        | root         |               | inner join, equal:[eq(test1.t_n.c_v, test1.t_n.c_v13rl)]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
|                                 ├─TableReader_271(Build)      | 99900000.00         | root         |               | data:Selection_270                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|                                 │ └─Selection_270             | 99900000.00         | cop[tikv]    |               | not(isnull(test1.t_n.c_v13rl))                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
|                                 │   └─TableFullScan_269       | 100000000.00        | cop[tikv]    | table:ref_5   | keep order:false, stats:pseudo                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
|                                 └─TableReader_268(Probe)      | 99900000.00         | root         |               | data:Selection_267                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|                                   └─Selection_267             | 99900000.00         | cop[tikv]    |               | not(isnull(test1.t_n.c_v))                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
|                                     └─TableFullScan_266       | 100000000.00        | cop[tikv]    | table:ref_4   | keep order:false, stats:pseudo                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+---------------------------------------------------------------+---------------------+--------------+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

@XuHuaiyu
Copy link
Contributor

XuHuaiyu commented Jun 7, 2024

A minor case that can reproduce this issue:

select 
  1 
from 
  t_tjov 
where 
  (
    select 
      t_tjov.c_tutmh1 as c0 
    from 
      (
        select nth_value(t_n.c_tf_us39fv, 1) over (partition by t_db8a.c_jr2kq78hqt order by t_n.c_mm8jnf) as c5 from t_db8a 
        join 
        t_n
      ) as tbl 
    limit 
      1
  );
tidb:4000 [test1]> desc select    1  from    t_tjov  where    (     select        t_tjov.c_tutmh1 as c0      from        (         select nth_value(t_n.c_tf_us39fv, 1) over (partition by t_db8a.c_jr2kq78hqt orde
r by t_n.c_mm8jnf) as c5 from t_db8a          join          t_n       ) as tbl      limit        1   );
+-----------------------------------------------+---------+-----------+---------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                                            | estRows | task      | access object                                                             | operator info                                                                                                                                                             |
+-----------------------------------------------+---------+-----------+---------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Projection_20                                 | 8.00    | root      |                                                                           | 1->Column#22                                                                                                                                                              |
| └─Apply_22                                    | 8.00    | root      |                                                                           | CARTESIAN inner join                                                                                                                                                      |
|   ├─IndexReader_29(Build)                     | 8.00    | root      |                                                                           | index:IndexFullScan_28                                                                                                                                                    |
|   │ └─IndexFullScan_28                        | 8.00    | cop[tikv] | table:t_tjov, index:t_xs_idx_1(c_irfjyds3, c_k4duc4l, c_tutmh1, c_kq8lwb) | keep order:false, stats:pseudo                                                                                                                                            |
|   └─Selection_30(Probe)                       | 6.40    | root      |                                                                           | Column#21                                                                                                                                                                 |
|     └─Projection_31                           | 8.00    | root      |                                                                           | test1.t_tjov.c_tutmh1->Column#21                                                                                                                                          |
|       └─Limit_34                              | 8.00    | root      |                                                                           | offset:0, count:1                                                                                                                                                         |
|         └─Shuffle_48                          | 8.00    | root      |                                                                           | execution info: concurrency:3, data sources:[HashJoin_37]                                                                                                                 |
|           └─Window_35                         | 8.00    | root      |                                                                           | nth_value(test1.t_n.c_tf_us39fv, 1)->Column#19 over(partition by test1.t_db8a.c_jr2kq78hqt order by test1.t_n.c_mm8jnf range between unbounded preceding and current row) |
|             └─Sort_47                         | 7168.00 | root      |                                                                           | test1.t_db8a.c_jr2kq78hqt, test1.t_n.c_mm8jnf                                                                                                                             |
|               └─ShuffleReceiver_49            | 896.00  | root      |                                                                           |                                                                                                                                                                           |
|                 └─HashJoin_37                 | 7168.00 | root      |                                                                           | CARTESIAN inner join                                                                                                                                                      |
|                   ├─IndexReader_41(Build)     | 32.00   | root      |                                                                           | index:IndexFullScan_40                                                                                                                                                    |
|                   │ └─IndexFullScan_40        | 32.00   | cop[tikv] | table:t_db8a, index:c_jr2kq78hqt(c_jr2kq78hqt)                            | keep order:false, stats:pseudo                                                                                                                                            |
|                   └─TableReader_45(Probe)     | 1792.00 | root      |                                                                           | data:TableFullScan_44                                                                                                                                                     |
|                     └─TableFullScan_44        | 1792.00 | cop[tikv] | table:t_n                                                                 | keep order:false, stats:pseudo                                                                                                                                            |
+-----------------------------------------------+---------+-----------+---------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
16 rows in set (0.00 sec)

@XuHuaiyu
Copy link
Contributor

XuHuaiyu commented Jun 7, 2024

The issue if fixed after the following modification.

diff --git a/pkg/executor/pipelined_window.go b/pkg/executor/pipelined_window.go
index 9bd7ea5256..78dabceac3 100644
--- a/pkg/executor/pipelined_window.go
+++ b/pkg/executor/pipelined_window.go
@@ -90,6 +90,17 @@ func (e *PipelinedWindowExec) Open(ctx context.Context) (err error) {
        e.dropped = 0
        e.data = make([]dataInfo, 0)
        e.dataIdx = 0
+       e.curRowIdx = 0
+       e.newPartition = false
+       e.lastStartRow = 0
+       e.lastEndRow = 0
+       e.stagedStartRow = 0
+       e.stagedEndRow = 0
+       e.rowCnt = 0
+       e.whole = false 
+       e.rowStart = 0
+       e.rowCnt = 0
+       e.initializedSlidingWindow = false
        e.slidingWindowFuncs = make([]aggfuncs.SlidingWindowAggFunc, len(e.windowFuncs))
        for i, windowFunc := range e.windowFuncs {
                if slidingWindowAggFunc, ok := windowFunc.(aggfuncs.SlidingWindowAggFunc); ok {
tidb:4001 [test1]> select    1  from    t_tjov  where    (     select        t_tjov.c_tutmh1 as c0      from        (         select nth_value(t_n.c_tf_us39fv, 1) over (partition by t_db8a.c_jr2kq78hqt order by t_n.c_mm8jnf) as c5 from t_db8a          join          t_n       ) as tbl      limit        1   );
+---+
| 1 |
+---+
| 1 |
| 1 |
| 1 |
| 1 |
| 1 |
| 1 |
| 1 |
+---+
7 rows in set (0.00 sec)

@XuHuaiyu XuHuaiyu added affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. and removed may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.1 may-affects-6.5 may-affects-7.1 may-affects-7.5 may-affects-8.1 labels Jul 18, 2024
@ti-chi-bot ti-chi-bot bot closed this as completed in a541a4f Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. affects-7.5 This bug affects the 7.5.x(LTS) versions. affects-8.1 This bug affects the 8.1.x(LTS) versions. component/executor impact/panic severity/major sig/execution SIG execution type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants