executor: fix naaj panic caused by wrong field types check

[AilinKid]

What problem does this PR solve?

Issue Number: close #42459

Problem Summary:

a simplest case

create table t(a decimal(40,0), b bigint(20) not null);

insert into t values(7,8),(7,8),(3,4),(3,4),(9,2),(9,2),(2,0),(2,0),(0,4),(0,4),(8,8),(8,8),(6,1),(6,1),(NULL, 0),(NULL,0);

select ( table1 . a , table1 . b ) NOT IN ( SELECT 3 , 2 UNION  SELECT 9, 2 ) AS field2 from t as table1 order by field2;

ERROR 1105 (HY000): probeWorker[0] meets error: runtime error: index out of range [560] with length 128

What is changed and how it works?

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)

MYSQL [test]>  source ~/tmp/schema.sql
MySQL [subquery_agg10000]> source ~/tmp/test.sql
+--------+--------+--------+--------+
| field1 | field2 | field3 | field4 |
+--------+--------+--------+--------+
|      0 |      1 |      0 |      1 |
|      0 |      0 |      0 |      1 |
|      0 |      1 |      0 |      0 |
+--------+--------+--------+--------+
3 rows in set, 213 warnings (0.025 sec)

• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

executor: fix naaj panic caused by wrong field types check

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• fixdb
• winoros

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[sre-bot]

All committers have signed the CLA.

[AilinKid]

/retest

[AilinKid]

/retest

[windtalker]

I don't quite understand the root cause you explained in the pr description, in my understanding, before TiDB execution a join, the planner will make sure that join key from both left/right side belongs to the same type category(for example, both decimal or both int), in your example, the join key is (decimal, int) not in (int, int), and you fix seems to say that the join key's type is not the same catagory, so the error happens, but according to the explain result, the key types are the same:

mysql> explain select ( table1 . a , table1 . b ) NOT IN ( SELECT 3 , 2 UNION  SELECT 9, 2 ) AS field2 from t as table1 order by field2;
+----------------------------------+----------+-----------+---------------+-----------------------------------------------------------------------------------------------------+
| id                               | estRows  | task      | access object | operator info                                                                                       |
+----------------------------------+----------+-----------+---------------+-----------------------------------------------------------------------------------------------------+
| Sort_16                          | 10000.00 | root      |               | Column#10                                                                                           |
| └─HashJoin_19                    | 10000.00 | root      |               | Null-aware anti left outer semi join, equal:[eq(test.t.a, Column#15) eq(test.t.b, Column#9)]        |
|   ├─Projection_22(Build)         | 2.00     | root      |               | Column#9, cast(Column#8, decimal(20,0) BINARY)->Column#15                                           |
|   │ └─HashAgg_26                 | 2.00     | root      |               | group by:Column#8, Column#9, funcs:firstrow(Column#8)->Column#8, funcs:firstrow(Column#9)->Column#9 |
|   │   └─Union_30                 | 2.00     | root      |               |                                                                                                     |
|   │     ├─Projection_32          | 1.00     | root      |               | 3->Column#8, 2->Column#9                                                                            |
|   │     │ └─TableDual_33         | 1.00     | root      |               | rows:1                                                                                              |
|   │     └─Projection_34          | 1.00     | root      |               | 9->Column#8, 2->Column#9                                                                            |
|   │       └─TableDual_35         | 1.00     | root      |               | rows:1                                                                                              |
|   └─TableReader_21(Probe)        | 10000.00 | root      |               | data:TableFullScan_20                                                                               |
|     └─TableFullScan_20           | 10000.00 | cop[tikv] | table:table1  | keep order:false, stats:pseudo                                                                      |
+----------------------------------+----------+-----------+---------------+-----------------------------------------------------------------------------------------------------+

Note that Column#15 is already casted to Decimal(20,0) before join

[elsa0520]

LGTM

[ti-chi-bot]

@elsa0520: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

LGTM

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[windtalker]

In fact, I manually align the key type before join, and it throws the same error, which I think can prove that there must be some other unknown errors

mysql> explain select ( table1 . a , table1 . b ) NOT IN ( SELECT cast(3 as decimal(40,0)) , 2 UNION  SELECT cast(9 as decimal(40,0)), 2 ) AS field2 from t as table1 order by field2;
+--------------------------------+---------+-----------+---------------+-----------------------------------------------------------------------------------------------------+
| id                             | estRows | task      | access object | operator info                                                                                       |
+--------------------------------+---------+-----------+---------------+-----------------------------------------------------------------------------------------------------+
| Sort_15                        | 16.00   | root      |               | Column#10                                                                                           |
| └─HashJoin_18                  | 16.00   | root      |               | Null-aware anti left outer semi join, equal:[eq(test.t.a, Column#8) eq(test.t.b, Column#9)]         |
|   ├─HashAgg_23(Build)          | 2.00    | root      |               | group by:Column#8, Column#9, funcs:firstrow(Column#8)->Column#8, funcs:firstrow(Column#9)->Column#9 |
|   │ └─Union_27                 | 2.00    | root      |               |                                                                                                     |
|   │   ├─Projection_29          | 1.00    | root      |               | 3->Column#8, 2->Column#9                                                                            |
|   │   │ └─TableDual_30         | 1.00    | root      |               | rows:1                                                                                              |
|   │   └─Projection_31          | 1.00    | root      |               | 9->Column#8, 2->Column#9                                                                            |
|   │     └─TableDual_32         | 1.00    | root      |               | rows:1                                                                                              |
|   └─TableReader_20(Probe)      | 16.00   | root      |               | data:TableFullScan_19                                                                               |
|     └─TableFullScan_19         | 16.00   | cop[tikv] | table:table1  | keep order:false, stats:pseudo                                                                      |
+--------------------------------+---------+-----------+---------------+-----------------------------------------------------------------------------------------------------+
mysql>  select ( table1 . a , table1 . b ) NOT IN ( SELECT cast(3 as decimal(40,0)) , 2 UNION  SELECT cast(9 as decimal(40,0)), 2 ) AS field2 from t as table1 order by field2;
ERROR 1105 (HY000): probeWorker[0] meets error: runtime error: index out of range [560] with length 128

[windtalker]

/hold

[AilinKid]

I don't quite understand the root cause you explained in the pr description, in my understanding, before TiDB execution a join, the planner will make sure that join key from both left/right side belongs to the same type category(for example, both decimal or both int), in your example, the join key is (decimal, int) not in (int, int), and you fix seems to say that the join key's type is not the same catagory, so the error happens, but according to the explain result, the key types are the same:

mysql> explain select ( table1 . a , table1 . b ) NOT IN ( SELECT 3 , 2 UNION  SELECT 9, 2 ) AS field2 from t as table1 order by field2;
+----------------------------------+----------+-----------+---------------+-----------------------------------------------------------------------------------------------------+
| id                               | estRows  | task      | access object | operator info                                                                                       |
+----------------------------------+----------+-----------+---------------+-----------------------------------------------------------------------------------------------------+
| Sort_16                          | 10000.00 | root      |               | Column#10                                                                                           |
| └─HashJoin_19                    | 10000.00 | root      |               | Null-aware anti left outer semi join, equal:[eq(test.t.a, Column#15) eq(test.t.b, Column#9)]        |
|   ├─Projection_22(Build)         | 2.00     | root      |               | Column#9, cast(Column#8, decimal(20,0) BINARY)->Column#15                                           |
|   │ └─HashAgg_26                 | 2.00     | root      |               | group by:Column#8, Column#9, funcs:firstrow(Column#8)->Column#8, funcs:firstrow(Column#9)->Column#9 |
|   │   └─Union_30                 | 2.00     | root      |               |                                                                                                     |
|   │     ├─Projection_32          | 1.00     | root      |               | 3->Column#8, 2->Column#9                                                                            |
|   │     │ └─TableDual_33         | 1.00     | root      |               | rows:1                                                                                              |
|   │     └─Projection_34          | 1.00     | root      |               | 9->Column#8, 2->Column#9                                                                            |
|   │       └─TableDual_35         | 1.00     | root      |               | rows:1                                                                                              |
|   └─TableReader_21(Probe)        | 10000.00 | root      |               | data:TableFullScan_20                                                                               |
|     └─TableFullScan_20           | 10000.00 | cop[tikv] | table:table1  | keep order:false, stats:pseudo                                                                      |
+----------------------------------+----------+-----------+---------------+-----------------------------------------------------------------------------------------------------+

Note that Column#15 is already casted to Decimal(20,0) before join

not the cast problem.

In my cases: (table1 . a , table1 . b ) NOT IN ( SELECT 3 , 2) seen as column#8 column#15
table1.a is decimal, column#8 is also cast as decimal
table1.b is string, column#15 is also the string

for the build side
the chunk data layout is: column#15, column#8,
while the buildNAkey is column#8, column#15
and the naKeyColIdx is: 1, 0 (mapping the offset in the chunk)
and the build side allTypes is column#8, column#15

for the code change:
ok, err = codec.EqualChunkRow(c.sc, mayMatchedRow, needCheckBuildTypes, needCheckBuildRowPos, probeSideRow, needCheckProbeTypes, needCheckProbeRowPos)

we pass the needCheckBuildRowPos for example 0 here, it will fetch data with the offset 0 which is exactly the column#15 there, while once we pass allTypes here, it will use the enumeration from 0 to length(needCheckBuildRowPos), which is 0 there, and from the allTypes we can see: what it gets is just a decmial type for column#8, (it's wrong, we couldn't fetch a decmial data from column#15)

approach: mapping the needCheckPos and corresponding types it needs rather than allTypes.

trigger condition:
1: multi keys in NA-join one side
2: the underlying chunk data sequence is reversed/different from the keyCols' order

[AilinKid]

PTAL @windtalker

[gengliqi]

LGTM

[ti-chi-bot]

@gengliqi: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

LGTM

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[windtalker]

/unhold

[windtalker]

LGTM

[winoros]

/retest

[winoros]

/retest

[AilinKid]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: e4a53d4

[AilinKid]

/retest

[AilinKid]

/retest

[AilinKid]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 7e984fd

[AilinKid]

/retest

[AilinKid]

/retest

[winoros]

/retest

[ti-chi-bot]

In response to a cherrypick label: new pull request created to branch release-6.5: #42525.

[ti-chi-bot]

In response to a cherrypick label: new pull request created to branch release-7.0: #42526.