Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

executor: fix memory corrupt in COUNT/JSON_OBJECTAGG/GROUP_CONCAT #17106

Merged
merged 8 commits into from
May 13, 2020
Merged

executor: fix memory corrupt in COUNT/JSON_OBJECTAGG/GROUP_CONCAT #17106

merged 8 commits into from
May 13, 2020

Conversation

SunRunAway
Copy link
Contributor

@SunRunAway SunRunAway commented May 12, 2020 •
edited
Loading

What problem does this PR solve?

Issue Number: close #17148

Problem Summary:

What is changed and how it works?

Proposal: xxx

What's Changed:

Add a unit test, corrupt the original chunk memory after UpdatePartialResult is called. It exposes two more problem

  1. Fix memory reference problem of count(distinct <string>), It's a bug introduced by executor: optimize count distinct with single column #15323
  2. Fix memory reference problem of JSON_OBJECTAGG, exposed by the unit test.
  3. Fix memory reference problem of GROUP_CONCAT(ORDER BY <string>), exposed by the unit test.

Copy the string to avoid memory reference.

How it Works:

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Manual test (add detailed scripts or steps below)

Side effects

  • None

Release note

  • fix potential incorrect results in COUNT/JSON_OBJECTAGG/GROUP_CONCAT with varchar arguments.

@SunRunAway
Copy link
Contributor Author

/run-all-tests

@sre-bot

This comment has been minimized.

Sign in to view
1 similar comment
@sre-bot

This comment has been minimized.

Sign in to view
@sre-bot

This comment has been minimized.

Sign in to view
1 similar comment
@sre-bot

This comment has been minimized.

Sign in to view
@SunRunAway SunRunAway changed the title executor: fix memory corrupt in distinct agg function with string executor: fix memory corrupt in count(distinct string) May 12, 2020
@SunRunAway SunRunAway marked this pull request as ready for review May 12, 2020 14:15
@SunRunAway SunRunAway requested a review from a team as a code owner May 12, 2020 14:15
@ghost ghost requested review from XuHuaiyu and removed request for a team May 12, 2020 14:15
@SunRunAway SunRunAway added the type/bugfix This PR fixes a bug. label May 12, 2020
@SunRunAway SunRunAway added priority/P1 The issue has P1 priority. priority/release-blocker This issue blocks a release. Please solve it ASAP. labels May 12, 2020
@SunRunAway
Copy link
Contributor Author

/run-all-tests

@SunRunAway SunRunAway changed the title executor: fix memory corrupt in count(distinct string) executor: fix memory corrupt in COUNT/JSON_OBJECTAGG/GROUP_CONCAT May 13, 2020
@SunRunAway
Copy link
Contributor Author

/run-all-tests

@codecov
Copy link

codecov bot commented May 13, 2020 •
edited
Loading

Codecov Report

❗ No coverage uploaded for pull request base (master@3f2d35a). Click here to learn what that means.
The diff coverage is 100.0000%.

@@             Coverage Diff             @@
##             master     #17106   +/-   ##
===========================================
  Coverage          ?   79.9977%           
===========================================
  Files             ?        510           
  Lines             ?     139049           
  Branches          ?          0           
===========================================
  Hits              ?     111236           
  Misses            ?      18847           
  Partials          ?       8966

@zz-jason
Copy link
Member

Would it impact the performance of these functions?

@SunRunAway
Copy link
Contributor Author

Would it impact the performance of these functions?

It has a little impact which cant be seen in benchmark

winoros
winoros reviewed May 13, 2020
View reviewed changes
Copy link
Member

@winoros winoros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@SunRunAway SunRunAway requested a review from a team as a code owner May 13, 2020 08:06
@SunRunAway SunRunAway requested a review from zz-jason May 13, 2020 08:06
@ghost ghost removed their request for review May 13, 2020 08:06
zz-jason
zz-jason approved these changes May 13, 2020
View reviewed changes
Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zz-jason zz-jason added status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. labels May 13, 2020
@sre-bot
Copy link
Contributor

sre-bot commented May 13, 2020

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented May 13, 2020

@SunRunAway merge failed.

@github-actions github-actions bot added the sig/execution SIG execution label May 13, 2020
@SunRunAway SunRunAway merged commit a1763c1 into pingcap:master May 13, 2020
@SunRunAway SunRunAway deleted the aggdistinctstring branch May 13, 2020 08:58
sre-bot pushed a commit to sre-bot/tidb that referenced this pull request May 13, 2020
@SunRunAway @sre-bot
cherry pick pingcap#17106 to release-3.1
c13f228 
Signed-off-by: sre-bot <sre-bot@pingcap.com>
@sre-bot sre-bot mentioned this pull request May 13, 2020
Closed
@sre-bot
Copy link
Contributor

sre-bot commented May 13, 2020

cherry pick to release-3.1 in PR #17172

sre-bot pushed a commit to sre-bot/tidb that referenced this pull request May 13, 2020
@SunRunAway @sre-bot
cherry pick pingcap#17106 to release-4.0
796493c 
Signed-off-by: sre-bot <sre-bot@pingcap.com>
@sre-bot sre-bot mentioned this pull request May 13, 2020
Closed
@sre-bot
Copy link
Contributor

sre-bot commented May 13, 2020

cherry pick to release-4.0 in PR #17173

SunRunAway added a commit to SunRunAway/tidb that referenced this pull request May 14, 2020
@SunRunAway
executor: fix memory corrupt in COUNT/JSON_OBJECTAGG/GROUP_CONCAT (pi…
ba6abf0 
…ngcap#17106)
@SunRunAway SunRunAway mentioned this pull request May 14, 2020
Merged
SunRunAway added a commit to SunRunAway/tidb that referenced this pull request May 14, 2020
@SunRunAway
executor: fix memory corrupt in COUNT/JSON_OBJECTAGG/GROUP_CONCAT (pi…
29a4569 
…ngcap#17106)
@SunRunAway SunRunAway mentioned this pull request May 14, 2020
Merged
SunRunAway added a commit that referenced this pull request May 14, 2020
@SunRunAway
executor: fix memory corrupt in COUNT/JSON_OBJECTAGG/GROUP_CONCAT (#1…
601a4be 
…7106) (#17193)
sre-bot pushed a commit that referenced this pull request May 18, 2020
@SunRunAway
executor: fix memory corrupt in COUNT/JSON_OBJECTAGG/GROUP_CONCAT (#1…
4211cb7 
…7106) (#17194)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@winoros winoros winoros left review comments

@zz-jason zz-jason zz-jason approved these changes

Assignees
No one assigned
Labels
component/expression priority/P1 The issue has P1 priority. priority/release-blocker This issue blocks a release. Please solve it ASAP. sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Incorrect results with count(distinct string)
4 participants
@SunRunAway @sre-bot @zz-jason @winoros