Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

util: fix bad number error with DISTINCT when dividing long decimals #21783

Merged
merged 5 commits into from
Dec 18, 2020

Conversation

TszKitLo40
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #15563

Problem Summary:

What is changed and how it works?

What's Changed:
fix bad number error with DISTINCT when dividing long decimals
How it Works:
set precision and frac before calling EncodeDecimal

Related changes

Check List

Tests

  • Unit test
  • Integration test

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM

Release note

  • No release note

@TszKitLo40 TszKitLo40 marked this pull request as ready for review December 15, 2020 03:54
@TszKitLo40 TszKitLo40 requested a review from a team as a code owner December 15, 2020 03:54
@TszKitLo40 TszKitLo40 requested review from qw4990 and removed request for a team December 15, 2020 03:54
@github-actions github-actions bot added the sig/execution SIG execution label Dec 15, 2020
@TszKitLo40
Copy link
Contributor Author

/run-all-tests

@ichn-hu ichn-hu mentioned this pull request Dec 15, 2020
@b41sh
Copy link
Member

b41sh commented Dec 15, 2020

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Dec 15, 2020
@XuHuaiyu XuHuaiyu added the type/bugfix This PR fixes a bug. label Dec 15, 2020
Comment on lines 1217 to 1219
if frac > mysql.MaxDecimalScale {
frac = mysql.MaxDecimalScale
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add this to EncodeDecimal rather than here? Because EncodeDecimal is also called elsewhere, this problem may also occur here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think so and I have moved the logic to EncodeDecimal.

Copy link
Contributor

@lzmhhh123 lzmhhh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot removed the status/LGT1 Indicates that a PR has LGTM 1. label Dec 18, 2020
@ti-srebot ti-srebot added the status/LGT2 Indicates that a PR has LGTM 2. label Dec 18, 2020
@lzmhhh123
Copy link
Contributor

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Dec 18, 2020
@ti-srebot
Copy link
Contributor

Your auto merge job has been accepted, waiting for:

  • 21844

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot ti-srebot merged commit e9b11b7 into pingcap:master Dec 18, 2020
ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Apr 21, 2021
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor

cherry pick to release-4.0 in PR #24188

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bad Number: error with DISTINCT when dividing long decimals
7 participants