Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LogBuffer 解析 Decimal 数据未补齐末尾0的精度问题 #5313

Open
F-ca7 opened this issue Oct 30, 2024 · 0 comments
Open

LogBuffer 解析 Decimal 数据未补齐末尾0的精度问题 #5313

F-ca7 opened this issue Oct 30, 2024 · 0 comments

Comments

@F-ca7
Copy link

F-ca7 commented Oct 30, 2024

复现方式:

// 字段类型 decimal(64, 16)
public static void main(String[] args) {
    byte[] bytes = new byte[] {-128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
    LogBuffer logBuffer = new LogBuffer(bytes, 0, bytes.length);
    BigDecimal decimal = logBuffer.getDecimal(64, 16);
    System.out.println(decimal.toPlainString());
}

预期输出:"0.0000000000000000" (16个0)

实际输出:"0.000000000" (9个0)

错误原因:解析最后一段小数的时候,没有补齐0至原始数据类型定义的小数精度

修复方案:补齐指定scale后的0

for (int i = pos - mark; i < frac; i++) {
    buf[pos++] = ('0');
}

MySQL 源码中最后也有补齐0的操作:

// Append padding if a fixed precision was specified.
for (int i = 0; i < fill; ++i) *to++ = '0';
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant