Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect behavior in from_str_radix #242

Closed
0e4ef622 opened this issue May 13, 2020 · 2 comments · Fixed by #248
Closed

Incorrect behavior in from_str_radix #242

0e4ef622 opened this issue May 13, 2020 · 2 comments · Fixed by #248
Assignees

Comments

@0e4ef622
Copy link
Contributor

use rust_decimal::prelude::*;
use num_traits::Num;

fn main() {
    // 32 ones
    println!("{:?}", Decimal::from_str_radix("11111111111111111111111111111111", 2));
}

Expected output: Ok(4294967295)
Actual output: Ok(536870912)

The binary representation of the actual output is:
100000000000000000000000000000
That's 1 one followed by 29 zeroes.

@paupino paupino self-assigned this May 20, 2020
@paupino
Copy link
Owner

paupino commented May 20, 2020

It appears this is triggering rounding - which of course, is absolutely incorrect. I can put in a simple fix, however I want to make sure this transitions across different bases.

@paupino
Copy link
Owner

paupino commented May 20, 2020

I think this is actually due to this logic:

if coeff.len() as u32 > MAX_PRECISION {
    maybe_round = true;
    break;
}

This works well when the base is 10+ as it's safe to exit out early. However this is a very naive exit when it's less than base 10.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants