Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: value/uncertainty AutoDigits does not normalize decimal input before finding bottom digit but value AutoDigits does #148

Closed
jagerber48 opened this issue Feb 8, 2024 · 1 comment

Comments

@jagerber48
Copy link
Owner

See

from decimal import Decimal

from sciform import AutoDigits, Formatter

formatter = Formatter(ndigits=AutoDigits)

print(formatter(Decimal("1.00")))
# 1
print(formatter(Decimal("1.000")))
# 1
print(formatter(Decimal("1.00"), Decimal("0.10")))
# 1.00 ± 0.10
print(formatter(Decimal("1.000"), Decimal("0.100")))
# 1.000 ± 0.100

These two behaviors shouldn't differ. It's not actually obvious which should be preferred. Due to current implementation details, the value decimal is always normalized before determining the least significant decimal place, while in the value/uncertainty mode the normalization does not necessarily happen.

My inclination is that normalization should happen at the first input step and at the formatting step to guarantee equal inputs get equal outputs. The argument against this is that one of the purposes of the Decimal encoding is to preserve significance of digits which are not technically significant digits.

@jagerber48 jagerber48 changed the title value/uncertainty AutoDigits does not normalize decimal input before finding bottom digit but value AutoDigits does Bug: value/uncertainty AutoDigits does not normalize decimal input before finding bottom digit but value AutoDigits does Feb 8, 2024
@jagerber48
Copy link
Owner Author

Closed by #149, All imports are normalized early

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant