-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
floating point errors #9237
Comments
@todbot I do see if I remove that last bit of mantissa (which CP doesn't have) it's |
Remove that last two bits for what CircuitPython has. As I mentioned in discord, the CircuitPython default for objects is OBJ_REPR_C. MicroPython defaults to OBJ_REPR_A. See Lines 105 to 107 in d8ca842
and following. See also #5802, #1220, #566, #230, and the MicroPython issues those issues reference. This is always going to be a problem with floats. It's worse the fewer bits of precision you have. MicroPython has improved its printing of floats over the years, and of course we've incorporated that, but there is is a trade-off between "reasonable" and seemingly more exact. We could consider going to boxed floats, but for lots of floats in lists (not array.array), it uses up a lot of storage. |
CircuitPython version
Code/REPL
Behavior
Prints out a "3" instead of a "4" on the second-to-last line and "3.999998" instead of "4.000000"
Description
The crux of the issue is that
int(0.004 * 1000)
prints out "3" instead of "4".It's unclear to me why it does this, both numbers have pretty simple representations in IEEE754 (using https://www.h-schmidt.net/FloatConverter/IEEE754.html, 0.004 is actually 0.0040000001899898052215576171875, 1000 is actually 1000)
This error does not exist in MicroPython v1.22.0 on 2023-12-27.
Here are two wokwis to compare:
Additional information
No response
The text was updated successfully, but these errors were encountered: