sensors: Fix overflow in default decoder #61186
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@bperseghetti this is the hotfix. I'm going to work on unit tests, but I wanted to get the fix for you ASAP. |
yperess
force-pushed
the
peress/sensor_value
branch
from
537dca5 to
85ccedf
Compare
|
Math for computing the Q value was updated with slight improved accuracy. |
The default decoder would take the micro-unit value of the old sensor
value and multiply it by INT32_MAX. This would, at times, cause an
overflow for the int64_t which is the cause of some bugs like when
-7952 was used (-7952000000 * INT32_MAX < INT64_MIN). Instead the new
math converts:
- `value_u * INT32_MAX / ((1 << header->shift) * 1000000)`
to a bitmap:
- `sample.val1` consumes the upper `N` bits
- `sample.val2 * BIT(32 - N) / 1000000` consumes the lower `32-N`
bits
This both improves the accuracy, and avoids the overflow since
`shift` is guaranteed to be between 0 and 31.
Signed-off-by: Yuval Peress <peress@google.com>
yperess
force-pushed
the
peress/sensor_value
branch
from
85ccedf to
d65f159
Compare
|
Is there an easy edge case that can be tested for this? |
I want to set up a test for the default decoder separate from the sensor tests that @tristan-google is working on, but to be completely transparent I'm on vacation and I wanted to do the bare minimum to fix the issue that was found. I manually tested it by overriding the value from the akm sensor on the TDK board and checking several values at different orders of magnitude. |
teburd
approved these changes
Aug 7, 2023
tristan-google
approved these changes
Aug 8, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The default decoder would take the micro-unit value of the old sensor
value and multiply it by INT32_MAX. This would, at times, cause an
overflow for the int64_t which is the cause of some bugs like when
-7952 was used (-7952000000 * INT32_MAX < INT64_MIN). Instead the new
math converts:
value_u * INT32_MAX / ((1 << header->shift) * 1000000)to a bitmap:
sample.val1consumes the upperNbitssample.val2 * BIT(32 - N) / 1000000consumes the lower32-Nbits
This both improves the accuracy, and avoids the overflow since
shiftis guaranteed to be between 0 and 31.