-
Notifications
You must be signed in to change notification settings - Fork 15
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Enhance Precision of Fixed-Point Square Root Computations
The current square root calculation for fixed-point numbers cannot handle values less than 1 accurately. To improve precision, replace the existing method with the digit-by-digit calculation method. This approach, combined with offset manipulation, will minimize precision loss during the square root calculation.
- Loading branch information
1 parent
dd8b482
commit ac884a2
Showing
1 changed file
with
58 additions
and
29 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters