Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Precise vector test #15

Closed
wants to merge 4 commits into from
Closed

Precise vector test #15

wants to merge 4 commits into from

Conversation

haynes1
Copy link

@haynes1 haynes1 commented Jun 18, 2019

all tests pass

@haynes1
Copy link
Author

haynes1 commented Jun 18, 2019

Original Comment**

All but one test pass.

The test that doesn't pass is test_precise_round_trip(). This test converts a f64 vector to a precise vector, then converts it back to an f64 vector and normalizes it. The result should be equivalent to just normalizing the original f64 vector but it isn't.

The problem is that after conversion, the post-conversion normalized vector is accurate to one more decimal place than the pre-conversion normalized vector.

pre: 0.2672612419124244 post: 0.26726124191242434

Potential Research:
rust-lang/rust#24557
https://stackoverflow.com/questions/50361151/how-to-deal-with-inexact-floating-point-arithmetic-results-in-rust

**problem solved with an accuracy level comparison absoluteValue(f1 -f2) < 0.000000

@haynes1 haynes1 closed this Aug 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant