Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test edge clipping #3

Closed
wants to merge 8 commits into from
Closed

Conversation

haynes1
Copy link
Contributor

@haynes1 haynes1 commented Jun 18, 2019

All but one test pass.

The test that doesn't pass is test_precise_round_trip(). This test converts a f64 vector to a precise vector, then converts it back to an f64 vector and normalizes it. The result should be equivalent to just normalizing the original f64 vector but it isn't.

The problem is that after conversion, the post-conversion normalized vector is accurate to one more decimal place than the pre-conversion normalized vector.

pre: 0.2672612419124244 post: 0.26726124191242434

Potential Research:
rust-lang/rust#24557
https://stackoverflow.com/questions/50361151/how-to-deal-with-inexact-floating-point-arithmetic-results-in-rust

@haynes1 haynes1 closed this Jun 18, 2019
@haynes1 haynes1 deleted the test-edge-clipping branch June 18, 2019 13:22
@haynes1 haynes1 restored the test-edge-clipping branch June 18, 2019 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants