Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ROUND does not work properly for inline floats #172

Open
mensfeld opened this issue Nov 28, 2018 · 6 comments
Open

ROUND does not work properly for inline floats #172

mensfeld opened this issue Nov 28, 2018 · 6 comments

Comments

@mensfeld
Copy link

Given rule like that: ROUND(1.23456789, 3) executing Dentaku like so:

Dentaku::Calculator.new.evaluate!('ROUND(1.23456789, 3)') #=> 0.1235e1

what is surprising is the argument is not an inline:

Dentaku::Calculator.new.evaluate!('ROUND(nr, 3)', nr: 1.2345678) #=> 1.235
@mensfeld
Copy link
Author

ROUNDUP and ROUNDDOWN work as expected.

@rubysolo
Copy link
Owner

rubysolo commented Nov 29, 2018

When parsing numeric literals, the value will either become an integer or a BigDecimal (see token_scanner.rb:94), and rounding a BigDecimal still returns a BigDecimal. On the other hand, context values are used as-is without any conversion, so the second result is still a float. ROUNDUP/DOWN use floor or ceil on the scaled value and then divide by a float, so they will always coerce to float.

Maybe it would be more consistent to always force numeric functions to return integer or BigDecimal?

@mensfeld
Copy link
Author

Maybe it would be more consistent to always force numeric functions to return integer or BigDecimal?

BigDecimal would be much better and much more natural imho.

@mensfeld
Copy link
Author

mensfeld commented May 7, 2019

Not rushing but any changes here? :)

@rubysolo
Copy link
Owner

Not so far, but I'll try to take a look at this soon.

@mensfeld
Copy link
Author

Thank you. Don't hesitate if you need any help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants