use cents instead of decimal

[chadwhitacre]

I'm using Decimal everywhere for dollar values but it turns out that best practice is to deal with cents. That's how the Balanced API is set up (Stripe, too). It makes it much easier cause then we can use plain ints instead of more complicated datatypes, in both Python and Postgres.

[sigmavirus24]

👍

[zbynekwinkler]

👍

[seanlinsley]

IRC

[seanlinsley]

@whit537 can you expand on how this is beneficial? Even if we moved everything to pennies so we used whole numbers everywhere, we'd still have to use Decimal to calculate fees.

[MikeFair]

I agree with Sean, and if this is impacting the tips table, then it will
also impact funds which calculate everything based on percentages and
negatively impact the ability to deal with alternate currencies in the
future. For instance, the Korean Won and Japanese Yen don't have the
concept of "fractional" units (in essence they price everything in pennies
to begin with).

I agree about having a precision limitation but it should at least 8
decimal places and preferably 10.

[zbynekwinkler]

If we want to keep it in the new "Ready to Start" label we should reach an agreement on what the benefits are and why we want this. Right now we have 2 decimal places so going for cents does not take anything from us. What would we gain?

[chadwhitacre]

I got the impression from @mjallday at Balanced that cents is sort of the normal way to implement currency. I do see it as being simpler to work with than numeric(35, 2) and Decimal everywhere.

[chadwhitacre]

No one has assigned this to themselves, so I am removing from "Ready to Start."

[seanlinsley]

I would say close this. I'm not convinced it's something worth doing.

[zbynekwinkler]

Yeah, could anyone hash out the benefits of doing this?

[chadwhitacre]

we can use plain ints instead of more complicated datatypes, in both Python and Postgres.

I don't have anything beyond that. I have a nagging feeling that this could become an issue at scale(?) and it will be easier to fix now. I suppose we can close for now, though, in a fit of not prematurely optimizing.

[zbynekwinkler]

As I understand it we would have to use Decimal in Python anyway, wouldn't we?

[steveklabnik]

If I don't answer this in a few days, you should make me feel bad, as I do have strong feels about this...

[seanlinsley]

Context: https://twitter.com/seanlinsley/status/424003606157590528

[chadwhitacre]

!m @stevekinney @seanlinsley

[chadwhitacre]

Also, !m @steveklabnik, and @seanlinsley again (IRC). :-)

[steveklabnik]

Whew. So.

The issue basically boils down to two things:

1. Actual problems with non-integers
2. Social signaling

Let's do both in turn.

Problems with non-integer math

Okay, so, as you know, basically, computers can't really represent ℝ. So we use floating point numbers. The issue with IEEE 754 numbers is that from the ℝ perspective, you introduce inaccuracies. For example, Python, which uses double-precision IEEE 754 floats:

>>> sum = 0.0
>>> for i in range(10):
...     sum += 0.1
...
>>> sum
0.9999999999999999

Integers, however, do not have this problem. In our case, we want to represent tenths, so we can say that our base unit is 0.1, and do this instead:

>>> sum = 0
>>> for i in range(100):
...     sum += 1
...
>>> sum
10

By explicitly choosing a precision and then using integer maths, you can avoid whole classes of these kinds of errors.

You also can't Superman III: http://en.wikipedia.org/wiki/Salami_slicing

Social signaling

That said, you're probably okay. After all, the worst that happens is you have a cent or two here off, and Gittip isn't doing the kind of volume that would make this prohibitive.

But by using cents, you communicate "we understand this issue, and are following other best practices." To me, using floating point numbers in your API says "I don't understand issues around representing fractional numbers," and since that's a huuuuge part of doing money stuff....

And that's how you end up with snarky tweets like mine. Which wasn't actually even pointed at Gittip.

Conclusion

So that's what's up. Purely for the social signaling value, I think it's worth your time to change, though it might not be worth enough to prioritize it immediately.

[bruceadams]

Gittip is not using floating point for dollars. We are using a decimal type, which does have exact representations for a few digits to the right of the decimal point.

[jonah-williams]

@bruceadams sure but as per @steveklabnik's point about social signaling above I think it's valuable to make sure that is communicated clearly. When I look at https://www.gittip.com/about/paydays.json or https://www.gittip.com/about/charts.json I see an API returning monetary values as floats. This makes me suspect that Gittip may not be handling those values safely (despite internal use of Decimals). It also requires any API consumer working with that data to convert responses into a safer representation before manipulating them or worse encourages use of monetary values in an unsafe format.

I think it would be valuable to expose such data in a form which reflects a clear understanding of common problems manipulating monetary values and which encourages good practices on the part of consumers of your data. Integer amounts would be a good start, a set of an amount, unit, and currency might be even better. Perhaps deciding on a desired public representation of your data types can drive what, if any, change is needed internally.

[seanlinsley]

@jonah-carbonfive can you point us to APIs that we should emulate?

[jaredonline]

@seanlinsley in @whit537 original ticket, he points to the Balanced and Strip APIs.

Balanced: https://docs.balancedpayments.com/current/api#retrieve-a-credit
Stripe: https://stripe.com/docs/api#charges

[jaredonline]

@jonah-carbonfive touches on this, but I think it's good to make it an explicit goal of the API: You should make it difficult for users to do the wrong thing with your API. If you present the number in integer form, then a consumer of your API has to make the decision to convert it to a float.

Hopefully folks are using something like the the money gem to handle working with currency.

[jonah-williams]

As @jaredonline mentioned I think the Ruby 'money' gem is a good example of how to represent these values. I think Square offers a reasonable example of how to represent money in an API response at https://connect.squareup.com/docs/connect/datatypes though it may be useful for responses to specifically note the units used to express each currency amount.

[bruceadams]

@jonah-carbonfive makes extremely good observations about the Gittip code. I am certainly in favor of moving to using integers for money in Gttip (and elsewhere). I had supposed it wasn't too important due to the use of Decimal. I had not noticed that we aren't consistent about using decimal. Thank you @jonah-carbonfive for seeing that and pointing it out here!

[haileys]

Using Decimal everywhere to avoid floating point issues should be just fine. Using integer cents is just another means to the same end.

As for the API, the JSON spec doesn't mention anything about the internal representation of numbers. It does say this, which I found interesting:

Numeric values that cannot be represented as sequences of digits (such as Infinity and NaN) are not permitted.

This suggests that as far as JSON is concerned, there's nothing wrong with serializing monetary values as numbers and that a parser that read numbers as decimals is correct.

In fact, you could even make the argument that JSON parsers that return numbers as floats are lossy and actually incorrect. 😀

[pixeltrix]

Some arguments against using integer cents to represent currency:

1. Not all currencies are decimal based, though in reality none of them matter. However if you're programming a Harry Potter simulator you're going to struggle with integer cents.
2. Not all currencies are cent based - some have 4 digits to the right of the decimal point and some have none. If you're programming a system that has to handle multiple currencies you'll need to handle this so you now have to use 64 bit integers since dividing by 1000 will limit your maximum value to a few million.
3. Fractional sales tax rates and interest rates. Until recently the UK VAT rate was 17.5% which if you're limiting yourself to cents will result in rounding errors when calculating invoice totals. So you end up having to using an integer to represent 1/10 or 1/100 of a cent and before you know it you're re-implemented fixed point math

In 15 years of developing e-commerce systems I've never had a problem which could've been fixed by changing from decimal types to integer types.

[zbynekwinkler]

I had not noticed that we aren't consistent about using decimal.

@bruceadams Where are we not consistent?

[steveklabnik]

@pixeltrix yup, that's why you need to choose a precision too.

[chadwhitacre]

Fractional sales tax rates and interest rates.

#1673 is an example of a limitation of dealing with a fractional fee in a cent-based system.

[chadwhitacre]

It also requires any API consumer working with that data to convert responses into a safer representation before manipulating them or worse encourages use of monetary values in an unsafe format.

This is the strongest point in this thread, imo. @jeresig hit a snag using the Gittip API around this (though I don't have the reference in front of me atm).

[chadwhitacre]

+1 from @frew in IRC.

[frew]

@whit537 I haven't done anything with gittip nor have I been on IRC anywhere in the past couple of days. Maybe you meant another frew? :)

[chadwhitacre]

Sorry to bother you, @frew! :-(

[chadwhitacre]

Ping @frioux.

[chadwhitacre]

:-)

[chadwhitacre]

+1 from @rummik in IRC.

[frioux]

Some pretty smart people recently released an OSS billing system. To avoid loss of precision involved in conversions (cents are not the smallest division of currency in the US, see your local gas station for evidence) they use integer millicents, IE 1000 millicents == 1¢. Just another option, but ultimately, floats and money are not a great idea.

[rummik]

@frioux We're actually using Decimal, which maintains precision, but I find the math for cents to be the easiest to work with. That dot somehow breaks the number up too much for me >.>

Millicents would probably also be fairly easy for me to work with

[Changaco]

There are no significant advantages in using cents instead of decimal.