charge minimum in arrears, not in advance

[chadwhitacre]

Right now we charge $10 for a $1 tip and trickle it out over another ~9 weeks. We should do like Patreon (which I learned about under here) and let the $1 charge build up for 10 weeks and then charge the tipper. This is a little more jittery for the receiver, but a) the jitteriness will even out for any given receiver as ntippers grows, and b) this will remove a source of escrow (#1383).

As part of this, it may make sense to refund all of the <$10 balances that we can. Aaaaaaand if we want to do that, then we better get it done before Balanced shuts down. They're supporting refunds through October 9.

[chadwhitacre]

Taking this off the Balanced Shutdown milestone. We don't have the hard zero-escrow requirement as we once thought.

[chadwhitacre]

Erm, sorry. Reopening. 😊

[rorepo]

Thinking of the following approaches:
i) Add a new field to the participants table, something like 'giving_due'. The value to build up by the total of subscriptions each week
ii) Alter the usage of the 'giving' field in participants table to behave as above
iii) Add a new field to the subscriptions table to indicate the last paid date. Use this date to calculate total due
iv) Create a separate record in the subscriptions table for each week, with a new field to indicate 'paid'. In this case, we can calculate the total of unpaid subscriptions based on this field.

In each case, we check the total due, charging when we reach/pass the minimum charge threshold. In (i) and (ii), we reset the value to 0 after charging. In (iii) and (iv), we set the relevant paid date/flag after charging.

(i) and (ii) appear to be the simplest. (ii) might affect current behaviour e.g. will mean relying on a different mechanism to show a participant's current 'giving' value i.e. the current total of all weekly subscriptions
(iii) could cause difficulties if/when a subscriber's subscription values change
(iv) will mean much higher data volumes in the subscriptions table. However, changes in subscription values will not cause problems and this approach also does provide a quick view of subscription history

[chadwhitacre]

I think we need to track "giving buildup" separately from both individual intentions to give (subscriptions) as well as total amount intending to give (participants.giving), because we need to keep track of giving buildup for each team.

If Alice intends to give $3/wk to The A Team and $2/wk to The B Team, then we should charge her $10 every other week (assuming we stick with the $10 minimum charge). When we charge Alice $10, we need to be able to know that $6 should go to A, and $4 to B.

You raise a good point that we do want to track the aggregate buildup, and satisfy the minimum charge according to that.

How do we express this in terms of double-entry accounting (#3618)?

[chadwhitacre]

After one week, Alice has a $5 liability, A has a $3 asset, and B has a $2 asset. In #3618 we're looking at a ledger with reference to Gratipay, but it seems here that we might be more interested in a ledger with reference to each ~user and Team.

[chadwhitacre]

At this point I want to talk to a CPA for advice on structuring our escrow accounting. :(

[rorepo]

@whit537, the double entry accounting bit I suppose will only become clear after your discussions with the CPA. However, I think it would be ok to go ahead with the provision to monitor the giving build up, as we'd need this in any case to drive the 'charging in arrears' logic. And from your comments, approach (i) above, a new column (giving_due) in the participants table, seems to fit the bill. Shall I proceed with this?

[chadwhitacre]

the double entry accounting bit I suppose will only become clear after your discussions with the CPA. However, I think it would be ok to go ahead with the provision to monitor the giving build up

Agreed.

And from your comments, approach (i) above, a new column (giving_due) in the participants table, seems to fit the bill.

Well, but that doesn't account for buildup per team, right? It seems to me like a new giving_due column would work to track the total buildup for a given ~user; we can query that during payday to know whether they're ready to be charged. However, it seems to me that we'll need a new buildup (or some such) table to track the individual tip actualizations that go into the buildup.

[chadwhitacre]

giving_due will be a cache over buildup in the same way that giving is a cache over subscriptions.

Does that make sense, @rorepo?

[rorepo]

Does make sense, considering a user may subscribe to different teams at different times and thus may have different (no. of weeks) pending subscriptions.
What if we put the new 'giving_due' field in subscriptions rather than participants?

[chadwhitacre]

Could do. Are we looking to surface this number in the UI? That'd be the reason to cache it. The other would be payday: we could sum giving_due per-participant for payday, ya?

[rorepo]

Yes, we could certainly show this in the UI. Summing giving_due per participant, of course. It also occurs to me that it might make sense to have a giving_due column in participants as well - to cache the sum.

[chadwhitacre]

Working with the above example, we need to be able to go back and see that Alice's $6 due to The A Team is because of $3 in week N and another $3 in week N+1. What's your plan for that, @rorepo?

[rorepo]

Not sure I've got the question, @whit537. If the subscriptions table shows that Alice has a weekly subscription to team A of $3 and the giving_due column (in the same record) shows $6, that would obviously be the result of 2 weeks' subscriptions being outstanding?

[chadwhitacre]

@rorepo What happens if Alice changes her subscription amount while giving_due is $3?

[chadwhitacre]

Back on Balanced Shutdown. We didn't need this for June 11 (when they ended their charging API), but we do need it for October 9 (when they turn off their refund API).

[rohitpaulk]

but we do need it for October 9 (when they turn off their refund API)

Why? Fixing this won't affect the charges we've made in the past - they'll have to be refunded anyway. Right?

[rohitpaulk]

Out of the sources of escrow that we have, this is a minor contributor. A rough indicator for this -

SELECT SUM(balance) FROM participants; #=> 146020.55
SELECT SUM(balance) FROM participants WHERE balance < 10; #=> 13337.09
SELECT SUM(balance) FROM participants WHERE balance > 10; #=> 132563.46

[chadwhitacre]

I think I may have intended my comment to go on #1383.

[rorepo]

@whit537, the scenario of subscriptions changing while giving_due has a value - might necessitate a separate table to track giving_due. However, a couple of questions here. What currently happens when a subscription value is changed? Is the amount in the existing record amended? If so, a further question - is subscription history being recorded in any way currently?

[chadwhitacre]

What currently happens when a subscription value is changed? Is the amount in the existing record amended?

No. subscriptions is an INSERT-only table (well, we do modify is_funded via current_subscriptions). See this comment (which I've cleaned up here).

[rorepo]

Well, that confuses me a bit. If we have multiple records in subscriptions corresponding to each individual subscription (modification), seems to me we could use the giving_due column in subscriptions itself and still have a capability to trace giving_due back to individual instances of the subscription. So, is there any other aspect to this?

[chadwhitacre]

@rorepo I'm about to go afk for a week of vacation. :(

Maybe start coding based on what you think makes sense, and I or someone else can review when we have code to look at?

[rorepo]

Sure, @whit537 :) Enjoy your vacation.

[chadwhitacre]

Code is in! !m @rorepo 💃

What do we want to do with existing balances due to having charged in advance? Copying from @rorepo at #3675 (comment):

A few thoughts on refunding remaining balances charged in advance:

1. We'd need a threshold for this as well, otherwise we'd end up trying to refund even below the min payout value, say.
2. There's also the aspect that with small balances which would be consumed in a couple of weeks, there may not be much point in doing the refund. So maybe we can link the threshold to this, say, 2 weeks into subscription amount.
3. In any case, I think we should try to eliminate the situation where we charge (in arrears) and end up refunding excess balance right away.
4. Remembering @rohitpaulk's comment a while back, that excess balances as a result of being charged in advance were a relatively low percentage of transaction volumes - what would be the percentage of such balances once we've switched to charging in arrears?

[chadwhitacre]

@rohitpaulk's numbers are above at #3378 (comment). They roughly line up with my estimation of Gratipay 1.0 balances at #3744 (comment): e.g., ~10% of our escrow is from payin/out minimums, and ~90% is from Gratipay 1.0, with a trace amount due to failed payouts.

[chadwhitacre]

We'd need a threshold for this as well, otherwise we'd end up trying to refund even below the min payout value, say.

I'm not sure what the issue is here: refunds use a separate API from payouts, and the payout minimums don't apply to refunds. As far as I'm aware, there is no minimum for refunds.

There's also the aspect that with small balances which would be consumed in a couple of weeks, there may not be much point in doing the refund.

If we decide not to refund all advances, we'll certainly want to check by threshold. We are holding some escrow that was charged in advance but now the payments themselves have stopped entirely for one reason or another (see #3342, e.g.). Those should definitely be refunded.

In any case, I think we should try to eliminate the situation where we charge (in arrears) and end up refunding excess balance right away.

How would we trigger this situation? Now that we charge in arrears, there is no excess balance to refund, correct? The entire balance every week in which we charge will be directed to the receivers. (Unless, as suggested by point 1, there's a confusion here between refunding and paying out?)

[chadwhitacre]

Remembering @rohitpaulk's comment a while back, that excess balances as a result of being charged in advance were a relatively low percentage of transaction volumes - what would be the percentage of such balances once we've switched to charging in arrears?

Properly speaking, the comment was about percentage of existing escrow, not about transaction volume. Now that we're charging in arrears, our escrow-due-to-having-charged-in-advance will:

• decrease week to week as active subscriptions continue, but
• never reach zero (without further action) since some subscriptions that resulted in escrow-due-to-having-charged-in-advance have been turned off.

[chadwhitacre]

So maybe we can link the threshold to this, say, 2 weeks into subscription amount.

The easiest way to do this will be to simply run with #3675 for a couple weeks and then see where we stand. We're targeting October 8 for the big refund of 1.0 balances (#3539). How about we target October 1 to refund advances? That can act as a practice run for the 1.0 balances, and that gives us 3+ weeks to shake out any regressions from #3675, and to work on cleaning up our accounting (#3653).

[jiri-janousek]

How does charging in arrears affect receivers (teams)? Do receivers have to wait for their supporters to reach the $10 threshold to actually receive money from them to the team balance?

[chadwhitacre]

@fenryxo Yes, that is correct. I've added a note to the blog post:

For Teams receiving on Gratipay, this change means that your income may drop somewhat in the short term while your givers build back up to the minimum charge threshold of $10. Sorry for the inconvenience. :-(

I've reticketed giving more visibility into funding build-up as #3779.