Removing oscillations from the ticket pricing algorithm

[raedah]

Related to historic issue and PR: #584 #666

The current ticket price algorithm shows a regular pattern of wave oscillations which usually decrease in form as it swings back and forth over the desired target. Any volatility in the amount of tickets being purchased will start this turbulent cycle. https://explorer.dcrdata.org/charts#ticket-price

Running the current ticket price algorithm in the simulator (https://github.com/davecgh/dcrstakesim/) in 'full' buying mode (buy whenever possible to buy) shows the same pattern in a controlled environment. Due to the larger spikes, the waves appear smaller here but it is simply relative sizing.

The emulator also shows the same pattern in the size of the ticket pool itself.

The same pattern is found on the mainnet ticket pool size data as well. https://explorer.dcrdata.org/charts#ticket-pool-size

The oscillation pattern is far more desirable than the highly volatile original ticket price algorithm which was in use before July 2017, but there is still room for improvement. The current ticket price oscillations can be intentionally invoked by a participant or group with enough influence. Due to the lost opportunity cost of not buying tickets, its not clear there is a good financial incentive to do so. Mainnet data show that price swings have been generated that span as high as 30% and took 4 months to settle. It could also be caused unintentionally by a large staker (or group) pulling out of the market, but would still have the same effect.

The goal of this proposal is to improve the ticket price algorithm by removing the oscillations.

The algorithm currently in use, proposal 7, uses this equation at its core. Please refer to the issues mentioned at the top for an in depth explanation.

nextDiff := float64(curDiff) * poolSizeChangeRatio * targetRatio

Using a series of conditions it is possible to apply the above ratios only when needed so that the oscillation can be removed. The forces that were being applied get removed when they cross over the target and are moving in the wrong direction.

targetBalancer := 1.0
relativeBalancer := 1.0
if curPoolSizeAll > prevPoolSizeAll { // trending up
        if targetRatio < 1.0 { // below target, right direction, continue
                relativeBalancer = poolSizeChangeRatio // raise price
        }
        if targetRatio > 1.0 { // above target, wrong direction, reverse
                targetBalancer = targetRatio // raise price
        }
}
if curPoolSizeAll < prevPoolSizeAll { // trending down
        if targetRatio > 1.0 { // above target, right direction, continue
                relativeBalancer = poolSizeChangeRatio // lower price
        }
        if targetRatio < 1.0 { // below target, wrong direction, reverse
                targetBalancer = targetRatio // lower price
        }
}

nextDiff := float64(curDiff) * relativeBalancer * targetBalancer

The resulting chart shows the oscillations removed.

The pool size does go to the desired target amount and the oscillations are removed, but the initial ramp up is now not filling up the pool quick enough. To solve this I modify the current upper bound equation from
maximumStakeDiff := (float64(s.tip.totalSupply) / float64(ticketPoolSize))
to
maximumStakeDiff := (float64(s.tip.totalSupply) / float64(targetPoolSizeAll)) * targetRatio

The result now shows the pool filling up properly. The ticket price and pool size are both noticeably less volatile.

Another improvement can be made. Similar to how the upper bound was just set, we can also correct volatility caused on the down swing by using a lower price bound. The following equation can be used.

weightedIdealDiff := (float64(s.tip.stakedCoins) / float64(targetPoolSizeAll)) * targetRatio

The result shows one less major price swing. This is the final result. This algorithm brings the expired ticket ratio to 0.56% while the current algorithm in use had an expired ticket ratio of 0.60%. The ticket pool size is tracking very closely to the target, ticket price stability is achieved and oscillations no longer occur.

Here are some zoomed in comparisons to see more details.

Current algorithm, zoomed price on simulator blocks 200k - 250k.

Proposed algorithm, zoomed price on simulator blocks 200k - 250k.

Lets zoom in further to get a better scale.

Current algorithm, zoomed price on simulator blocks 215k - 250k.

Proposed algorithm, zoomed price on simulator blocks 215k - 250k.

Here is a comparison of the pool sizes zoomed in.

Current algorithm, zoomed pool size on simulator blocks 200k - 250k.

Proposed algorithm, zoomed pool size on simulator blocks 200k - 250k.

Both pool size and ticket price are able to achieve stability on the simulator in half of the amount of blocks it previously took. This occurs on a near direct correction path rather than through volatile oscillations. Zoomed comparisons show similar results on the ticket surges as well. I wont post the screenshots because it would be similar data, but it can be viewed on the simulator.

Functioning simulator example is Proposition 8 at raedah/dcrstakesim@c7b6f92

Notes:

With some modifications to the simulator it may be possible to run tests against actual recent mainnet ticket purchase data for further observation.

Tests run against ticket buying modes 'a' and 'b' on the simulator show comparable performance. Due to the the data observed from mainnet above, 'full' mode is the most similar to actual mainnet buying behavior.

In the presence of intelligent buyers who dont buy when the price is higher than necessary, I expect the proposed algorithm would return to the stable price faster than the current simulator results. In the absence of constant price oscillations, the market will be more aware of what the stable price is and as well can trust the stable price would soon return.

The only deficiency found was a slightly lower pool fill rate on simulator mode 'a'. (34805 / 40960) is 15% below target for a short period of time. I do not believe the simulator here is showing a possible real world scenario. It would require a coordinated stopping of ticket purchasing by the entire market for a period of several price windows, and then it would require a coordinated pattern of delayed buying by all participants in the market. Due to the extreme unlikely condition, I recommend not considering this particular minor test deficiency in the overall efficiency of the proposed algorithm. Even if considered as a valid deficiency, the performance is still well within acceptable limits.

All variables in the proposed algorithm need to be available for use in a production setting outside of the simulator. The variable which was previously not used is s.tip.stakedCoins and I believe it is available for production.

Ticket pool size including immature tickets can be calculated in two ways. ticketMaturity was used above. It tests with the lowest amount of expires. Using immatureTickets instead will generate a smoother ticket price pattern.
targetPoolSizeAll := ticketsPerBlock * (ticketPoolSize + ticketMaturity)
targetPoolSizeAll := (ticketsPerBlock * ticketPoolSize) + immatureTickets

There is an additional algo 9 in the commit, which optimizes for the smoothest ticket price possible. Its pool expiration rate is 1% higher. There may be a way to improve it.

[davecgh]

We discussed this a bit on matrix and agreed that it's good to have this information available here for future reference, however, for the time being, the current algorithm isn't really causing any issues and there are higher priority things to work on for now.

It is definitely something that would be reasonable to revisit in the future.