update calculate_absolute_max_short

[dpaiton]

Calculate Absolute Max Short

PR summary

This PR rewrites the calculate absolute max short function such that the return value is guaranteed to

1. be solvent
2. be within bond_tolerance additional bonds of insolvency
3. return early if a solution is found

A user can tune bond_tolerance and max_iterations to trade off between accuracy and runtime.

The function uses a combination of Newton’s Method and binary search. I had to do this because

1. the optimal metric, solvency, does not behave appropriately for Newton’s Method to work alone (see below)
2. while FixedPoint supports negative numbers, much of the library still does not support negative numbers or insolvent pool states

In most test cases, I found that the binary search was either not used at all, or was required for almost all steps of Newton’s method. From what I can tell, the difference is based on the initial pool state. Requiring a binary search means Newton’s Method overshot the maximum and we had to back off to a solvent amount. The number of binary search steps required increases with each iteration of Newton’s Method, which is expected because the algorithm is nearing the optimal result. I opted to hardcode the maximum number of binary search steps to 100k. The maximum required steps that I noticed in my testing was order 100.

The primary test is fuzz_calculate_absolute_max_short. You can tune bonds_tolerance and max_iterations to get it to pass. I was able to get it to pass with max_iterations=500 and bonds_tolerance=fixed!(1e11) with 10k fuzz runs. The PR uses 100 fuzz runs to improve CI time.

Minor changes:

• lots of docs rewrites to unify language used
• rename maybe_tolerance argument in calculate_short_bonds_given_deposit to maybe_base_tolerance

Long-form summary of a solvency issue that I had to understand to fix this function.

tl;dr — opening a short (and possibly other functions) does not affect pool reserves in a regular way because of fees, and we must take care when using optimization functions.

Problem statement:

Solvency after short does not always go down and is not guaranteed to be monotonic.

Solution:

While both problems are true in theory, the former (increasing or decreasing solvency) is very likely in testing and the latter (non-monotonicity) doesn’t appear to happen in testing. Thus, Newton’s Method is probably still a fine optimization algorithm as long as you include safeguards to ensure the output is valid.

eli5 (years working for DELV):

Solvency is a metric that can loosely be thought of as "how many reserves are available in the pool for trading." The Hyperdrive actions that (usually) decrease solvency are opening longs, opening shorts, and removing liquidity. We're only going to worry about solvency wrt opening a short here. In this case, solvency is measured in shares and (usually) goes down as the short size goes up. The minimum shares required for solvency is never zero because of all kinds of factors, including exposure from previous trades and the minimum share reserve level set in PoolConfig.

Let's say we want to open a short. The amount we want to trade, in bonds, is ∆y. In math terms, we can define a "short function" ∆z = f(∆y), which reads "the change in pool shares equals some function of the amount of bonds shorted". In the case of Hyperdrive, we adjust the shares by subtracting ∆z, aka z_1 = z_0 - ∆z. The shares (usually) go down, so ∆z > 0 most of the time. Solvency is a check that the final z_1 is good; so we can think of some solvency value S(∆z) that tells us the amount of available shares remaining in the pool after the short. In practice we pass a bond amount to check solvency after a short, which is computing S(f(∆y)).

Finally, in addition to observing whether ∆z goes up or down as we increase ∆y, we are interested in knowing if it always has that behavior. This is known as monotonicity. It’s not about the direction of change, but only about consistency. Here is an ad lib style definition:

Any [choose: positive OR negative] ∆y always leads to [choose: positive OR negative] ∆z.

As long as “any” and “always” are true, then it is monotonic for any choice of “positive” or “negative” in either position. Monotonicity is often a requirement for convergence of root-finding algorithms, such as Newton’s Method.

Given the above, I have identified two properties of shorts:

1. The function value ∆z = f(∆y) does not always go down as we increase ∆y. Therefore, solvency does not always decrease with increased ∆y.
2. For a fixed constants, f(∆y) is not always monotonic. Therefore, solvency is not always monotonic.

This matters when we are doing complicated stuff, like any of our so-called "inverse functions": calculate_absolute_max_short, calculate_short_bonds_given_deposit, preview_close_short, etc.

Analytical evidence:

Recall the function that determines the final share amount, z(∆y):

z_1 = 1/µ * ( µ/c * (k - (y0 + ∆y)^(1-t))^(1/(1-t)) + φ_c * (1-p) * (1-φ_g) * ∆y/c
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                   yieldspace term                             fee term

Increasing or decreasing solvency

The shares used to mint bonds when opening a short come from the liquidity providers. This is equivalent to saying it comes from the pool share reserves. However, the shares used to pay curve fee comes from the trader. It is possible that the fee term will dominate the yieldspace term for a reasonable set of constants. In this case, the trader will pay more shares in fees than the LP will pay to mint bonds, and thus the net delta share reserves will be positive. This is increasingly likely as the price of bonds goes down and the fee constants go up.

In this case, solvency becomes constrained by the yieldspace dynamics themselves. The fee term shifts k, and it is bounded wrt reserve levels.

Monotonicity

The yieldspace term is exponential in ∆y and the fee term is linear in ∆y. This means that the dominating term will switch as we increase ∆y from zero to infinity.

If 0<t<1, then (y0+∆y)^(1-t) should strictly increase. Therefore, k-(y0+∆y)^(1-t) will strictly decrease. The rest of the yieldspace terms will not change this, and so the yieldspace function will decrease.

The linear term has a bunch of constants times ∆y, and thus will strictly increase with increasing ∆y.

Because z_1 is the sum of one strictly decreasing and one strictly increasing function, the net result can be non-monotonic. Whether or not it is monotonic in practice will depend on the range of values considered for all variables.

Empirical evidence:

Increasing or decreasing solvency

Here is a Rust pseudocode test to show that this is true:

let hyperdrive_state = gen_random_state();
let trade_amount_in_bonds = gen_random_valid_trade_amount();

// Define solvency for a few increasing trade amounts.
// Solvency is S(f(∆y)) and is expected to decrease with increasing ∆y.
let s_dy = hyperdrive_state.solvency_after_short(trade_amount_in_bonds);
let s_dy_plus_i = hyperdrive_state.solvency_after_short(trade_amount_in_bonds + fixed!(1e14));
let s_dy_plus_big_i = hyperdrive_state.solvency_after_short(trade_amount_in_bonds + fixed!(10e14));
let s_dy_plus_bigger_i = hyperdrive_state.solvency_after_short(trade_amount_in_bonds + fixed!(100e14));
let s_dy_plus_biggest_i = hyperdrive_state.solvency_after_short(trade_amount_in_bonds + fixed!(1_000e14));

assert!(s_dy > s_dy_plus_i);
assert!(s_dy_plus_i > s_dy_plus_big_i);
assert!(s_dy_plus_big_i > s_dy_plus_bigger_i);
assert!(s_dy_plus_bigger_i > s_dy_plus_biggest_i);

This test fails for some random states. Here are printouts for an example failure mode:

s_dy                = 35012909.221354955208362398
s_dy_plus_i         = 35012909.221355086287280709
s_dy_plus_big_i     = 35012909.221356265951869039
s_dy_plus_bigger_i  = 35012909.221368061814284225
s_dy_plus_biggest_i = 35012909.221486020540448835

The solvency goes up each time. The test itself passed 1,297 times before failing in this example. This test failed every time out of 10-20 of attempts with a 10k fuzz loop. I didn't compute the actual fail rate.

Monotonicity

The PR includes a new test to hunt for monotonicity. This test consistently passes, which indicates to me that while we cannot guarantee monotonicity, we can assume it to be true in practice with the values we consider in testing.