Update Rusk SDK to be compatible with event-based system

[akhamisa]

• Currently Block Analitica (BA) can’t leverage the rust SDK for calcs because the BA dashboard is event-based
• Adding event-based calcs to the SDK would give us a unified calc for trading volume, LP APY, open positions, etc
  • Dylan: I think it would be smart if BA used our code via Rust (or js or python) SDK as much as possible because (1) we test it thoroughly and are more confident it is correct and (2) we have been changing some things, which will be cumbersome to sync if they're using their own implementations. FYI - we do require that BA build a "pool state" that has info + config for any given calculation.
• Sheng: being able to build hyperdrive state from a set of events is actually something we were talking about being useful for sims. If we were able to do this, we can use the Rust SDK for calculations using events. For example, given initial pool conditions and all past events, we would be able to replay and get state for every block. Then we can use our existing SDK functions that take pool state using only events.

[dpaiton]

The requirements for this are different between BA and the FE. Primarily because BA can run a database and the FE can't. I think a solution for the FE would work for BA, but not the other way around. The FE solution is more difficult.

FE friendly solution:

One solution to this is to build a new rust API that operates over events. This requires us to

• standardize & format event data into a rust-native data structure

• implement efficient batch loading & reduce operations

  • define an interface for event loading, post-processing, etc.

• build the hyperdrive transition functions in rust to compute state from events (Implement state transitions into Rust sdk #39). this is a 1-2 week project assuming we're squeezing it in with the current work load. less if we can dedicate someone full time to it.

We should spend some more time noodling on this. It would be cool if we made it work. But building state from events is a slow process so I don't really know how one could deliver things like pnl in real time building from events on a static site. My understanding is we would require a browser-side computation to replay the events and build the current state from the pool deployment block to the current block. There's no storage capability, so we have to do this fresh every time. It seems untenable but this is my first pass so hopefully I'm wrong here.

BA solution that does not work for FE:

Another solution is BA hosts a database (db) of events that they can work with directly. I assume BA is already using a db, so they'd have to run ours instead of or in addition to theirs. I believe this (a hosted db that aggregates events and computes stats) is a significant portion of what they're offering in the first place (not to discount the frontend or intermediate calculations). The difference would be that if they use our code we can have better visibility & control of how they compute any given quantity.

• we have an existing issue (Update chainsync to work with events agent0#1453) for updating chainsync to work with events. this is required, and is a <1 week project.

• we again need the Hyperdrive transition functions

• we need to build a backfill operation that fills the db with state once we have the transition functions

• given the above, it's straight forward to write endpoints for common computations like volume and pnl

We need this for some other stuff we are doing, so we're going to be working on this anyway. I'll wait to hear input from others before weighing in on timing & priority.

Other solutions?

I'd love to hear if anyone else has additional requirements I missed or new ideas entirely!

[jalextowle]

Due to the way the share price is encoded in events, perfectly recreating the state from events doesn’t seem that feasible to me. We know that the share price in events != the share price used during smart contract execution. Assuming these unavoidable rounding errors aren’t a blocker, I agree with most of what @dpaiton said. One optimization we could make for the frontend which makes this solution make more sense IMO is to load the state by calling pool info when the frontend loads and then use events from then on.

Practically speaking, this approach would cause some problems for accuracy (aside from the inherent rounding problems) vs the current solution because the state (which includes the share price) would only be updated when events are processed. In practice, I think consumers would still want to call getPoolInfo periodically, but I could be wrong.

[jalextowle]

We could add the starting share price to the events if this feature is a high-priority for us. That said, if this is speculative and isn’t a high-priority for us or our integrators, we should think about the extra cost of emitting an extra field in every event and think about whether or not it is really worth it.