SPIKE: determine current state (token balances) retrieval strategy

[aristidesstaffieri]

Wallets need a way to index token balances and potentially other contract data for custom tokens.
We recently did a spike on this for Freighter and came to the conclusion that a good solution would be to have balances be indexed and to be able to track the TTL in order to display the appropriate UI during/after a balance data entry has expired.

Should wallet backend index token balances?
Should it also index things like decimals and symbol?
Should it track the TTL for the entries that are being indexed?

[JakeUrban]

Should wallet backend index token balances?

Given that the only alternative is querying the ledger state via RPC and the entry might be archived, I think the answer to this is yes.

However, something to note is that some tokens' balances are not the sum of all credits and debits of the token. Some tokens use a rebasing approach, where a user's balance increases or decreases based on a multiple stored and updated in the contract's state. In this case, the indexer would need to add support for ingesting changes to the multiplier, which would imply changes to all balances of the token. We can decide not to support this type of token in the wallet backend, at least initially, and default to reading ledger state within freighter on a special case basis.

Should it also index things like decimals and symbol?

This is tricky because the number of decimals and the token's symbol is a value returned from calling a function, and that function's implementation could change. Maybe we can account for this by monitoring for token contract's wasm being updated and calling these functions to fetch the potentially updated values.

Alternatives are assuming the number of decimals and the symbol is static, or working with the ecosystem to have these attributes in the contract's meta as static values.

Should it track the TTL for the entries that are being indexed?

I don't see why the indexer would need to track TTLs unless we wanted to create some kind of notification service that notified users that relevant entires are expiring soon. I'm not sure if its worth doing that vs. just letting them restore entries as needed.

[aristidesstaffieri]

However, something to note is that some tokens' balances are not the sum of all credits and debits of the token. Some tokens use a rebasing approach, where a user's balance increases or decreases based on a multiple stored and updated in the contract's state. In this case, the indexer would need to add support for ingesting changes to the multiplier, which would imply changes to all balances of the token. We can decide not to support this type of token in the wallet backend, at least initially, and default to reading ledger state within freighter on a special case basis.

It seems like this won't be the only variant of a token that we could decide to support, but I think even tokens that behave like your typical SEP-41 asset could have very different storage layouts for users balances. How would we approach indexing the data generically?

This is tricky because the number of decimals and the token's symbol is a value returned from calling a function, and that function's implementation could change. Maybe we can account for this by monitoring for token contract's wasm being updated and calling these functions to fetch the potentially updated values.

Could valid tokens implement a decimals or asymbol that changes without a wasm upgrade? This seems possible but maybe so edge case that we would not want to support it or care about when it happens.

I don't see why the indexer would need to track TTLs unless we wanted to create some kind of notification service that notified users that relevant entires are expiring soon. I'm not sure if its worth doing that vs. just letting them restore entries as needed.

The only reason is to be able to show a UI state when an entry related to an asset has been archived, but yeah it sounds like this isn't a huge win for the work it would take.

[JakeUrban]

How would we approach indexing the data generically?

Generally speaking off chain applications are expected to index a contract's state using the events the contract emits. SEP-41 defines when tokens should emit events and what their format should be. In fact, maybe the rebasing token could still emit transfer events when the multiplier is updated, which would remove the need for us to treat it as a special case.

Could valid tokens implement a decimals or a symbol that changes without a wasm upgrade?

Yes, the function could be implemented to return different values based on an arbitrary set of variables, its up to them. Thats partly why I think decimals and symbol should have been required meta attributes rather than functions.

The only reason is to be able to show a UI state when an entry related to an asset has been archived, but yeah it sounds like this isn't a huge win for the work it would take.

Yea I view this sort of functionality as an optimization, not a requirement. The default approach should be supporting restoration when necessary, and we can improve the experience opportunistically from there.

[JakeUrban]

I just spoke with the core team, and they said that in Protocol 23, transactions that require archived entires will automatically restore them prior to executing the transaction.

That means we'd be able to simulate calls to a token contract's balance() function and get the result even if the balance is archived.

Given that, we may not need to index token balances, or any ledger state for that matter. The wallet backend could index state transitions (i.e. token transfers), but when Freighter needs a user's current balance, it could use RPC for this.

cc @gouthamp-stellar

[gouthamp-stellar]

We can implement this feature in the wallet backend and have freighter get an account's token balances from the wallet backend instead of having to call rpc. This will have the added advantage that getting a custom token's balance will not require an expensive call to rpc, and it will be a nice feature for other wallets to have as well

A wallet (like Freighter) could tell the wallet backend which tokens it wants to track balances for. The waller backend could then fetch the current balances of these tokens via the rpc calls getLedgerEntries and calling custom tokens' balance() functions. These balances can then be updated every time the state of an account changes. This way, a wallet could query the wallet backend directly to get account balances. Note that Freighter will only have to tell the wallet backend which tokens to track for an account once, instead of having to do it every time a user re-installs Freighter

We may have to periodically call rpc to make sure that the balances in the wallet backend for an account are the exact same as the ones rpc returns.

I think the next step here is to write an RFC for this feature, which we could get to work on after the state change indexer is implemented. I will create a ticker in the backlog for this