Custom tokens should standardize how balances are stored

[chowbao]

Proposal

Any custom token balance should be stored the same as SACs

Changes

pub struct BalanceValue {
    pub amount: i128,
    pub authorized: bool,
    pub clawback: bool,
    // These can be extended in the value map with any custom attributes that the contract would want to track
}

Reasoning

Having a standard structure for token balances would make it easier to analyze token balance information. Instead of tracing contract events, the balance can be found for a token consistently through ledger entries instead. This reduces the overhead for point in time balance lookups by skipping the need to process contract events all together. This would also make processing custom tokens and SACs largely similar making a token/balance tracking pipeline much easier to implement

[dmkozh]

I believe this SEP goes against the smart contract design philosophy. Data storage is considered an implementation detail. Public interface provides the necessary getters in an abstract fashion. In order to get the respective field the getter has to be called. While this philosophy is not set in stone, it is what we've used and promoted thus far.

I realize that unified data layout might have some benefits, but I'm not sure if standardizing it is really feasible. It seems to restrict the token implementation details significantly.

[dmkozh]

If it isn't fair to ask a contract to store a balance in a simple way

The simplest way is to just store balance in an entry with key Balance(Address) and i128 value. The requirement to wrap the value into struct is redundant for the tokens that don't deal with authorization and clawback (which is a reasonable expectation for the simpler tokens, such as liquidity pool tokens).

[chowbao]

I'd be open to Balance(Address) and i128 as well
Just as an example what would contracts like blend do (if they were a custom token)?

They currently store like

# key
[
  "Positions"sym,
  [GDTQ…G6TK](https://stellar.expert/explorer/public/account/GDTQB7BO5Y7BG2QQPTEFRATHUPPSDL3XEMFZ7HVDPYX6MQUK63DZG6TK)
]

# value
{
  "collateral"sym: {
    1u32: 92714i128
  },
  "liabilities"sym: {},
  "supply"sym: {}
}

[dmkozh]

I'd be open to Balance(Address) and i128 as well

My point is that format is not compatible with SAC. That's why I'm not feeling like there is really a one-size-fits-all standard possible.

Just as an example what would contracts like blend do (if they were a custom token)?

I'm not sure I understand your question. In general, if the contract has already been deployed on the pubnet and is serving the users, performing any sort of data format migration is a pretty complex and tedious task to perform.

[chowbao]

I think data migration aside, I want to know if there is any merit in creating some kind of standard.

Like I mentioned, blend is just a hypothetical example. The custom tokens and SACs data entry do have an address: balance form. There would be no change to any existing smart contract tokens. I'm proposing defining that standard before someone comes along and changes it. Otherwise how would you propose the tracking of this data? Only through contract events?

[dmkozh]

Otherwise how would you propose the tracking of this data? Only through contract events?

Yes, events are designed for an abstract way of tracking balances. Current balance can be retrieved with a balance() getter that's in the token standard as well.

[leighmcculloch]

Having a standard structure for token balances would make it easier to analyze token balance information.

I don't think this is a sufficient reason to constrain innovation and to exclude contract tokens that don't adhere to a standard data storage mechanism. In other ecosystems there is sufficient evidence that data ingestion products are able to navigate a variety of token implementations without relying only on the storage data. Making an external integration slightly easier at the expense of contract dev flexibility (which will impair innovation) seems like not a great trade off.

Is the lack of a standard storage format currently inhibiting progress in the ecosystem, or products built on Stellar?

Does it solve problems that cannot already be solved by the standard interop points that already exist: functions (internally) and events (externally)?

[chowbao]

Is the lack of a standard storage format currently inhibiting progress in the ecosystem, or products built on Stellar?

No

Does it solve problems that cannot already be solved by the standard interop points that already exist: functions (internally) and events (externally)?

No

I'm fine with dropping this proposal but I do worry that some day it might be useful. As it stands today no one actually deviates from the SAC like balance data entry. That might not be true in the future. If it does change in the future then we might be more limited from a data exploration perspective but ultimately not detrimental as of our current use cases. Effectively we are trading the "maybe more flexible for contracts" for "will be difficult to standardize later"

[leighmcculloch]

As it stands today no one actually deviates from the SAC like balance data entry...If it does change in the future then we might be more limited from a data exploration perspective

While Stellar does have a richly typed storage system that it is easily queryable, and while contracts are creating the same or similar data entries today out of happenstance, storage was not intended to be a replacement for events. Using storage instead of events could unnecessarily exclude participants or discourage innovation. And so, data tooling should probably avoid creating a dependence on particular storage structures, even if certain uniformity exists today.

I can understand storage could be seen as an optimisation for certain tasks, so I guess, is that necessary? How do systems that use this optimisation avoid discouraging innovation? How do data exploration systems create better access to events such that events are so richly supported that storage holds little over it.

[chowbao]

I can understand storage could be seen as an optimisation for certain tasks, so I guess, is that necessary?

At this point no

How do systems that use this optimisation avoid discouraging innovation?

I guess because I don't see what innovation there will be I don't see how saying "you should hold balance in a data entry and call it 'balance'" stops innovation

How do data exploration systems create better access to events such that events are so richly supported that storage holds little over it

I do agree that all this information will be available through events (or derived from events). At the end of the day analytics or exploration systems can create a mart/agg/final of the data that (black box output is the same). To me it is nice to have the direct "state" of the balance from the data ledger entry for that point in time. This makes validation easier because now you have calculated balance from contract events as well as the contract's own tracking and current balance that is the accepted and final balance for that contract.

I understand that the balance() function would retrieve the balance but realistically systems like Hubble would never invoke host functions

[chowbao]

I think this discussion can be closed. Because there is no required use case there is no need to add a standard. This topic can be revisited again when a use case that isn't solvable through contract events or other means does arise