-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature]: Replace ABCIMessageLogs .String() method with faster JSON library #18906
Comments
As long as it's compatible with stdlib JSON, I think it should be OK and state compatible. In such a case, we can consider https://github.com/sugawarayuuta/sonnet |
there are somethings that will change, mainly because amino is protobuf style json which is not normal json. the changes are minimal and probably better in terms of ux since people dont use this with protobuf but normal json decoders/encoders |
ABCI Message Logs is a slice of ABCIMessage , which itself is a struct of just various nested strings (and one uint): https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/base/abci/v1beta1/abci.proto#L53-L79 A bit surprised theres even an incompatability! |
For large txs during block sync with big events, were actually seeing this take 15% of the state machine execution time! (Defined as deliverTx/Beginblock/ENdblock time measured in a blocksync pprof of |
I'm glad you're really benchmarking various aspects of the module codebase that have largely remained untouched for a long time. Do you have a recommendation for an alternative JSON library? |
The standard library for now would make this much faster! In the future, sometime compiled would be great, as this structure is super simple -- but no need to complicate it for now! Glad the speed investigations are appreciated! Was feeling really committed to improving state machine sync speeds! And then we keep facing issues on Osmosis that are related, so its becoming more and more of a priority haha. The hot loop of looking at CPU profiles and making stuff go away is really nice though :) Tons of wins are actually already in SDK main / the new store work as well, which is super exciting! |
Cool! Ok, so I think let's just convert this to using stdlib JSON then. |
Summary
ABCIMessageLogs.String() uses legacy amino for JSON marshalling. As far as I can tell, this usage is fully compatible with the stdlib.
We see this method is taking a surprising amount of time on Osmosis. Its taking about 2% of the state execution time during block sync.
cosmos-sdk/types/result.go
Lines 45 to 56 in 77b2366
Can we replace this method with a faster, standard JSON marshalling library? I am making an issue instead of a PR just to sanity check this is integrator compatible. I don't see a way it wouldn't be compatible, but this does seem bad to break
Problem Definition
This is taking too much CPU time
Proposed Feature
Replace amino JSON marshalling here with a faster JSON marshalling library.
The text was updated successfully, but these errors were encountered: