You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Around 80% of the total memory usage can be attributed to symbolization-related things:
metastore: stores all the stacktraces, and tracks the ones that haven't been symbolized yet (using badgerdb)
symbolizer: the different symbolizer implementations (DWARF, gopclntab, symtab) all load entire sections into memory, as well as caches parts of the processing
The metastore additionally causes headaches as it can't just be rebuilt from scratch whenever it's wiped.
I think both of these can be significantly improved by changing our symbolization strategy a bit:
store stack traces in the columnar database (unsymbolized if they are not symbolized at write time)
perform read-time symbolization: instead of performing symbolization for absolutely everything in time asynchronously, only symbolize and cache those things that are actually being queried when they're being queried
improve symbolizer implementations to not require loading the whole section into memory (this will require forking the stdlib DWARF, gopclntab and symtab implementations but shouldn't be too difficult since just all accesses straight to a byte slice just need to be replaced with an io.ReaderAt)
This should remove most of the excessive memory usage.
The text was updated successfully, but these errors were encountered:
Parca's memory usage is very unpredictable, mainly because of symbolization.
A memory profile from https://demo.parca.dev/: https://pprof.me/4e9382bfefe0002d4b786fe7524eed4e
Around 80% of the total memory usage can be attributed to symbolization-related things:
The metastore additionally causes headaches as it can't just be rebuilt from scratch whenever it's wiped.
I think both of these can be significantly improved by changing our symbolization strategy a bit:
io.ReaderAt
)This should remove most of the excessive memory usage.
The text was updated successfully, but these errors were encountered: