Optimize cache persistence

[kolesnikovae]

The PR includes number of improvements:

1. LFU cache moved from pyroscope-io/lfu-go (to be archived).
2. Fixed write back procedure: before, modified items might not be written to disk, therefore in case of a crash we lose data.
3. Optimised serialisation/deserialisation for dictionaries and trees.
4. Now in-memory trees are truncated at serialisation.
5. Introduced TTL-based eviction for cached items (2m for all caches).
6. Disabled memory-pressure-based evictions for all but trees.
7. Fixed race in cache.GetOrCreate.
8. Refactored cache initialisation.
9. Breaking changes:
   • Removed pyroscope_storage_cache_hits_total - the way cache lookup is implemented this metric would require quite a dirty code branch. Meanwhile the metric value can be derived from pyroscope_storage_cache_misses_total.
   • pyroscope_storage_cache_persisted_total counter replaced with histograms:
     • pyroscope_storage_cache_disk_writes
     • pyroscope_storage_cache_disk_reads

Performance implications

One of the main goal was to decrease pyroscope server resource consumption:

Notably that GC is not a problem anymore:

[codecov]

Codecov Report

Merging #385 (800cf90) into main (0403cee) will decrease coverage by 0.02%.
The diff coverage is 70.31%.

@@            Coverage Diff             @@
##             main     #385      +/-   ##
==========================================
- Coverage   49.52%   49.51%   -0.01%     
==========================================
  Files         121      124       +3     
  Lines        5499     5608     +109     
==========================================
+ Hits         2723     2776      +53     
- Misses       2508     2568      +60     
+ Partials      268      264       -4     

Impacted Files	Coverage Δ
pkg/storage/periodic.go	56.00% <ø> (-19.47%)	⬇️
pkg/storage/tree/serialize.go	56.38% <16.67%> (-13.70%)	⬇️
pkg/storage/dict/dict.go	75.76% <60.00%> (-9.42%)	⬇️
pkg/storage/cache/lfu/lfu.go	60.40% <60.40%> (ø)
pkg/storage/cache/cache.go	67.61% <64.87%> (+10.75%)	⬆️
pkg/storage/codec.go	75.00% <75.00%> (ø)
pkg/storage/dict/trie.go	75.44% <81.82%> (+0.44%)	⬆️
pkg/storage/storage.go	57.68% <85.72%> (-5.90%)	⬇️
pkg/storage/metrics.go	100.00% <100.00%> (ø)
pkg/storage/tree/tree.go	83.34% <100.00%> (+0.36%)	⬆️
... and 4 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0403cee...800cf90. Read the comment docs.

[eh-am]

whoa, that memory drop is impressive!

(placeholder for after I read the code)

Re: 9, can you update the dashboard in monitoring/lib and make it?

[kolesnikovae]

whoa, that memory drop is impressive!

(placeholder for after I read the code)

Re: 9, can you update the dashboard in monitoring/lib and make it?

Sure thing, done

[petethepig]

Looks good, merging