[EPIC] Lotus Profiling

[jennijuju]

Lotus profiling ideas proposed by @arajasek

Tasks

Beta Give feedback

1. Message Execution Profiling #10892
   11 of 11
   +0
   
2. Mining profiling #10891
   0 of 7
   +0
3. Chain datastore profiling #10890
   +0
4. RPC latency metrics #10960
   +0
5. SQLite DBs profiling #10959
   +0

[jennijuju]

@arajasek to make sure we have the maximum impact on this effort - I think our end goal would be something like https://github.com/filecoin-project/boost/tree/main/docker/monitoring. where we launch a monitoring board easily with any node we have, and get stats on how things are changing over time, or when we have a new release and so on. wdyt?

[arajasek]

Step: The Lotus team should have a single node syncing mainnet, running splitstore, that we have to keep alive, collect metrics from, and deploy new features to

• Start with a "discard" node that only has recent state
• Add a second, archival node later

Step: Add documentation for how you can view Lotus's existing metrics on a Grafana dashboard

Step: Define metrics that we want to be gathering, add tracing for new ones as needed

Step: It's easy to configure Lotus to dump these metrics, configuration (env vars), system details (HW), Lotus version, etc. so that we can troubleshoot user issues

• Leave it on by default, and opt-outable

[jennijuju]

some ideas https://geth.ethereum.org/docs/monitoring/dashboards

[mur-me]

Speaking about dashboards, we already have dashboards built for our RPC use-cases.
Additionally lotus team created few new dashboards to monitor consequences of the Pi Day.

Probably, this dashboards could be a good starting point or contribute in the common goal.

Link: https://grafana.com/orgs/openworklabbot/dashboards

Fill free to ping Glif team for details