feat: add sync-v2 benchmark to CI

[glevco]

Motivation

This PR implements the basic infrastructure for running benchmarks in our CI through GitHub Actions and the Bencher tool, which automatically tracks changes in performance across PRs (though it is more flexible than that). Bencher is open source and provides their cloud to run the Bencher service, which we're using through a free tier plan. We can consider upgrading our plan or running our own self-hosted service if necessary, but I don't see any reasons for now. This means our benchmarking results are public on the Bencher Console.

We setup a benchmark for tracking sync-v2 performance. It works by:

1. Setting up a temporary node connected to testnet that exits after downloading 20k blocks.
2. Running a server node in the background, connected only on localhost, and using the database with the previously downloaded blocks.
3. Running a Hyperfine benchmark integrated with Bencher, which starts a new full node that syncs 20k blocks with the server node, and exits.

By doing this we isolate the network and test only the node performance itself. Of course there may be unpredictable variations in GitHub machines, but we'll only understand this impact by observing the metrics over time.

The job described above runs in two workflows, one for every push on master, which sets the baseline for our metrics, and one for each PR to master, which compares the metric value to the baseline and triggers a CI error if it's above a certain threshold. This is configured though the Bencher console directly. I configured both upper and lower thresholds of 10% (it's useful to set a lower threshold to track performance improvements, as an unpredicted drop in time may indicate new bugs).

There are improvements to be made in the future:

• Instead of using a node connected to testnet and syncing 20k blocks every time, we could have a pre-downloaded database and save it in S3, for example. Currently, this downloading step is way slower than the benchmark itself (to be clear: it does not affect the benchmark time, but it does affect the CI job duration).
• We currently only sync 20k blocks from testnet, and that doesn't include any transactions. We could test the improvement above with different database sizes and find a tradeoff to include transactions. We could even create a fake database with specific blocks/transactions ratios and benchmark them.
• We can add other benchmarks such as node initialization time, vertex (de)serialization time, API response times, etc.
• We could add this to other projects, not only hathor-core.

Acceptance Criteria

• Create a new reusable action setup-hathor-env, used by our main CI and the new benchmarking workflows.
• Add new Continuous Benchmarking base branch and Continuous Benchmarking PRs jobs.

Checklist

• If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.02%. Comparing base (dcb876c) to head (7cee0f6).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1083   +/-   ##
=======================================
  Coverage   85.01%   85.02%           
=======================================
  Files         314      314           
  Lines       23957    23957           
  Branches     3621     3621           
=======================================
+ Hits        20367    20369    +2     
+ Misses       2879     2878    -1     
+ Partials      711      710    -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Bencher

Report	Mon, August 5, 2024 at 15:41:06 UTC
Project	hathor-core
Branch	feat/benchmarking
Testbed	ubuntu-22.04

Click to view all benchmark results

Benchmark	Latency	Latency Results nanoseconds (ns) | (Δ%)	Latency Lower Boundary nanoseconds (ns) | (%)	Latency Upper Boundary nanoseconds (ns) | (%)
sync-v2 (up to 20000 blocks)	✅ (view plot)	161,709,658,426.00 (0.00%)	145,538,692,583.40 (90.00%)	177,880,624,268.60 (90.91%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

[jansegre]

Looks good to me.