From 489237cf9d90375047b3f625f15ae54d87d29404 Mon Sep 17 00:00:00 2001 From: Brian Olson Date: Mon, 29 Aug 2022 16:49:53 -0400 Subject: [PATCH] describe scripts in this dir --- test/heapwatch/README.md | 50 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 45 insertions(+), 5 deletions(-) diff --git a/test/heapwatch/README.md b/test/heapwatch/README.md index 27cb54d316..04c0568be6 100644 --- a/test/heapwatch/README.md +++ b/test/heapwatch/README.md @@ -1,14 +1,54 @@ # Heap Watch -Tools for checking if algod has memory leaks. +Collect RAM, bandwidth, and other stats over the course of a test cluster run. -Run a local private network of three nodes and two pingpongs. +Produce reports and plots from data. -Periodically sample pprof memory profiles. +## Scripts -Watch memory usage from `ps` and write to a CSV file for each algod. +* heapWatch.py + * collect data from algod + * heap profiling, /metrics, cpu profiling, block headers, goroutine profile + * capture from local algod by data dir or cluster from terraform-inventory.host + * convert profiles to svg or other reports -# Usage +* block_history.py + * Capture block headers every round from a running `algod` + +* block_history_relays.py + * Capture block headers every round from one or more running `algod` + * Talk to a set of relays found in a terraform-inventory.host file. + +* block_history_plot.py + * Plot the output of test/heapwatch/{block_history.py,block_history_relays.py} + +* client_ram_report.py + * Process heap profiles (*.heap) collected from heapWatch.py + * Create a report on `algod` RAM usage + +* plot_crr_csv.py + * Plot the output of test/heapwatch/client_ram_report.py --csv + +* metrics_delta.py + * Process /metrics data captured by heapWatch.py + * Generate text report on bandwidth in and out of relays/PN/NPN + * optionally plot txn pool fullness + +* start.sh stop.sh + * Run a local private network of three nodes and two pingpongs. + * Periodically sample pprof memory profiles. + * Watch memory usage from `ps` and write to a CSV file for each algod. + +* bwstart.sh stop.sh + * Run a local private network of 3 relays and 8 leafs + * Run 40 TPS of payment txns through it. + * Record metrics for bandwidth analysis. + +* runNodeHost.py nodeHostTarget.py + * run new ec2 host with npn and pn algod on it pointed at one relay (no DNS needed) + + +## heapWatch.py local cluster usage To start: