adding network osu metric python parser (#20)

* adding network osu metric python parser

this will parse the data into a structured format, and
next I need to add plotting functions for it.

Signed-off-by: vsoch <vsoch@users.noreply.github.com>

Makefile

@@ -262,6 +262,11 @@ test-deploy: manifests kustomize
 	cd config/manager && $(KUSTOMIZE) edit set image controller=${DEVIMG}
 	$(KUSTOMIZE) build config/default > examples/dist/metrics-operator-dev.yaml
 
+.PHONY: test-deploy-recreate
+test-deploy-recreate: test-deploy
+	kubectl delete -f examples/dist/metrics-operator-dev.yaml || true
+	kubectl apply -f examples/dist/metrics-operator-dev.yaml
+
 # Build a catalog image by adding bundle images to an empty catalog using the operator package manager tool, 'opm'.
 # This recipe invokes 'opm' in 'semver' bundle add mode. For more information on add modes, see:
 # https://github.com/operator-framework/community-operators/blob/7f1438c/docs/packaging-operator.md#updating-your-existing-operator

README.md

@@ -8,7 +8,7 @@ Note that I haven't started the operator yet because I'm [testing ideas for the
 To learn more:
 
 - ⭐️ [Documentation](https://converged-computing.github.io/metrics-operator/) ⭐️
-- 🐯️ [Python module](https://pypi.org/project/metricsoperator/0.0.0/) 🐯️
+- 🐯️ [Python Module](https://pypi.org/project/metricsoperator/) 🐯️
 
 ## Dinosaur TODO
 
@@ -18,8 +18,11 @@ To learn more:
 - Python function to save entire spec to yaml (for MetricSet and JobSet)?
 - Metrics parsers to do (need to add separators, formatting, python parser):
   - perf-sysstat
-  - netmark / osu-benchmark
-
+  - netmark
+- Plotting examples needed for
+  - perf-sysstat
+  - netmark
+  - io-sysstat
 
 ## License
 

docs/_static/data/metrics.json

@@ -1,4 +1,11 @@
 [
+ {
+  "name": "network-osu-benchmark",
+  "description": "point to point MPI benchmarks",
+  "type": "standalone",
+  "image": "ghcr.io/converged-computing/metric-osu-benchmark:latest",
+  "url": "https://mvapich.cse.ohio-state.edu/benchmarks/"
+ },
  {
   "name": "perf-sysstat",
   "description": "statistics for Linux tasks (processes) : I/O, CPU, memory, etc.",
@@ -19,12 +26,5 @@
   "type": "standalone",
   "image": "vanessa/netmark:latest",
   "url": ""
- },
- {
-  "name": "network-osu-benchmark",
-  "description": "point to point MPI benchmarks",
-  "type": "standalone",
-  "image": "ghcr.io/converged-computing/metric-osu-benchmark:latest",
-  "url": "https://mvapich.cse.ohio-state.edu/benchmarks/"
  }
 ]

docs/getting_started/metrics.md

@@ -78,7 +78,7 @@ The following examples are provided alongside the operator. Each directory has a
  - [perf-lammps](https://github.com/converged-computing/metrics-operator/tree/main/examples/tests/perf-lammps)
  - [io-host-volume](https://github.com/converged-computing/metrics-operator/tree/main/examples/tests/io-host-volume)
  - [network-netmark](https://github.com/converged-computing/metrics-operator/tree/main/examples/tests/network-netmark) (code still private)
- - [network-osu-benchmarks](https://github.com/converged-computing/metrics-operator/tree/main/examples/tests/network-osubenchmarks) (code still private)
+ - [network-osu-benchmark](https://github.com/converged-computing/metrics-operator/tree/main/examples/tests/network-osu-benchmark)
 
 ### Apps to be Measured
 
@@ -103,8 +103,6 @@ The following tools are folded into the metrics above. Often, one tool can be bu
 
  - [ghcr.io/converged-computing/metric-sysstat](https://github.com/converged-computing/metrics-operator/pkgs/container/metric-sysstat)
 
-text.startsWith("Hello");
-
 Sysstat is stored as a general metrics analyzer, as it provides several different metric types; It generally provides utils to monitor system performance and usage, including:
 
 - *iostat* reports CPU statistics and input/output statistics for block devices and partitions.

examples/python/io-host-volume/run-metric.py

@@ -3,6 +3,7 @@
 import argparse
 import os
 import json
+import time
 from metricsoperator import MetricsOperator
 import metricsoperator.utils as utils
 
@@ -33,9 +34,12 @@ def main():
     # Create a metrics operator with our metrics.yaml
     m = MetricsOperator(metrics_yaml)
     m.create()
+
+    print('Sleeping one minute to allow container to pull...')
+    time.sleep(60)
     for output in m.watch():
         print(json.dumps(output, indent=4))
-        utils.write_json(output, os.path.join(here, "metrics.json"))
+        utils.write_json(output, args.out)
 
 if __name__ == "__main__":
     main()

examples/python/network-osu-benchmark/README.md

@@ -0,0 +1,18 @@
+# OSU Benchmarks Python
+
+This is a quick example to show running and plotting a few of the benchmarks!
+This assumes you have a running cluster with JobSet and the Metrics Operator installed!
+
+```bash
+$ python run-metric.py
+```
+
+The script will use the metricsoperator library to submit the associated yaml under [tests](../../tests)
+and then wait for the pod to complete and parse the output in the log.
+
+## Example Plots
+
+![img/OSU-MPI_Accumulate-latency-Test-v5.8.png](img/OSU-MPI_Accumulate-latency-Test-v5.8.png)
+![img/OSU-MPI_Get_accumulate-latency-Test-v5.8.png](img/OSU-MPI_Get_accumulate-latency-Test-v5.8.png)
+![img/OSU-MPI_Get-latency-Test-v5.8.png](img/OSU-MPI_Get-latency-Test-v5.8.png)
+![img/OSU-MPI_Put-Latency-Test-v5.8.png](img/OSU-MPI_Put-Latency-Test-v5.8.png)

examples/python/network-osu-benchmark/img/OSU-MPI_Accumulate-latency-Test-v5.8.csv

@@ -0,0 +1,24 @@
+,Size,Latency (us)
+0,1.0,0.56
+1,2.0,0.41
+2,4.0,0.36
+3,8.0,0.3
+4,16.0,0.27
+5,32.0,0.25
+6,64.0,0.26
+7,128.0,0.29
+8,256.0,0.39
+9,512.0,0.57
+10,1024.0,0.78
+11,2048.0,1.49
+12,4096.0,2.36
+13,8192.0,4.68
+14,16384.0,9.47
+15,32768.0,18.38
+16,65536.0,35.33
+17,131072.0,68.67
+18,262144.0,138.39
+19,524288.0,271.58
+20,1048576.0,542.47
+21,2097152.0,1085.58
+22,4194304.0,2288.26

examples/python/network-osu-benchmark/img/OSU-MPI_Fetch_and_op-latency-Test-v5.8.csv

@@ -0,0 +1,2 @@
+,Size,Latency (us)
+0,8.0,0.5

examples/python/network-osu-benchmark/img/OSU-MPI_Get-latency-Test-v5.8.csv

@@ -0,0 +1,24 @@
+,Size,Latency (us)
+0,1.0,0.57
+1,2.0,0.48
+2,4.0,0.34
+3,8.0,0.28
+4,16.0,0.19
+5,32.0,0.18
+6,64.0,0.15
+7,128.0,0.15
+8,256.0,0.15
+9,512.0,0.13
+10,1024.0,0.13
+11,2048.0,0.16
+12,4096.0,0.16
+13,8192.0,0.23
+14,16384.0,0.5
+15,32768.0,0.91
+16,65536.0,1.85
+17,131072.0,3.75
+18,262144.0,9.63
+19,524288.0,20.8
+20,1048576.0,45.42
+21,2097152.0,76.36
+22,4194304.0,303.25

examples/python/network-osu-benchmark/img/OSU-MPI_Get_accumulate-latency-Test-v5.8.csv

@@ -0,0 +1,24 @@
+,Size,Latency (us)
+0,1.0,2.96
+1,2.0,1.89
+2,4.0,1.83
+3,8.0,1.75
+4,16.0,1.71
+5,32.0,1.68
+6,64.0,1.75
+7,128.0,1.9
+8,256.0,2.21
+9,512.0,2.65
+10,1024.0,2.99
+11,2048.0,3.66
+12,4096.0,5.6
+13,8192.0,10.8
+14,16384.0,23.46
+15,32768.0,46.46
+16,65536.0,75.88
+17,131072.0,166.75
+18,262144.0,256.08
+19,524288.0,456.7
+20,1048576.0,1026.29
+21,2097152.0,1970.85
+22,4194304.0,4172.06

examples/python/network-osu-benchmark/img/OSU-MPI_Put-Latency-Test-v5.8.csv

@@ -0,0 +1,24 @@
+,Size,Latency (us)
+0,1.0,0.1
+1,2.0,0.1
+2,4.0,0.1
+3,8.0,0.1
+4,16.0,0.11
+5,32.0,0.11
+6,64.0,0.1
+7,128.0,0.11
+8,256.0,0.11
+9,512.0,0.11
+10,1024.0,0.13
+11,2048.0,0.16
+12,4096.0,0.23
+13,8192.0,0.3
+14,16384.0,0.6
+15,32768.0,1.03
+16,65536.0,2.49
+17,131072.0,5.46
+18,262144.0,10.59
+19,524288.0,19.91
+20,1048576.0,39.43
+21,2097152.0,81.25
+22,4194304.0,282.09