Summarize improvements

[aemous]

Description of changes:

• Modified the summarize script to output the details of each individual run in addition to the existing aggregate statistics. This is useful for downstream processing that may produce other statistical computations on the raw data.
• Fixed a rare bug where a process would be actively running while we check its status , then it became a zombie process right after we check its status, so the program would crash when trying to read its statistics.

Description of tests:

• Ran the following commands to benchmark a file upload with aws s3 cp and summarize the results in the new detailed JSON format:

% ./benchmark "aws s3 cp ~/file.dmg s3://test-bucket-aem278/file.dmg" --data-interval 0.1 --output-file perf1.csv
% ./benchmark "aws s3 cp ~/file.dmg s3://test-bucket-aem278/file.dmg" --data-interval 0.1 --output-file perf2.csv
% python3 ./summarize.py perf1.csv perf2.csv --output-format json

Output:

{
  "executions": [
    {
      "execution_time": 263.8085448741913,
      "average_memory": 123170974.75968993,
      "max_memory": 123600896.0,
      "average_cpu": 0.07325581395348836,
      "max_cpu": 0.2,
      "end_time": 1735586030.26786
    },
    {
      "execution_time": 198.05210328102112,
      "average_memory": 116410262.43298969,
      "max_memory": 117899264.0,
      "average_cpu": 0.09432989690721647,
      "max_cpu": 0.2,
      "end_time": 1735586230.422445
    }
  ],
  "aggregate_stats": {
    "execution_time": 230.9303240776062,
    "std_dev_execution_time": 32.87822079658508,
    "average_memory": 119790618.5963398,
    "std_dev_average_memory": 3380356.16335012,
    "max_memory": 120750080.0,
    "std_dev_max_memory": 2850816.0,
    "average_cpu": 0.08379285543035242,
    "std_dev_average_cpu": 0.010537041476864052,
    "max_cpu": 0.2,
    "std_dev_max_cpu": 0.0
  }
}

And similarly for the table output format:

+------------------------+---------------------+---------------------+-----------+----------------------+
| Metric over 2 run(s)   | Run 1               | Run 2               | Mean      | Standard Deviation   |
+========================+=====================+=====================+===========+======================+
| Total Time (seconds)   | 263.8085448741913   | 198.05210328102112  | 230.930   | 32.87822079658508    |
+------------------------+---------------------+---------------------+-----------+----------------------+
| Maximum Memory         | 117.9 MiB           | 112.4 MiB           | 115.2 MiB | 2.7 MiB              |
+------------------------+---------------------+---------------------+-----------+----------------------+
| Maximum CPU (percent)  | 0.2                 | 0.2                 | 0.2       | 0.0                  |
+------------------------+---------------------+---------------------+-----------+----------------------+
| Average Memory         | 117.5 MiB           | 111.0 MiB           | 114.2 MiB | 3.2 MiB              |
+------------------------+---------------------+---------------------+-----------+----------------------+
| Average CPU (percent)  | 0.07325581395348836 | 0.09432989690721647 | 0.1       | 0.010537041476864052 |
+------------------------+---------------------+---------------------+-----------+----------------------+

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[jonathan343]

Having this table be generated dynamically seems like it will make it unreadable in the terminal when summarizing more than a few runs. Have you tried testing this?