Simplify multiple scripts, logs in nodejsbenchmarking infrastructure

[uttampawar]

As I was working to add node-dc-eis to the nodejsbenchmarking, I realized the unnecessary complexity we have due to multiple scripts scatter in each benchmark. Frankly I think it's not sustainable as we add more more tests/benchmarks. I can contribute to this task if we as a group think it's important.

Here is a list of scripts in use today,
fp.sh (function to get memory footprint, rss, heap, etc)
cpuParse.sh (cpu statistics per process, node, any client and mongod)
mongodb.sh (start mongodb instance)
kill_node_linux (stops all node processes), and some 'awk' scripts.
In addition there are scripts per workload/benchmark (run_acmeair.sh, run-dc-eis.sh, etc.)

On top of that, each of these script generate their own log output in different directory, processes it for whatever output it needs.

I'm proposing to put these scripts in one place, get sourced as needed and generate output in one log file.

Here is my initial assessment of current execution flow,

1. Preparation - setting up workspace
2. Build a node version
3. If test needs database,
   • start a database (eg. mongodb)
4. Start benchmark
5. Start a client
6. Wait for fixed timeout
7. Stop everything after waiting period
8. Post results
9. Archive results
10. Cleanup workspace

Data collection today,

1. Memory footprint before the run (fp.sh)
2. CPU stats for each process, viz. node, client (if needed), database (if needed) during the run (cpuParse.sh)
3. Memory footprint after the run (fp.sh)
4. Throughput (ops/sec)
5. Latency

[davisjam]

@mhdawson Can we add this to the agenda for #245?

[mhdawson]

@davisjam done