Matching simulation comparison tool #6287
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changed?
Adding a new tool which runs all the matching simulation configs, parses their result and generates a CSV file.
Example: https://gist.github.com/taylanisikdemir/b039e6156289e4ccf76121c33cb845ba
Note: Each simulation run spits a txt file containing oneline and multiline measurements that are generated from the json formatted event logs using jq/awk etc. Parsing those outputs required some regex magic. Alternative is to load all the event logs into a sqlite table and then run queries on top instead of parsing outputs of jq in this tool. I will look into this option soon.
Why?
Without such tool, it's hard to compare simulation results.
How did you test it?
Run all the scenarios and compare:
If you have already run some scenarios before and made changes in the output/comparison then run in Compare mode