Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Matching simulation comparison tool #6287

Conversation

taylanisikdemir
Copy link
Member

@taylanisikdemir taylanisikdemir commented Sep 16, 2024

What changed?
Adding a new tool which runs all the matching simulation configs, parses their result and generates a CSV file.
Example: https://gist.github.com/taylanisikdemir/b039e6156289e4ccf76121c33cb845ba

Note: Each simulation run spits a txt file containing oneline and multiline measurements that are generated from the json formatted event logs using jq/awk etc. Parsing those outputs required some regex magic. Alternative is to load all the event logs into a sqlite table and then run queries on top instead of parsing outputs of jq in this tool. I will look into this option soon.

Why?
Without such tool, it's hard to compare simulation results.

How did you test it?
Run all the scenarios and compare:

go run tools/matchingsimulationcomparison/*.go --out simulation_comparison.csv

If you have already run some scenarios before and made changes in the output/comparison then run in Compare mode

go run tools/matchingsimulationcomparison/*.go --out simulation_comparison.csv \
    --ts 2024-09-12-18-16-44 \
    --mode Compare

Copy link

codecov bot commented Sep 16, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.10%. Comparing base (04add2d) to head (bb71f22).
Report is 5 commits behind head on master.

Additional details and impacted files

see 7 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 04add2d...bb71f22. Read the comment docs.

@taylanisikdemir taylanisikdemir merged commit e86b73e into cadence-workflow:master Sep 17, 2024
20 checks passed
@taylanisikdemir taylanisikdemir deleted the taylan/simulation_comparison branch September 17, 2024 02:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants