Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update summary metrics #134

Closed
katxiao opened this issue Jan 26, 2022 · 0 comments
Closed

Update summary metrics #134

katxiao opened this issue Jan 26, 2022 · 0 comments
Assignees
Labels
feature request Request for a new feature
Milestone

Comments

@katxiao
Copy link
Contributor

katxiao commented Jan 26, 2022

Problem Description

Current, the summary metrics from the sdgym summary command are % coverage, avg time, avg score. We can update them to be more informative about the synthesizers' comparative performance and accuracy.

Expected behavior

Performance (time) metrics: One idea is to use a leaderboard approach. We can display how many times a given synthesizer had the best relative modeling time.

Quality metrics: TBD

@katxiao katxiao added feature request Request for a new feature under discussion Issue is currently being discussed labels Jan 26, 2022
@katxiao katxiao self-assigned this Jan 26, 2022
@katxiao katxiao removed the under discussion Issue is currently being discussed label Jan 26, 2022
@katxiao katxiao added this to the 0.6.0 milestone Feb 1, 2023
@katxiao katxiao closed this as completed Feb 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Request for a new feature
Projects
None yet
Development

No branches or pull requests

1 participant