-
-
Notifications
You must be signed in to change notification settings - Fork 192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
⚖️📄 Compare with reproduction results, if available #642
Conversation
@cthoyt any thoughts about the metric name resolution? |
I made a new utility in b6132f3 (functional, i.e. we do not need to instantiate a |
trigger ci
I think I'd go for updating to be able to handle |
trigger ci
trigger ci
Trigger CI
can you show what the output looks like now when this is run? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
needs an emoji, then merge when ready
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LSTM!
This PR compares the results obtained by the reproducibility pipeline to those present in the configuration
Update metric name resolutionPipelineResult.get_metric
cannot resolve some metric names as present in the configuration, e.g., "hits_at_k.best.10".