-
Notifications
You must be signed in to change notification settings - Fork 114
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Richer search for experiment tracking #1039
Comments
Thanks a lot for this request, @antoinebon! This is something we've been discussing within the team and hope to implement in the near future. We'll keep you updated here with any developments as we build this out. |
Just to add to the above: this is absolutely a feature we should try to add, and was always on the roadmap, but I think it's not going to be easy. In fact it's impossible until we've worked out what should be the role of the session store. All previous discussions on this assumed that we would use the SQLite database to help query by metric, but I know that @idanov is not a fan of that idea. He believes that the session store should be purely for metadata (e.g. author, run command, etc.) and not storing anything on metrics. It would be very useful to understand how a search functionality would work in that scheme. For reference so we have it in one place, this is where we have discussed this in the past: kedro-org/kedro#1070. The original issue https://github.com/quantumblacklabs/private-kedro/issues/1192 (not publicly accessible, sorry) has some additional comments (e.g. on SQLite) which are useful for context also. I definitely recommend skimming through the whole thing if you have access to it. Some relevant quotes below (all of which assume the SQLite model). From @limdauto:
From @AntonyMilneQB:
From @limdauto:
|
Thanks a lot for this @antoinebon , we are currently doing user testing for a new Kedro Experiment Tracking feature. Would you be interested in participating? If yes, kindly provide an email address. |
ContextThis was a common feature request that came up in the experiment tracking user testing synthesis #1627 Users want to be able to filter values they are tracking for further investigation. Supporting quotes"But actually I haven't been able to find a way yet in Kedro-Viz to to filter on this value that I'm tracking”. "Maybe some kind of way to filter runs by metric values, sort of like ones where the error rate was less than the 0.5 or something like that”. |
We'll reopen this ticket if we look at this again downstream. |
Description
In Kedro-Viz experiment tracking, I would like to be able to search in my experiments based on a value of a variable that I am tracking, but it seems that the search is currently only going thru the experiment metadata
Context
In order to track how a particular dataset is generated, I generate a run hash that is a hash of all the hyper-parameters that have an impact on the data generating process, which I then track via kedro viz experiment tracking. Since my datasets grow incrementally in time, I would like to be able to filter all the runs that have a particular hash, so that I can browse the history of the runs
Checklist
The text was updated successfully, but these errors were encountered: