-
-
Notifications
You must be signed in to change notification settings - Fork 147
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Warning if fitted sklearn model being used #989
Conversation
) | ||
except NotFittedError: | ||
# model is not fitted, as is required | ||
pass |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought in the Python call we discussed that perhaps we would check this at the first call to run_model_on_task
?
In either case I would extract this to a separate method _raise_warning_if_fitted
to make sure the functions don't get too big (they already are).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alright, will make it into a function and push.
As for its placement, I reconsidered it given that irrespective of what is called, run_model_on_task
or run_flow_on_task
, this function is what the call is reduced to. Hence went ahead with this placement for this snippet of code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
run_model_on_task
actually calls run_flow_on_task
. That said, then we would need to add a function to the extension interface that will indicate if a model is already fit, otherwise we can't check it in a general way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mfeurer do you think this is something we should want? or do we just leave it to the extension devs to implement a warning if they see it fit?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the idea of having this function as callback that can be implemented by the extension deves. And yes, I expected this function to be called from the run_model_on_task function.
Co-authored-by: PGijsbers <p.gijsbers@tue.nl>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR looks good now, but you accidentally committed a .orig file. Could you please remove that again?
Done |
Hey, it's a bit hard to see this but among all the failures it says:
Could you please have a look? |
@all-contributors please add @Neeratyoy for code |
I've put up a pull request to add @Neeratyoy! 🎉 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, if the failing unit test that @mfeurer pointed out is fixed.
@all-contributors please add @Neeratyoy for code (sorry for the spam) |
I've put up a pull request to add @Neeratyoy! 🎉 |
Given the error message, it might be that the older versions of sklearn had I updated the design of the check to be agnostic to the sklearn versions and also to the kind of model passed. |
Codecov Report
@@ Coverage Diff @@
## develop #989 +/- ##
===========================================
- Coverage 87.91% 87.86% -0.06%
===========================================
Files 36 36
Lines 4551 4565 +14
===========================================
+ Hits 4001 4011 +10
- Misses 550 554 +4
Continue to review full report at Codecov.
|
As far as I can tell these failures are a combination of timeouts and server issues, am I overlooking something? |
Yes, except for:
|
Yes this is what I referred to with timeouts (not server timeouts), my bad for leaving it ambiguous.
Will do. |
Reference Issue
Addresses #968.
What does this PR implement/fix? Explain your changes.
Prints a warning if the user passes an already fitted sklearn model to
run_model_on_task
.How should this PR be tested?