-
Notifications
You must be signed in to change notification settings - Fork 73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix hypothesis health checks #680
Fix hypothesis health checks #680
Conversation
Actually, since this happens in monitor_functions and those now live outside IC we could just erase the files and forget about their existance. |
What do you mean they live outside IC? |
Ah, this: https://github.com/nextic/Olivia So, the kdst functions are no longer used, while the pmaps and rwf all are living as clients instead of inside IC. |
Ah, if these functions are zombies, kill 'em all! (but keep the tests in the other repo! :)) |
Shell we then remove the files altogether? @Aretno , @gonzaponte whats the status of this? |
a9aa6e4
to
f48bf1a
Compare
@jacg, can I ask for your advice here? Hypothesis is throwing a ton of these messages when running the Travis build (this doesn't happen on my machine):
so I attempted to raise the deadline for the specific tests that were throwing the warning. However, after doing so, more and more tests keep popping up with this warning. This makes me think that there are huge fluctuations in the time it takes Travis to run a given test and therefore it doesn't make sense to set this deadline on a test-by-test basis. Do you recommend disabling the deadline when running in Travis, maybe setting a deadline on the global test-suite execution time? Do you have any other suggestions on how to address this? |
Hmm. I don't really have any words of wisdom on this. It would be helpful to understand what is going on with Travis. I would try disabling the deadlines on Travis, and see how long the tests are taking. If the tests exceed the deadlines relatively modestly, then it might just be best to (take on some technical debt, and) keep them switched off for now, thus putting off the need to understand what is causing this. If the tests turn out to be unacceptably slow, you'll probably have to dig deeper to find out why this is happening. If it gets to that, it might be worthwhile trying it locally in a Travis Docker image. |
4720a24
to
04f0821
Compare
In the end, I have realized that part of the problem was that the Travis config had hardcoded the number of processors to use (thanks to @mmkekic for her input). This value was higher than the actual number of available processors. Nonetheless, some tests were still hitting the hypothesis deadline, which will become an error in the future and therefore must be fixed. Execution times vary from machine to machine and Travis seems to be particularly slow. Thus, I have created a new hypothesis profile specifically for Travis with a slightly higher deadline. I have checked and now we don't get those warnings anymore. We still get a warning about a global timeout for hypothesis, but this feature will disappear in a future release and therefore is not a concern. As far as I am concern, this PR is ready for review. |
To see if this is causing the problem
04f0821
to
3fb5a6a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR registers a new hypothesis profile for travis-ci with extended deadline and auto number of processes. Hopefully this will resolve occasional Travis failures due to hypothesis health checks. Also once (if ever) we setup our own CI server ( #661, miguelsimon#3 ) this will not be an issue any more.
next-exp#680 [author: gonzaponte] Every now and then the travis build fails because of data generation being too slow. This is a combination of two things: `data_frames` strategy is out of our hands the large number of columns increases execution time significantly and can't/shouldn't be reduced. This PR suppresses the HealthCheck for those tests using this strategy. [reviewer: mmkekic] This PR registers a new hypothesis profile for travis-ci with extended deadline and auto number of processes. Hopefully this will resolve occasional Travis failures due to hypothesis health checks. Also once (if ever) we setup our own CI server (next-exp#661, miguelsimon#3) this will not be an issue any more.
Every now and then the travis build fails because of data generation being too slow. This is a combination of two things:
data_frames
strategy is out of our handsThis PR suppresses the HealthCheck for those tests using this strategy.