coverage testing #38
coverage testing #38
Conversation
|
This is nice! Also, can we add one of those cool 'build passing' indicators? |
|
Yup! Did you find the coverage reports? There's cobertura and coverage.py (thanks @galbramc!) |
|
Note that there seems to be a duplicate section in https://acdl.mit.edu/csi/job/robust_PullRequest/1/buildnode=macys,optimizer=mosek/cobertura/robust_testing/t_primitives_py/ |
|
You can get the build passing indicator links here: https://acdl.mit.edu/csi/job/gpkit_ResearchModel_robust_Push/badge/ |
|
Is this not considered a research model? Why do we have both robust_PullRequest job and gpkit_ResearchModel_robust_Push? |
|
I think this will be transitioning from a research model to having pylint, coverage tests etc., one step at a time. |
|
Ok so you want the same type of testing as you have for gpkit? |
|
I think so, but it's @1ozturkbe's show; Berk whaddya want? |
|
What is the purpose of this repo? Is it a different category than other research models? Why don't we have lint and coverage information for ResearchModels? |
|
Because it doesn't work with the dynamic-import stuff they run tests with :-/ |
|
which, writing a python file to run all your tests isn't hard, but most of those repos are on life support as is. |
|
It's going to be a process, but yes, I'd like this repo to have the same type of testing as gpkit. |
|
test this please |
|
Hmm, things are much easer to manage if we have families of jobs so we can use the job templates to manage things. Would it be possible to unity things so so all repos run their respective tests in the same way? Can we update repos that needed and deprecate repos that are on life support? |
|
test this please |
|
@galbramc yup, I'll look into using templates |
|
@1ozturkbe looks like you're testing about half of your code right now, not too bad for the number of tests you have! Let me know if you want a copilot for this, else I'll just fix the Windows testing errors and leave it to ya.
|
|
That's a cool screenshot @bqpd! Super useful tool. |
|
@bqpd I'm actually surprised as well about the coverage! I knew it was bad, and it seems like the last few PRs helped. I do have ideas for drastically improving coverage (>80%) since some of the functionalities have are completely untested. I also want the testing to somewhat like documentation, so robust can be a little more user friendly. But this is an awesome start! |
| assert(mm.substitutions.get(key) == m.substitutions.get(key) * | ||
| (1.+mm.setting.get("gamma")*key.pr/100.)) | ||
| self.assertGreater(margin_solution['cost'], mm.nominal_cost) | ||
|
|
There was a problem hiding this comment.
I hadn't even noticed this... Definitely a merge issue, thanks @bqpd.
|
test this please |
1 similar comment
|
test this please |
|
I was only suggesting deprecating anything that is really not worth keeping. So who is going to do the TLC to get the research models updated. In particular so they can be tested with coverage and pylint? |
|
test this please |
1 similar comment
|
test this please |
|
@galbramc I think they're fine as they are, but it'll be me doing that if it gets done :p |
|
test this please |
|
test this please |
|
I don't know what changed, or what this error means: |
|
does it occur on any other tests?
…On Tue, Sep 3, 2019, 18:19 Marshall Galbraith ***@***.***> wrote:
I don't know what changed, or what this error means:
ImportError: Could not load MOSEK library: ImportError('No module named lib',)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#38?email_source=notifications&email_token=AALKAGARZDRHNJBL634SYV3QH3NQPA5CNFSM4IS7WZ7KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5ZW7OQ#issuecomment-527658938>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AALKAGHUR3YETRXAEQ2X2C3QH3NQPANCNFSM4IS7WZ7A>
.
|
|
gpkit_Push is failing the same way. I don't know what changed... I didn't do anything |
|
I don't think an out of date license file would manifest that way, but maybe worth double-checking? |
|
The license is checked above and it checks our cleanly. |
|
it's not finding mosek during installation... |
|
oh it's because it's not finding ctypesgencore |
|
ah, it's a new version of ctypesgen that's being pulled down by pip |
|
Ahh, and since gpkit_install does not expliclty test for mosek, it didn't fail when it found the new ctypesgen. One purpose of gkit_install is to sort out when dependencies break gpkit. Looking at the log I see: Maybe gpkit_install should also explicitly run cvxopt and mosek? |
|
naw I think this is working-as-intended; because the solvers are explicitly
specified it was trying to solve with mosek despite not finding it
… |
|
Yes, I think gpkit_install should have failed as well because ctypesgen didn't work. Do you not agree? This means you have a false positive saying you think gpkit from pip is working when in fact it does not work with mosek and the latest ctypesgen right now. |
|
oh, yeah that's true. The problem right now is that anaconda (used on the windows machines) seems to come with the new version of ctypesgen, so it can't be uninstalled...seeing if I can fix that with |
|
@galbramc that didn't work, so it now works on everything but windows... https://acdl.mit.edu/csi/job/gpkit_PullRequest_Unit/2820/ anyway I'll try to get it working with ctypesgen 1.0.0 |
No description provided.