-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
addrmgr/addrmanager: Allow for additional test deviations #127
Conversation
Also like in my last PR#125 🥇 |
Yet another flaky test, or a flaky function, but failure is unrelated to this PR.
Lovely - so, another test that passes 99.1% of the time. Noted to look into it and see what the source may be. |
nitpick: you don't need |
* Adjust patch to match the known deviation. * Comment text & formatting now consistent.
Corrected.
I have no idea what I was thinking. I think I should define some new constants... two=len("to"), three=len("tre") ... haha. Anyway, additional testing (currently at somewhere over 4,000,000 iterations) has shown that the deviation can (rarely) be as much as 2 (and apparently only on a fast machine), so I've updated the patch as appropriate. Also, I believe that the decred guys may have recently corrected the fault in the getAddresses function - if so, I'll back-port their changes and then disallow any deviations in the test results again, as part of the future PR. If I'm mistaken, and they haven't, I'm sure I'll be able to find the race condition eventually (and correct it), given a sufficient number of instrumented test iterations. For now:
Sorry for the slow response - I wanted to make sure this passed at least those half-million iterations before sending the revised commit. |
if len(addrs) != len(expectedAddrs) { | ||
t.Fatalf("expected to find %d adddresses, found %d", | ||
len(expectedAddrs), len(addrs)) | ||
if len(addrs) >= (len(expectedAddrs) - 2) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- 1 ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additional testing (currently at somewhere over 4,000,000 iterations) has shown that the deviation can (rarely) be as much as 2 (and apparently only on a fast machine), so I've updated the patch as appropriate.
There's a small but very real (and reproducible, though only on fast machines and verified <0.2% probability) of getting three addresses instead of 5. The probability of getting four instead of five is near 20% - it's a much, MUCH lower rate, though, on Travis - the speed of the system seems to be the biggest factor here.
Anyway, as long as we get more than one address back from the generator, we can do the rest of the tests, and it's not worth aborting us otherwise. At least not without running hundreds (or hundreds of thousands) of iterations of the test with race detection enabled.
Doing a million runs without the race detector is ~12 hours, so I estimate it could be 10+ days with the race detector enabled, and, somewhat annoyingly, the race detector slows things down enough that the race happens less often, so it's going to take quite a bit to track down the actual race involved, which is why I haven't fixed the root cause yet.
But, yeah, -2 is appropriate, for the test cases here, for now.
go test -count=10000