Dates test hang intermittently

[quinnj]

I've heard a few reports of this now from @tkelman and @StefanKarpinski at least. I think I may have seen this a while ago when developing Dates.jl as a package, but I don't think I've seen it for a long time. It'd be great to nip this in the bud and figure out what's going on. Does anyone have a reproducible case or can narrow it down to a file or platform?

[StefanKarpinski]

On a related note, running make test-dates doesn't seem to actually do anything – it would be nice if that make target ran the dates tests.

[quinnj]

Does that work for other tests? I didn't even know about that.

[StefanKarpinski]

It does. Probably should be better advertised.

[Keno]

dates just needs to be added here:

julia/test/Makefile

Lines 3 to 11 in d5bbc6b

	TESTS = all core keywordargs numbers strings unicode collections hashing \
	remote iobuffer arrayops reduce reducedim \
	simdloop linalg blas fft dsp sparse bitarray random \
	math functional bigint sorting statistics spawn parallel arpack file \
	git pkg resolve suitesparse complex version pollfd mpfr broadcast \
	socket floatapprox priorityqueue readdlm regex float16 combinatorics \
	sysinfo rounding ranges mod2pi euler show lineedit \
	replcompletions backtrace repl test goto llvmcall

[quinnj]

Ok, added dates to the test/Makefile and bumped it up in the testing lineup per @tkelman's suggestion since it takes a little longer. Commit ref: 9d395c7

[tkelman]

@quinnj I'm still seeing issues on win32, will have to put the printlns back and let you know if it's happening at the same place.

[tkelman]

Maybe I wasn't updated all the way, sorry, now it's a failure:

exception on 4: ERROR: test failed: typeof(len) <: Int
while loading D:\cygwin64\home\Tony\julia32\test\dates\ranges.jl, in expression starting on line 222
while loading dates.jl, in expression starting on line 12
ERROR: test failed: typeof(len) <: Int

[quinnj]

Ah, not sure how the test didn't get updated. Just pushed the updated range tests. You should hopefully only see the no method error for similar now on win32.

[StefanKarpinski]

Update: I ran the dates tests in a loop 100 times and it finished, so it didn't hang. It's still possible that the dates tests were/are interacting with other tests in a way that causes something to hang.

[kmsquire]

Can't look right now, but it might be a good idea to check for variable names which are shared with other tests (especially those tests which run on the same worker when the tests hang).

[quinnj]

I recently put all the date tests in their own module; that prevent any kind of variable name clashing, rigth?

[kmsquire]

I believe so. Has it hung for anyone since then?

[StefanKarpinski]

It did for me right after I posted that, but only for running all the tests.

[quinnj]

So it's definitely some kind of interaction with running other tests, right?

[tkelman]

I'm still seeing this at 3806506 on both my Linux machines when running all tests.

[quinnj]

Weird, I have a lubuntu 14.04 build and have run the full tests at least 10 times and have yet to see a hang.

[quinnj]

@tkelman, what are your linux setups?

[tkelman]

Ubuntu 14.04 x64, and RHEL5 x64, both Penryn - are you running tests in parallel?

[quinnj]

Yay, I can finally join the club! Just had my 1st hang on win32!

[quinnj]

So I've continued to try to work on this and have managed to see the tests hang on windows 64, intermittently (though I've yet to be able to reproduce on my linux setup which travis fails the most consistently on).

The specific test that hangs for me is types.jl#line 170, which calls into the code here. It's possibly due to some kind of method call confusion because of so many method definitions on Date? And the fact that this method has the signature Date(periods::Period...)? I'm really not sure.

I'm hoping @JeffBezanson, @Keno, or @vtjnash could perhaps take a look or suggest additional ways I could dig into this to figure out what's going on.

[tkelman]

Anyone still seeing this or should we close? Don't think I've hit it for some time.

[quinnj]

That's because I just commented out the tests that were causing the hang 😄

[tkelman]

Ah right, well in that case we should keep this open as an outstanding bug to try to fix. Can you leave a link here pointing to the relevant lines of code? (or was it just that one on line 170?) Could you ever get them to hang when called standalone, or only when running as part of the tests?

[JeffBezanson]

Worth trying to run them again and see what happens. I fixed 5-6 memory bugs in the past week.

[quinnj]

@tkelman, I think you were able to reproduce most consistently, would you mind trying the tests that hung? See the comment block here

[tkelman]

On one machine that had shown freezes things are looking promising with that block uncommented, maybe we should have a go at turning those tests back on. My other linux machine is currently out of commission, and I'll run these tests on windows in a little bit.

[tkelman]

Of course right after I post that, the 7th run in a row of make test does freeze on the dates test. So maybe it's less likely now than it was in August, but it's not gone.

[jakebolewski]

@quinnj can these tests be uncommented now?

[quinnj]

Definitely worth trying. I can probably try some testing in a little bit.

[IainNZ]

FWIW, Dates.jl (the 0.3 package) still hangs intermittently:
http://pkg.julialang.org/?pkg=Dates&ver=release -> Show version and test history

[yuyichao]

So is this the same with #7942 or is this not limited to windows ?

[IainNZ]

I've only ever witnessed this on Linux, http://pkg.julialang.org/?pkg=Dates&ver=release happens fairly often

[quinnj]

Has anybody seen this in any form or fashion recently?

[tkelman]

Didn't we comment out a set of these tests?

[quinnj]

Yep:

julia/test/dates/types.jl

Line 150 in bcb4ec7

#=y = Dates.Year(1)

[tkelman]

I doubt anybody has been testing this with those tests locally uncommented, but we could put them back on master and see what happens.

[tkelman]

Something is occasionally freezing but I don't think it's these (any more). Every time I've checked the logs of a timed-out build that froze somewhere it looked like the dates tests completed fine and that worker moved on to other things. Closing unless anyone sees this definitively in the dates tests.

[quinnj]

Sounds good to me.