'maximum recursion depth exceeded' when running unit tests

[jcrobak]

I ran unit tests with python test/test.py, and the tests seem to get into an infinite loop:

.mkdir: cannot create directory /tmp: File exists
EException RuntimeError: 'maximum recursion depth exceeded in __subclasscheck__' in <type 'exceptions.AttributeError'> ignored
Exception AttributeError: "'HdfsAtomicWritePipe' object has no attribute '_process'" in <bound method HdfsAtomicWritePipe.__del__ of <luigi.hdfs.HdfsAtomicWritePipe object at 0x26427d0>> ignored

It output the last two messages over and over again until I control-c'd, which showed:

^CException KeyboardInterrupt in <bound method HdfsAtomicWritePipe.__del__ of <luigi.hdfs.HdfsAtomicWritePipe object at 0x26427d0>> ignored

I'm using the CDH hadoop distro (3u3) and python2.6. Are these supported?

[erikbern]

Will try to repro this, thanks!

[erikbern]

Can't repro it, but I do see why it happens - if an exception is thrown in the constructor of HdfsAtomicWritePipe before the superconstructor is invoked, then _process won't be set and when it tries to access it, it will fall down into an infinite loop invoking getattr

So in fact it seems like two separate bugs, one triggering the other. We should fix both.

[erikbern]

This was a good summary of what's going on: http://users.cs.cf.ac.uk/J.P.Giddy/python/gotcha/getattr.html

[erikbern]

The recursion should be fixed by Ia690ba3b52f5f33320e654d85bb4d519c59d5c29 - dunno if that solves the underlying issue though.