assert isinstance(lineno, int) errors when collecting tests (2.8 regression)

[The-Compiler]

When cloning https://github.com/osiell/odoorpc/ and running py.test --collect-only I get:

=================================== ERRORS ===================================
_________________ ERROR collecting odoorpc/tests/test_env.py _________________
.venv/lib/python3.4/site-packages/_pytest/runner.py:149: in __init__
    self.result = func()
.venv/lib/python3.4/site-packages/_pytest/main.py:431: in _memocollect
    return self._memoizedcall('_collected', lambda: list(self.collect()))
.venv/lib/python3.4/site-packages/_pytest/main.py:311: in _memoizedcall
    res = function()
.venv/lib/python3.4/site-packages/_pytest/main.py:431: in <lambda>
    return self._memoizedcall('_collected', lambda: list(self.collect()))
.venv/lib/python3.4/site-packages/_pytest/python.py:600: in collect
    return super(Module, self).collect()
.venv/lib/python3.4/site-packages/_pytest/python.py:460: in collect
    l.sort(key=lambda item: item.reportinfo()[:2])
.venv/lib/python3.4/site-packages/_pytest/python.py:460: in <lambda>
    l.sort(key=lambda item: item.reportinfo()[:2])
.venv/lib/python3.4/site-packages/_pytest/python.py:396: in reportinfo
    assert isinstance(lineno, int)
E   assert isinstance(<function MetaModel.__getattr__.<locals>.rpc_method at 0x7fad5c4f2840>, int)
___________ ERROR collecting odoorpc/tests/test_field_many2many.py ___________
[...]
    assert isinstance(lineno, int)
E   assert isinstance(<function MetaModel.__getattr__.<locals>.rpc_method at 0x7fad5c51af28>, int)
___________ ERROR collecting odoorpc/tests/test_field_many2one.py ____________
[...]
    assert isinstance(lineno, int)
E   assert isinstance(<function MetaModel.__getattr__.<locals>.rpc_method at 0x7fad5c3738c8>, int)
___________ ERROR collecting odoorpc/tests/test_field_one2many.py ____________
[...]
    assert isinstance(lineno, int)
E   assert isinstance(<function MetaModel.__getattr__.<locals>.rpc_method at 0x7fad5c38b268>, int)
___________ ERROR collecting odoorpc/tests/test_field_reference.py ___________
[...]
    assert isinstance(lineno, int)
E   assert isinstance(<function MetaModel.__getattr__.<locals>.rpc_method at 0x7fad5c38bea0>, int)
________________ ERROR collecting odoorpc/tests/test_login.py ________________
[...]
    assert isinstance(lineno, int)
E   assert isinstance(<function MetaModel.__getattr__.<locals>.rpc_method at 0x7fad5c385048>, int)
________________ ERROR collecting odoorpc/tests/test_model.py ________________
[...]
    assert isinstance(lineno, int)
E   assert isinstance(<function MetaModel.__getattr__.<locals>.rpc_method at 0x7fad5c385b70>, int)

This looks similar to #1035 which was fixed by @nicoddemus, and only happened for Python 2.6 - this is with Python 3.4.3.

It works fine with pytest 2.7.3, and when reverting 3497aa0 from #921 the issue goes away as well. @tomviner

Looking at the repr, it seems it gets this function instead of a line number?

[The-Compiler]

Here is what I think happens:

• The test file does from odoorpc.models import Model
• odoorpc.models.Model inherits this __getattr__ from MetaModel which returns rpc_method
• The nose check does the equivalent of bool(getattr(Model, '__test__')) which is True
• The model gets collected as a test
• Something blows up when trying to get its line number

Some ideas:

• Maybe the nose __test__ check should actually check for is True? This is not a thing I'd generally approve of, but here it seems to make sense to be cautious
• If not that, at least it should probably handle the case where __test__ is a method, because then that's definitely not what an user intended.
• Maybe the code in reportinfo can be changed so it can get lineno correctly in such a case, or handle it more gracefully if it can't? I'm not sure what that code is doing, so I'm just guessing here.

[RonnyPfannschmidt]

from the looks of it we need to make test collection even more robust against magical thoughtless objects like rpc unverified wrappers

[RonnyPfannschmidt]

for simle implementation we should limit the nose compatibility check to ONLY functions, methods and classes

[The-Compiler]

Well, that is a class.

[nicoddemus]

Maybe the nose test check should actually check for is True? This is not a thing I'd generally approve of, but here it seems to make sense to be cautious

I agree with you, while it is not something I would use myself I think it makes sense here. Anyone cares to tackle this? If nobody has the time right now, I might be able to work on this later today or tomorrow.

[The-Compiler]

I opened two PRs now 😉

#1205 attempts to make the code getting the line number more robust in such a case, and #1206 implements the strategy of checking for is True.