3.2.0: exceptions / internal errors related to markers #2573

The-Compiler · 2017-07-14T12:27:35Z

I just tried the 3.2.0 pre-release with qutebrowser. First I get this during some tests:

__________________________________________________________ ERROR at setup of test_installed_package ___________________________________________________________

self = <_pytest.skipping.MarkEvaluator object at 0x7fbd0e725dd8>

    def istrue(self):
        try:
>           return self._istrue()

.tox/py36/lib/python3.6/site-packages/_pytest/skipping.py:99: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_pytest.skipping.MarkEvaluator object at 0x7fbd0e725dd8>

    def _istrue(self):
        if hasattr(self, 'result'):
            return self.result
        if self.holder:
>           if self.holder.args or 'condition' in self.holder.kwargs:

.tox/py36/lib/python3.6/site-packages/_pytest/skipping.py:123: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <MarkInfo Mark(name='xfail', args=(), kwargs={'reason': 'https://github.com/qutebrowser/qutebrowser/issues/1070', 'strict': False})>

    def warned(self):
>       warnings.warn(warning, stacklevel=2)
E       _pytest.deprecated.RemovedInPytest4Warning: MarkInfo objects are deprecated as they contain the merged marks

.tox/py36/lib/python3.6/site-packages/_pytest/mark.py:15: RemovedInPytest4Warning

During handling of the above exception, another exception occurred:

self = <CallInfo when='setup' exception: 'MarkEvaluator' object has no attribute 'expr'>
func = <function call_runtest_hook.<locals>.<lambda> at 0x7fbd0e9609d8>, when = 'setup'

    def __init__(self, func, when):
        #: context of invocation: one of "setup", "call",
        #: "teardown", "memocollect"
        self.when = when
        self.start = time()
        try:
>           self.result = func()

.tox/py36/lib/python3.6/site-packages/_pytest/runner.py:157: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/py36/lib/python3.6/site-packages/_pytest/runner.py:145: in <lambda>
    return CallInfo(lambda: ihook(item=item, **kwds), when=when)
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:745: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:339: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:334: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:613: in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:254: in _wrapped_call
    return call_outcome.get_result()
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:279: in get_result
    raise ex[1].with_traceback(ex[2])
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:265: in __init__
    self.result = func()
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:613: in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:254: in _wrapped_call
    return call_outcome.get_result()
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:279: in get_result
    raise ex[1].with_traceback(ex[2])
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:265: in __init__
    self.result = func()
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:613: in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:254: in _wrapped_call
    return call_outcome.get_result()
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:279: in get_result
    raise ex[1].with_traceback(ex[2])
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:265: in __init__
    self.result = func()
.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:614: in execute
    res = hook_impl.function(*args)
.tox/py36/lib/python3.6/site-packages/_pytest/skipping.py:189: in pytest_runtest_setup
    check_xfail_no_run(item)
.tox/py36/lib/python3.6/site-packages/_pytest/skipping.py:205: in check_xfail_no_run
    if evalxfail.istrue():
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_pytest.skipping.MarkEvaluator object at 0x7fbd0e725dd8>

    def istrue(self):
        try:
            return self._istrue()
        except Exception:
            self.exc = sys.exc_info()
            if isinstance(self.exc[1], SyntaxError):
                msg = [" " * (self.exc[1].offset + 4) + "^", ]
                msg.append("SyntaxError: invalid syntax")
            else:
                msg = traceback.format_exception_only(*self.exc[:2])
            fail("Error evaluating %r expression\n"
                 "    %s\n"
                 "%s"
>                % (self.name, self.expr, "\n".join(msg)),
                 pytrace=False)
E           AttributeError: 'MarkEvaluator' object has no attribute 'expr'

.tox/py36/lib/python3.6/site-packages/_pytest/skipping.py:110: AttributeError

and then it fails entirely:

tests/end2end/features/test_history_bdd.py .....
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/main.py", line 108, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/main.py", line 144, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/main.py", line 167, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 613, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 254, in _wrapped_call
INTERNALERROR>     return call_outcome.get_result()
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 279, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 265, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 613, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 254, in _wrapped_call
INTERNALERROR>     return call_outcome.get_result()
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 279, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 265, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 613, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 254, in _wrapped_call
INTERNALERROR>     return call_outcome.get_result()
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 279, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 265, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/pytest_rerunfailures.py", line 52, in pytest_runtest_protocol
INTERNALERROR>     if "reruns" in rerun_marker.kwargs:
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/mark.py", line 15, in warned
INTERNALERROR>     warnings.warn(warning, stacklevel=2)
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/skipping.py", line 99, in istrue
INTERNALERROR>     return self._istrue()
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/skipping.py", line 123, in _istrue
INTERNALERROR>     if self.holder.args or 'condition' in self.holder.kwargs:
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/mark.py", line 15, in warned
INTERNALERROR>     warnings.warn(warning, stacklevel=2)
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/skipping.py", line 99, in istrue
INTERNALERROR>     return self._istrue()
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/skipping.py", line 123, in _istrue
INTERNALERROR>     if self.holder.args or 'condition' in self.holder.kwargs:
INTERNALERROR>   File "/home/florian/proj/qutebrowser/git/.tox/py36/lib/python3.6/site-packages/_pytest/mark.py", line 15, in warned
INTERNALERROR>     warnings.warn(warning, stacklevel=2)
INTERNALERROR> _pytest.deprecated.RemovedInPytest4Warning: MarkInfo objects are deprecated as they contain the merged marks

The text was updated successfully, but these errors were encountered:

nicoddemus · 2017-07-14T12:38:12Z

Thanks!

qutebrowser turns warnings into errors right? At first glance it seems the deprecation is now an error (as it should) but the error handling for that is faulty, so it breaks twice ( AttributeError: 'MarkEvaluator' object has no attribute 'expr').

Looking at the message, it is also not helpful at all; most users don't know what a MarkInfo object even is, we should improve that as well either by clarifying that on the message or pointing to the relevant section in the docs.

@RonnyPfannschmidt could you take a look at this?

The-Compiler · 2017-07-14T12:40:13Z

I can reproduce this with a simple xfailing test:

import pytest

@pytest.mark.xfail()
def test_foo():
    assert False

and launching pytest with --pythonwarnings error.

nicoddemus · 2017-07-14T12:44:54Z

Thanks! This also makes it clear to me that we should always test deprecations, and that they should be tested regarding the message in normal operation and when using --pythonwarnings error.

nicoddemus · 2017-07-14T15:49:33Z

I assumed initially that qutebrowser was using marks in parametrized parameters, but now that I look at it I think the warning is wrong.

Executing your sample with --pythonwarnings once shows:

foo.py::test_foo
  c:\users\bruno\pytest\_pytest\skipping.py:123: RemovedInPytest4Warning: MarkInfo objects are deprecated as they contain the merged marks
    if self.holder.args or 'condition' in self.holder.kwargs:

So I think the warning is being issued incorrectly as there's nothing wrong with that usage.

@RonnyPfannschmidt what do you think?

The-Compiler · 2017-07-14T15:58:06Z

As far as I know, I refactored all usages of parametrized marks to use pytest.param.

nicoddemus · 2017-07-14T16:00:53Z

As far as I know, I refactored all usages of parametrized marks to use pytest.param.

Good guy Florian. 😁

RonnyPfannschmidt · 2017-07-16T07:04:58Z

@nicoddemus this relates to not extracting marks form markinfo, unfortunately im not able to follow up in a timely manner atm

The-Compiler · 2017-07-17T18:32:24Z

FWIW if we can't easily solve this I'd vote for removing the deprecation, releasing v3.2.0, and then re-introducing it once this is fixed.

RonnyPfannschmidt · 2017-07-17T22:07:06Z

👍

As discussed in pytest-dev#2573, this change introduced a serious regression. Fix pytest-dev#2573 This reverts commit 0d0b01b.

nicoddemus · 2017-07-18T00:34:11Z

Opened a PR reverting that change.

fixes pytest-dev#2573

RonnyPfannschmidt · 2017-08-01T10:48:39Z

@The-Compiler can this one be closed?

The-Compiler · 2017-08-01T10:53:19Z

Do we have a reminder to re-introduce the warning?

RonnyPfannschmidt · 2017-08-01T10:54:47Z

this will be part of providing a alternative and i expect to finish it for 3.4 o 3.4

The-Compiler mentioned this issue Jul 14, 2017

Release 3.2.0 #2572

Closed

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Jul 18, 2017

Revert "introduce deprecation warnings for legacy parametersets"

6006e6a

As discussed in pytest-dev#2573, this change introduced a serious regression. Fix pytest-dev#2573 This reverts commit 0d0b01b.

nicoddemus mentioned this issue Jul 18, 2017

Revert "introduce deprecation warnings for legacy parametersets" #2584

Closed

RonnyPfannschmidt self-assigned this Jul 18, 2017

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue Jul 18, 2017

remove the MARK_INFO_ATTRIBUTE warning until we can fix internal usage

0bc9e92

fixes pytest-dev#2573

RonnyPfannschmidt mentioned this issue Jul 18, 2017

remove the MARK_INFO_ATTRIBUTE warning until we can fix internal usage #2585

Merged

4 tasks

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue Jul 18, 2017

remove the MARK_INFO_ATTRIBUTE warning until we can fix internal usage

8df2635

fixes pytest-dev#2573

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue Jul 18, 2017

remove the MARK_INFO_ATTRIBUTE warning until we can fix internal usage

e8ccb71

fixes pytest-dev#2573

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue Jul 18, 2017

remove the MARK_INFO_ATTRIBUTE warning until we can fix internal usage

3dd31bb

fixes pytest-dev#2573

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue Jul 19, 2017

remove the MARK_INFO_ATTRIBUTE warning until we can fix internal usage

62556ba

fixes pytest-dev#2573

RonnyPfannschmidt closed this as completed Aug 1, 2017

pytestbot mentioned this issue Mar 22, 2018

Improve tracebacks for ImportErrors in conftest #3332

Closed

pytestbot mentioned this issue Apr 6, 2018

pytest.raises 3.5.0 breaks for exception classes that look iterable (e.g. spyne.error.RequestNotAllowed) #3372

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

3.2.0: exceptions / internal errors related to markers #2573

3.2.0: exceptions / internal errors related to markers #2573

The-Compiler commented Jul 14, 2017

nicoddemus commented Jul 14, 2017

The-Compiler commented Jul 14, 2017

nicoddemus commented Jul 14, 2017

nicoddemus commented Jul 14, 2017

The-Compiler commented Jul 14, 2017

nicoddemus commented Jul 14, 2017

RonnyPfannschmidt commented Jul 16, 2017

The-Compiler commented Jul 17, 2017

RonnyPfannschmidt commented Jul 17, 2017

nicoddemus commented Jul 18, 2017

RonnyPfannschmidt commented Aug 1, 2017

The-Compiler commented Aug 1, 2017

RonnyPfannschmidt commented Aug 1, 2017

3.2.0: exceptions / internal errors related to markers #2573

3.2.0: exceptions / internal errors related to markers #2573

Comments

The-Compiler commented Jul 14, 2017

nicoddemus commented Jul 14, 2017

The-Compiler commented Jul 14, 2017

nicoddemus commented Jul 14, 2017

nicoddemus commented Jul 14, 2017

The-Compiler commented Jul 14, 2017

nicoddemus commented Jul 14, 2017

RonnyPfannschmidt commented Jul 16, 2017

The-Compiler commented Jul 17, 2017

RonnyPfannschmidt commented Jul 17, 2017

nicoddemus commented Jul 18, 2017

RonnyPfannschmidt commented Aug 1, 2017

The-Compiler commented Aug 1, 2017

RonnyPfannschmidt commented Aug 1, 2017