Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PyTest internal error when using "except*" on an unhashable exception #10466

Closed
joseph-e-k opened this issue Nov 3, 2022 · 4 comments
Closed

PyTest internal error when using "except*" on an unhashable exception #10466

joseph-e-k opened this issue Nov 3, 2022 · 4 comments
Labels
type: bug problem that needs to be addressed

Comments

@joseph-e-k
Copy link

joseph-e-k commented Nov 3, 2022 •
edited
Loading

Tl;dr: A strange exception thrown by except* from "line -1" causes PyTest to crash when trying to retrieve the line it came from. Code to reproduce:

class MyException(Exception):
    __hash__ = None


def test_broken():
    try:
        raise ExceptionGroup("Foo", [
            MyException("Bar")
        ])
    except* Exception:
        pass

And then run that test function (test_broken) with pytest.

Expected result: test succeeds.

Expected result once you know about the weird behavior of expect* (see below): test fails with a TypeError.

Actual result: pytest fails with the following traceback; test is marked as "not run".

Launching pytest with arguments test_broken.py::test_broken --no-header --no-summary -q in C:\Users\Josep\PycharmProjects\sloge\tests

============================= test session starts =============================
collecting ... collected 1 item

test_broken.py::test_broken 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\main.py", line 270, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\main.py", line 324, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\main.py", line 349, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\runner.py", line 112, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\runner.py", line 131, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\runner.py", line 222, in call_and_report
INTERNALERROR>     report: TestReport = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_callers.py", line 55, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\skipping.py", line 265, in pytest_runtest_makereport
INTERNALERROR>     rep = outcome.get_result()
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\pluggy\_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\runner.py", line 366, in pytest_runtest_makereport
INTERNALERROR>     return TestReport.from_item_and_call(item, call)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\reports.py", line 349, in from_item_and_call
INTERNALERROR>     longrepr = item.repr_failure(excinfo)
INTERNALERROR>                ^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\python.py", line 1823, in repr_failure
INTERNALERROR>     return self._repr_failure_py(excinfo, style=style)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\nodes.py", line 484, in _repr_failure_py
INTERNALERROR>     return excinfo.getrepr(
INTERNALERROR>            ^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\_code\code.py", line 669, in getrepr
INTERNALERROR>     return fmt.repr_excinfo(self)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\_code\code.py", line 944, in repr_excinfo
INTERNALERROR>     reprtraceback = self.repr_traceback(excinfo_)
INTERNALERROR>                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\_code\code.py", line 871, in repr_traceback
INTERNALERROR>     reprentry = self.repr_traceback_entry(entry, einfo)
INTERNALERROR>                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\_code\code.py", line 822, in repr_traceback_entry
INTERNALERROR>     s = self.get_source(source, line_index, excinfo, short=short)
INTERNALERROR>         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "C:\Users\Josep\PycharmProjects\sloge\venv\Lib\site-packages\_pytest\_code\code.py", line 755, in get_source
INTERNALERROR>     lines.append(self.flow_marker + "   " + source.lines[line_index])
INTERNALERROR>                                             ~~~~~~~~~~~~^^^^^^^^^^^^
INTERNALERROR> IndexError: list index out of range

============================ no tests ran in 0.08s ============================

I'm running CPython 3.11.0 for Windows (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)], and pytest 7.2.0. My complete pip list:

Package    Version
---------- -------
attrs      22.1.0
colorama   0.4.6
iniconfig  1.1.1
packaging  21.3
pip        22.3
pluggy     1.0.0
pyparsing  3.0.9
pytest     7.2.0
setuptools 65.5.0

I think the cause of the bug is as follows.

Python 3.11's new except* syntax has an odd quirk: it throws an exception (from within Python itself) if you use it on a try block that throws an unhashable exception. Normal try ... except blocks don't do this. For example, this works fine:

try:
    raise ExceptionGroup("Foo", [
        MyException("Bar")
    ])
except Exception:
    pass

But this:

class MyException(Exception):
    __hash__ = None

try:
    raise ExceptionGroup("Foo", [
        MyException("Bar")
    ])
except* Exception:
    pass

Produces the following unhandled exception traceback:

  + Exception Group Traceback (most recent call last):
  |   File "C:\Users\Josep\AppData\Roaming\JetBrains\PyCharmCE2022.2\scratches\scratch_61.py", line 6, in <module>
  |     raise ExceptionGroup("Foo", [
  | ExceptionGroup: Foo (1 sub-exception)
  +-+---------------- 1 ----------------
    | MyException: Bar
    +------------------------------------

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Josep\AppData\Roaming\JetBrains\PyCharmCE2022.2\scratches\scratch_61.py", line -1, in <module>
TypeError: unhashable type: 'MyException'

I'm fairly sure this qualifies as an actual bug in the Python implementation I've got, as I couldn't find this behavior documented anywhere, but that's a ticket for a different repo.

Back to pytest. At the top of this bug report is a block of code reproducing this exception inside a test. Obviously the correct behavior would be for pytest to catch the TypeError, mark the test as failing due to an uncaught exception, and display it with full traceback - or, if that was completely impossible due to the "line -1" thing, then without one.

The most important thing is that the test should fail due to an uncaught exception, rather than pytest itself throwing an internal error and then marking the test as "not run".
@Zac-HD Zac-HD added the type: bug problem that needs to be addressed label Nov 6, 2022
@Zac-HD
Copy link
Member

Zac-HD commented Nov 6, 2022

This looks like a straightforward bug in our get_source() function, which should not (anymore?) assume that line_index is in source.lines; we might need some fallback logic for the can't-find-source case. Thanks very much for the report!

@asottile
Copy link
Member

asottile commented Nov 6, 2022

I think you should report the TypeError to cpython -- and then there's nothing to fix here I believe

This was referenced Nov 7, 2022
Closed
Closed
@Zac-HD
Copy link
Member

Zac-HD commented Nov 7, 2022

Confirmed as an upstream bug - thanks again for the report @joseph-e-k! I'd guess that this will be fixed in Python 3.11.1 🙂

@Zac-HD Zac-HD closed this as completed Nov 7, 2022
@joseph-e-k
Copy link
Author

Thanks very much! I'll just work around it for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: bug problem that needs to be addressed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@asottile @joseph-e-k @Zac-HD