Doctests with HTTPError fail on Python 3.11 with traceback problem

[mauritsvanrees]

TLDR: two doctest files should be rewritten to unit tests.

Long version:

I am trying the coredev buildout on Python 3.11. See plone/buildout.coredev#821 for some troubles with this. There are actually not many test failures left. Latest full run has 5 failures. Three in PortalTransforms are being handled in plone/Products.PortalTransforms#49.

This leaves two failures that seem to have the same root cause. Both bin/test -s Products.CMFPlone -m test_functional -t redirection.txt and bin/test -s plone.browserlayer fail with:

Error in test /Users/maurits/community/plone-coredev/311/src/Products.CMFPlone/Products/CMFPlone/tests/redirection.txt
Traceback (most recent call last):
  File "/Users/maurits/.pyenv/versions/3.11.0rc2/lib/python3.11/unittest/case.py", line 57, in testPartExecutor
    yield
  File "/Users/maurits/.pyenv/versions/3.11.0rc2/lib/python3.11/unittest/case.py", line 623, in run
    self._callTestMethod(testMethod)
  File "/Users/maurits/.pyenv/versions/3.11.0rc2/lib/python3.11/unittest/case.py", line 579, in _callTestMethod
    if method() is not None:
  File "/Users/maurits/.pyenv/versions/3.11.0rc2/lib/python3.11/doctest.py", line 2215, in runTest
    failures, tries = runner.run(
  File "/Users/maurits/.pyenv/versions/3.11.0rc2/lib/python3.11/doctest.py", line 1497, in run
    return self.__run(test, compileflags, out)
  File "/Users/maurits/.pyenv/versions/3.11.0rc2/lib/python3.11/doctest.py", line 1372, in __run
    exc_msg = traceback.format_exception_only(*exception[:2])[-1]
  File "/Users/maurits/.pyenv/versions/3.11.0rc2/lib/python3.11/traceback.py", line 159, in format_exception_only
    te = TracebackException(type(value), value, None, compact=True)
  File "/Users/maurits/.pyenv/versions/3.11.0rc2/lib/python3.11/traceback.py", line 696, in __init__
    self.__notes__ = getattr(exc_value, '__notes__', None)
  File "/Users/maurits/.pyenv/versions/3.11.0rc2/lib/python3.11/tempfile.py", line 477, in __getattr__
    file = self.__dict__['file']
KeyError: 'file'

When I go in with a pdb:

(Pdb) self
<HTTPError 404: 'Not Found'>
(Pdb) pp self.__dict__
{'code': 404, 'fp': None, 'hdrs': [], 'msg': 'Not Found'}
(Pdb) type(self)
<class 'urllib.error.HTTPError'>

Stepping a bit up in the traceback, it goes wrong here:

> /Users/maurits/.pyenv/versions/3.11.0rc2/lib/python3.11/doctest.py(1372)__run()
-> exc_msg = traceback.format_exception_only(*exception[:2])[-1]
(Pdb) l
1367 	                if check(example.want, got, self.optionflags):
1368 	                    outcome = SUCCESS
1369 	
1370 	            # The example raised an exception:  check if it was expected.
1371 	            else:
1372 ->	                exc_msg = traceback.format_exception_only(*exception[:2])[-1]

So this is in a doctest. The test expects an exception. And now the code tries to check if the actual exception is indeed the one that is expected. And there it fails.

It is basically this error that you can see on a Python prompt:

>>> from urllib.error import HTTPError
>>> fp = open("foo.txt") # assuming foo.txt exists
>>> HTTPError('url', 'code', 'msg', 'hdrs', fp).__dict__['file']
<_io.TextIOWrapper name='foo.txt' mode='r' encoding='UTF-8'>
>>> HTTPError('url', 'code', 'msg', 'hdrs', None).__dict__['file']
Traceback (most recent call last):
  File "<console>", line 1, in <module>
KeyError: 'file'

I expected this to fail on 3.11 and pass on 3.10, but it fails in both cases. Jenkins has 3.11 final already.
Still, in 3.11 some stuff has changed in exceptions, so that might be it. Might be a bug in core Python. Ah, indeed a bug was filed in December 2021: python/cpython#90113

In our tests, the part that fails is these lines. And here for plone.browserlayer.
The best way forward may be to rewrite these doctests to unit tests.

[mauritsvanrees]

PRs are merged, and 3.11 is green now: https://jenkins.plone.org/job/plone-6.0-python-3.11/148/