[3.14] annotationlib - get_annotations returns an empty annotations dict if an `AttributeError` is raised when `__annotations__` is accessed

[DavidCEllis]

Bug report

Bug description:

If there's an annotation with an incorrect attribute access, the AttributeError causes get_annotations to return an empty dictionary for the annotations instead of failing or returning a ForwardRef.

import typing
from annotationlib import get_annotations, Format

class Example2:
    real_attribute: typing.Any
    fake_attribute: typing.DoesNotExist

new_ann = get_annotations(Example2, format=Format.FORWARDREF)

print(f"{new_ann=}")

# This should fail, but instead returns an empty dict
value_ann = get_annotations(Example2, format=Format.VALUE)

print(f"{value_ann=}")

string_ann = get_annotations(Example2, format=Format.STRING)

print(f"{string_ann=}")

Output

new_ann={}
value_ann={}
string_ann={'real_attribute': 'typing.Any', 'fake_attribute': 'typing.DoesNotExist'}

I think this is due to _get_dunder_annotations catching AttributeError and returning an empty dict, intended for static types but catching this by mistake.

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Linked PRs

• gh-125618: Make FORWARDREF format succeed more often #132818

[JelleZijlstra]

Thanks, that's an interesting case. Not immediately clear to me how we can fix this; it's not clear how to tell where the AttributeError came from. Maybe we should make static types return an empty dict in their __annotations__ descriptor instead of throwing AttributeError.

[DavidCEllis]

I was actually testing it to see if I'd get a _Stringifier again or a ForwardRef based on the other issue and was a bit surprised when I got nothing instead.

[JelleZijlstra]

In the FORWARDREF format we try to return .__annotations__ first. If that succeeds, we're done. And in this case, it arguably does succeed.

[DavidCEllis]

If you directly access Example2.__annotations__ you get the AttributeError though. The 'success' is entirely because this error is being silenced.

I'd expect VALUE to raise this exception. I'd expect FORWARDREF to either also error in this way or return a ForwardRef for the (currently) invalid attribute.

Traceback (most recent call last):
  File "/home/david/src/scratch/annotation_issue.py", line 34, in <module>
    Example2.__annotations__
  File "/home/david/src/scratch/annotation_issue.py", line 19, in __annotate__
    fake_attribute: typing.DoesNotExist
                    ^^^^^^^^^^^^^^^^^^^
  File "/home/david/src/cpython/Lib/typing.py", line 3829, in __getattr__
    raise AttributeError(f"module {__name__!r} has no attribute {attr!r}")
AttributeError: module 'typing' has no attribute 'DoesNotExist'

[DavidCEllis]

So I came back to this and realised that this is actually worse if some of the types are from a submodule that's only imported in a TYPE_CHECKING block.

For instance:

ex/__init__.py

class SomeClass:
    ...

ex/protocols.py

class SomeProtocol:
    ...

demo.py

import typing
from dataclasses import dataclass

import ex

if typing.TYPE_CHECKING:
    import ex.protocols

@dataclass
class Demo:
    a: ex.SomeClass | None = None
    b: ex.protocols.SomeProtocol | None = None

print(Demo())

python -c "import demo"

Demo()

python -c "import ex.protocols; import demo"

Demo(a=None, b=None)

So the definition of the dataclass can change depending on whether ex.protocols is imported elsewhere before this class is defined.

[DavidCEllis]

Was writing a test for this and found another variation of the bug along with a confusing error message which I'd consider a separate bug.

in 3.13 if you try inspect.get_annotations on an instance you get an error message:

>>> inspect.get_annotations(X())
TypeError: <__main__.X object at 0x70591bb8c980> is not a module, class, or callable.

However in the current 3.14 main there are no such guards which leads to some unexpected behaviour.

The first case is the same bug as we have here in a different presentation.

from annotationlib import get_annotations, Format

class Example:
   a: int

ex = Example()

print(f"{get_annotations(ex) = }")
print(f"{get_annotations(Example) = }")
print(f"{get_annotations(ex) = }")

Output:

get_annotations(ex) = {}
get_annotations(Example) = {'a': <class 'int'>}
get_annotations(ex) = {'a': <class 'int'>}

The bug is this function:
https://github.com/python/cpython/blob/53908bd7905b849e110d2c6f4bce739bff037146/Lib/annotationlib.py#L833C1-L847C21

It assumes AttributeError means the object is a "static type" and returns an empty dictionary¹. As the instance ex doesn't have an __annotations__ attribute, it raises AttributeError and you see this empty dict. However, after get_annotations has been called on Example, __annotations__ has been created on the class and is now retrieved.

>>> class Example:
...     a: int
...     
>>> ex = Example()
>>> ex.__annotations__
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    ex.__annotations__
AttributeError: 'Example' object has no attribute '__annotations__'. Did you mean: '__annotate__'?
>>> Example.__annotations__
{'a': <class 'int'>}
>>> ex.__annotations__
{'a': <class 'int'>}

Edit: Accidentally linked to the wrong function - it should be _get_dunder_annotations.

Footnotes

1. Not actually the case, I realised later that this occurs in the else branch. Same outcome though. ↩

[DavidCEllis]

I wrote some test cases for this error.

My initial thought is that _get_dunder_annotations should return None instead of an empty dict to indicate that __annotations__ has not been found, and then usages of this function should check if __annotate__ exists before potentially trying to call it. Probably wrapping all of this in its own function?

The test cases - STRING passes but is included for completion.

class TestGetAnnotationsAttributeError(unittest.TestCase):
    def test_value(self):
        class HasError:
            missing_attribute: base.missing_attribute

        # Before creating 'base'
        with self.assertRaises(NameError):
            annotationlib.get_annotations(HasError)

        base = object()

        # Should now be an attributeerror for base.missing_attribute
        with self.assertRaises(AttributeError):
            annotationlib.get_annotations(HasError)

    def test_forwardref(self):
        class HasError:
            missing_attribute: base.missing_attribute

        missing_ref = support.EqualToForwardRef(
            "base.missing_attribute",
            owner=HasError,
            is_class=True,
        )

        # before creating 'base'
        forward_annos = annotationlib.get_annotations(HasError, format=Format.FORWARDREF)

        self.assertEqual(
            forward_annos,
            {"missing_attribute": missing_ref}
        )

        # Create base but the annotations should be the same
        base = object()

        forward_annos = annotationlib.get_annotations(HasError, format=Format.FORWARDREF)

        self.assertEqual(
            forward_annos,
            {"missing_attribute": missing_ref}
        )


    def test_string(self):
        class HasError:
            missing_attribute: base.missing_attribute

        string_annos = annotationlib.get_annotations(HasError, format=Format.STRING)

        self.assertEqual(
            string_annos,
            {"missing_attribute": "base.missing_attribute"}
        )

        base = object()

        string_annos = annotationlib.get_annotations(HasError, format=Format.STRING)

        self.assertEqual(
            string_annos,
            {"missing_attribute": "base.missing_attribute"}
        )

[JelleZijlstra]

#132195 makes some related changes.

[DavidCEllis]

From these tests, #132195 makes test_value behave as expected - NameError in the first case and AttributeError in the second.

The test_forwardref as written above still fails as it seems Format.FORWARDREF now raises an AttributeError but I think it should return a ForwardRef the same as it would be before the import. The error is better than the incorrect result, but I don't think this format should fail in this way.

[Edit: rephrasing for clarity]