[DOC] typing.get_type_hints() raises for type aliases with forward references

[mhils]

BPO	44926
Nosy	@gvanrossum, @ambv, @mhils, @miss-islington, @Fidget-Spinner
PRs	bpo-44926: get_type_hints: Add note about type aliases with forward refs #27859 [3.10] bpo-44926: get_type_hints: Add note about type aliases with forward refs (GH-27859) #27860

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-10-18.07:55:48.195>
created_at = <Date 2021-08-16.17:15:16.657>
labels = ['3.11', 'type-bug', 'library', '3.10', 'docs']
title = '[DOC] typing.get_type_hints() raises for type aliases with forward references'
updated_at = <Date 2021-10-18.07:55:48.191>
user = 'https://github.com/mhils'

bugs.python.org fields:

activity = <Date 2021-10-18.07:55:48.191>
actor = 'iritkatriel'
assignee = 'docs@python'
closed = True
closed_date = <Date 2021-10-18.07:55:48.195>
closer = 'iritkatriel'
components = ['Documentation', 'Library (Lib)']
creation = <Date 2021-08-16.17:15:16.657>
creator = 'mhils'
dependencies = []
files = []
hgrepos = []
issue_num = 44926
keywords = ['patch']
message_count = 13.0
messages = ['399660', '399662', '399664', '399672', '399675', '399677', '399746', '399774', '399829', '399850', '399852', '399974', '400086']
nosy_count = 6.0
nosy_names = ['gvanrossum', 'docs@python', 'lukasz.langa', 'mhils', 'miss-islington', 'kj']
pr_nums = ['27859', '27860']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44926'
versions = ['Python 3.10', 'Python 3.11']

[mhils]

Someone reported this rather interesting issue where typing.get_type_hints crashes on type aliases with forward references. The original report is at mitmproxy/pdoc#290. Here's an extended minimal example:

foo.py:

import typing

FooList1: typing.TypeAlias = list["Foo"]
FooList2: typing.TypeAlias = typing.List["Foo"]

class Foo:
    pass

bar.py:

import typing
import foo

def func1(x: foo.FooList1):
    pass

def func2(x: foo.FooList2):
    pass

print(typing.get_type_hints(func1))  # {'x': list['Foo']}
print(typing.get_type_hints(func2))  # NameError: name 'Foo' is not defined.

Observations:

1. func1 doesn't crash, but also doesn't resolve the forward reference. I am not sure if this expected behavior.
   If it isn't, this should eventually run in the same problem as func2.
2. func2 crashes because "Foo" is evaluated in the context of bar.py (where class Foo is unknown) and not in the context of foo.py. ForwardRef._evaluate would
   somehow need to know in which context it was defined. bpo-41249 (TypedDict inheritance doesn't work with get_type_hints) introduced
   ForwardRef.__forward_module__, which would be a logical place for that information. I'm not sure if it is a good idea
   to use __forward_module__ more widely.
3. This may end up as quite a bit of complexity for an edge case, I'm fine if it is considered wontfix.
   The reason I'm bringing it up is that PEP-613 (Explicit Type Aliases) decidedly allows forward references in type aliases.

For the record, PEP-563 (postponed evaluations) does not change the outcome here. However, postponed evaluations often make it possible to avoid the forward references by declaring the aliases last.

[Fidget-Spinner]

1. func1 doesn't crash, but also doesn't resolve the forward reference. I am not sure if this expected behavior.

Good observation! That's indeed the current behavior. The reason is a little subtle - list[...] is using a special builtin type (https://docs.python.org/3/library/stdtypes.html#types-genericalias), while typing.List is the one from typing. The typing version wraps all strings in ForwardRef(), while the builtin version shouldn't import from typing, so it doesn't have that luxury. get_type_hints could probably support the builtin version, though I'd imagine we'd need some rework.

Re: 2. You're right there too. Storing the defined module in __forward_module__ is an interesting proposal.

For your specific use case (where the user is using Python 3.6), you could pass in globalns and localns to get_type_hints as a temporary workaround. Off the top of my head:
get_type_hints(func2, globalns=foo.__dict__) might work. Would that work for your library?

[mhils]

For your specific use case (where the user is using Python 3.6), you could pass in globalns and localns to get_type_hints as a temporary workaround. Off the top of my head:
get_type_hints(func2, globalns=foo.__dict__) might work. Would that work for your library?

I guess the hard part is knowing that the type annotation comes from foo. In the example here we can of course hardcode it, but that doesn't work in the general case or for pdoc, the documentation generator I'm working on (https://pdoc.dev). I have experimented quite a bit with walking the AST to figure out where type aliases are imported from to then re-executing ForwardRefs with that globalns. Long story short, trying to reverse-engineer forward_module quickly becomes a tangled hot mess where you need to adjust for import aliases, reimports, and so on.