typing._type_check no longer recognizes several kinds of non-types

[davidfstr]

Bug report

The internal function typing._type_check() is used extensively to check whether a particular value represents a type. Its docstring begins with:

def _type_check(arg, msg, is_argument=True, module=None, *, allow_special_forms=False):
    """Check that the argument is a type, and return it (internal helper).

In Python 3.11b1 this function no longer errors on various kinds of non-types:

>>> import sys
>>> sys.version_info
sys.version_info(major=3, minor=11, micro=0, releaselevel='beta', serial=1)
>>> import typing
>>> typing._type_check(5, "Expected a type.")
# no error!
>>> typing.ClassVar[5]
typing.ClassVar[5]  # no error!

By contrast on Python 3.10:

>>> import sys
>>> sys.version_info
sys.version_info(major=3, minor=10, micro=1, releaselevel='final', serial=0)
>>> import typing
>>> typing._type_check(5, "Expected a type.")
TypeError: Expected a type. Got 5.
>>> typing.ClassVar[5]
TypeError: typing.ClassVar accepts only single type. Got 5.

Your environment

Regression introduced somewhere between Python 3.10 and Python 3.11b1.

[davidfstr]

CC @JelleZijlstra

[Fidget-Spinner]

I think commit 870b22b changed this behavior. It's intentional. Please see the explanation at #31151 (review).

[davidfstr]

I think commit 870b22b changed this behavior. It's intentional.

Understood. I'm currently using typing._type_check() inside the trycast library - which is a runtime type checker - to provide better error messages to users who mistakenly pass a regular Python value where a typeform is expected. For example the following common case:

trycast(int, 5)  # OK
trycast(5, int)  # ERROR: TypeError: trycast() requires a type as its first argument. Got 5.

It looks like typing._type_check() was intentionally loosened to not require the maybe-typeform it was checking to be callable so that:

1. ParamSpecArgs can be used inside Annotation,
2. InitVar can be used inside Annotation,
3. future typeform-likes can be used inside Annotation

It was understood that loosening this check would cause _type_check() to potentially deviate from how static type checkers recognize a "type", but that giving additional flexibility to the runtime typing implementation was considered to be a good tradeoff.

My particular usage of typing._type_check() in trycast desires to only allow garden-variety/universal typeforms (that are generally useable anywhere) but not "only-allowed-in-special-places" typeforms like ParamSpecArgs or InitVar. And it also doesn't want to allow "definitely-not-typeforms" like values of {int, float, bool} (ex: 5).

I would argue that questionable constructions like ClassVar[5] also would benefit from disallowing "definitely-not-typeforms" as well, as they did in Python ≤ 3.10.

Suggested remediations at the typing.py layer:

• Revert typing._type_check() to only permit "universal" typeforms (than can be used anywhere) by default, and have things like Annotated[] explicitly allow non-universal typeforms via something like typing._type_check(maybe_typeform, allow_nonuniversal=False).
  • Pro: The majority of typing._type_check() callers revert to disallowing non-typeforms and non-universal typeforms, with certain special callers like Annotated[] being able to opt-in to them.
  • Detail: Ways that a "universal" typeform could be recognized:
    • Proposal 1: callable(maybe_typeform) # former status-quo
    • Proposal 2: isinstance(maybe_typeform, (type, typing._SpecialForm))
    • Proposal 3: isinstance(maybe_typeform, typing._UniversalSpecialForm), where typing._UniversalSpecialForm is a new ABC, which at least {type, typing._SpecialForm} implement
• Add a deny-list to typing._type_check() to guard against "definitely-not-typeforms" like values of {int, float, bool}
  • Pro: Provides better best-effort recognition (and denial) of non-typeforms than the current implementation.
  • Con: A deny-list will necessarily be incomplete.
  • Con: A deny-list, since it will add potentially a lot of extra checks, will be slow to execute at runtime.

Suggested remediations at the trycast layer (for @davidfstr to consider if no remediations at the typing.py layer are accepted):

• Alter trycast to use a wrapper around typing._type_check() that reverts it to the Python ≤3.10 behavior of disallowing non-callables (which is currently a very reasonable way to filter out non-universal typeforms).

[davidfstr]

CC @GBeauregard as the originator of bpo-46644
CC @AlexWaygood as a +1er to @JelleZijlstra 's comment on that PR to accept it

[JelleZijlstra]

This was an intentional change to reduce the number of bugs where typing.py would reject current or future exotic but valid type annotations.

Also, _type_check is an internal implementation detail, subject to change at any time. It's also not very good at recognizing valid type annotations: things like ClassVar[len] were always accepted. You should implement your own, more precise, check for valid types in your library.

[davidfstr]

You should implement your own, more precise, check for valid types in your library.

Will do. Thanks for taking a look!