Cache types.GenericAlias in getitems

[eltoder]

The "old-style" generic classes from the typing module (List, Dict, etc) cached typing._GenericAlias instances retuned from __getitem__:

>>> from typing import List
>>> List[int] is List[int]
True

But when this functionality was ported to built-in types, caching was not implemented:

>>> list[int] is list[int]
False

Since it is very common to subscript the same types with the same arguments, caching will be very beneficial here and will reduce memory footprint of heavily type-annotated modules.

Linked PRs

• gh-101859: Add caching of types.GenericAlias objects #103541

[Fidget-Spinner]

Oh yeah. I think @gvanrossum envisioned that at some point https://github.com/python/cpython/blob/main/Objects/genericaliasobject.c#L917

We should definitely implement some type of cache. Even if it doesn't speed things up much, it should save space.

[gvanrossum]

Agreed.

[sobolevn]

@Fidget-Spinner I am working on this right now, but I have several technical questions.

Usually, we place cache in a module state. But, since genericaliasobject is special, it does not have any module-level state (and should not have one).

Another option is to have a static PyObject *TYPE_CACHE global variable, but:

1. We try to get rid of those
2. I am not sure how to clean it up properly

Any ideas? :)

[Fidget-Spinner]

Builtin object caches tend to go into interpreter state. See for example the slice cache

cpython/Include/internal/pycore_interp.h

Line 137 in 03089fd

PySliceObject *slice_cache;

There's the dict, tuple and other cache also in there.

You would need to setup and clear the state in the interpreter state initializer and destructor (sorry I forgot where those are).

[sobolevn]

Thanks, I was not aware of that! 👍

[AlexWaygood]

Cc. @samuelcolvin, who just mentioned at the typing summit that the current caching for typing.List etc causes problems for pydantic

[eltoder]

@AlexWaygood Are there any details re what kind of issues?

[samuelcolvin]

The issue is that the order of union members can be cached after the first union is created. Causing problems for runtime type checking where order can matter.

[AlexWaygood]

@AlexWaygood Are there any details re what kind of issues?

Yes, please see:

• gh-101859: Add caching of types.GenericAlias objects #103541 (comment) (and the following discussion)
• https://discuss.python.org/t/changing-runtime-behavior-of-types-uniontype-to-respect-order-in-equality-and-hashing/26071
• Make sure that typing cache differentiate Union[int, str] and Union[str, int] #103749