Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-89263: Add typing.get_overloads #31716

Merged
merged 37 commits into from
Apr 16, 2022
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
2ee377d
initial
JelleZijlstra Mar 6, 2022
831b565
Implementation, tests, and docs
JelleZijlstra Mar 7, 2022
f03f8a9
fix versionadded
JelleZijlstra Mar 7, 2022
404668a
Merge branch 'main' into funcregistry
JelleZijlstra Mar 8, 2022
7a5b0d1
make get_key_for_callable private
JelleZijlstra Mar 8, 2022
6998255
doc updates; remove unnecessary try-except
JelleZijlstra Mar 9, 2022
26bb908
Merge remote-tracking branch 'upstream/main' into funcregistry
JelleZijlstra Mar 27, 2022
f52b757
rename method
JelleZijlstra Mar 27, 2022
fc6a925
Don't store singledispatch in the registry
JelleZijlstra Mar 27, 2022
b524244
more tests
JelleZijlstra Mar 27, 2022
e95558e
and another
JelleZijlstra Mar 27, 2022
31fd72d
fix line length in new tests
JelleZijlstra Mar 27, 2022
7041ad3
Update Doc/library/functools.rst
JelleZijlstra Mar 27, 2022
e26b0db
Update Doc/library/typing.rst
JelleZijlstra Mar 27, 2022
1bf89fb
only for overload
JelleZijlstra Apr 2, 2022
83ac432
Merge remote-tracking branch 'upstream/main' into funcregistry
JelleZijlstra Apr 2, 2022
dfdbdc7
fix tests
JelleZijlstra Apr 2, 2022
e16c8d0
undo stray changes, fix NEWS entry
JelleZijlstra Apr 2, 2022
b3d2227
remove extra import
JelleZijlstra Apr 2, 2022
9727eee
Apply suggestions from code review
JelleZijlstra Apr 2, 2022
2e374b8
Apply suggestions from code review
JelleZijlstra Apr 3, 2022
ff03b12
Guido's feedback
JelleZijlstra Apr 3, 2022
17f0710
Optimizations suggested by Guido and Alex
JelleZijlstra Apr 3, 2022
2346970
inline _get_firstlineno, store outer objects for classmethod/staticme…
JelleZijlstra Apr 3, 2022
f2053a0
use defaultdict
JelleZijlstra Apr 3, 2022
b6131ad
another optimization
JelleZijlstra Apr 4, 2022
506bd66
Update Lib/typing.py
JelleZijlstra Apr 7, 2022
e9a2100
Merge remote-tracking branch 'upstream/main' into funcregistry
JelleZijlstra Apr 8, 2022
2b1a5cc
Merge remote-tracking branch 'upstream/main' into funcregistry
JelleZijlstra Apr 9, 2022
103bfd4
Simpler implementation (thanks Guido)
JelleZijlstra Apr 9, 2022
d453f7f
More comments and tests
JelleZijlstra Apr 9, 2022
450afeb
Merge remote-tracking branch 'upstream/main' into funcregistry
JelleZijlstra Apr 14, 2022
ea62287
simplify clear_overloads
JelleZijlstra Apr 14, 2022
905253c
use partial
JelleZijlstra Apr 14, 2022
debbf8a
add test
JelleZijlstra Apr 14, 2022
754c134
docs changes (thanks Alex)
JelleZijlstra Apr 14, 2022
1ad8224
Merge remote-tracking branch 'upstream/main' into funcregistry
JelleZijlstra Apr 14, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions Doc/library/typing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2254,6 +2254,32 @@ Functions and decorators

See :pep:`484` for details and comparison with other typing semantics.

.. versionchanged:: 3.11
Overloaded functions can now be introspected at runtime using :func:`get_overloads`.


.. function:: get_overloads(func)

Return a sequence of :func:`@overload <overload>`-decorated definitions for *func*. *func* is
the function object for the implementation of the overloaded function.
For example, given the definition of ``process`` in the documentation for
:func:`@overload <overload>`, ``get_overloads(process)`` will return a sequence of three
Fidget-Spinner marked this conversation as resolved.
Show resolved Hide resolved
function objects for the three defined overloads.

This function can be used for introspecting an overloaded function at runtime.

.. versionadded:: 3.11


.. function:: clear_overloads(func=None)

Clear all registered overloads for the given *func*. If *func* is None, clear
all overloads stored in the internal registry. This can be used to reclaim the
memory used by the registry.

.. versionadded:: 3.11


.. decorator:: final

A decorator to indicate to type checkers that the decorated method
Expand Down
77 changes: 73 additions & 4 deletions Lib/test/test_typing.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
from copy import copy, deepcopy

from typing import Any, NoReturn, Never, assert_never
from typing import overload, get_overloads, clear_overloads
from typing import TypeVar, TypeVarTuple, Unpack, AnyStr
from typing import T, KT, VT # Not in __all__.
from typing import Union, Optional, Literal
Expand Down Expand Up @@ -3794,11 +3795,22 @@ def test_or(self):
self.assertEqual("x" | X, Union["x", X])


@lru_cache()
def cached_func(x, y):
return 3 * x + y


class MethodHolder:
@classmethod
def clsmethod(cls): ...
@staticmethod
def stmethod(): ...
def method(self): ...


class OverloadTests(BaseTestCase):

def test_overload_fails(self):
from typing import overload

with self.assertRaises(RuntimeError):

@overload
Expand All @@ -3808,8 +3820,6 @@ def blah():
blah()

def test_overload_succeeds(self):
from typing import overload

@overload
def blah():
pass
Expand All @@ -3819,6 +3829,65 @@ def blah():

blah()

def set_up_overloads(self):
def blah():
pass

overload1 = blah
overload(blah)

def blah():
pass

overload2 = blah
overload(blah)

def blah():
pass

return blah, [overload1, overload2]

def test_overload_registry(self):
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved
impl, overloads = self.set_up_overloads()

self.assertEqual(list(get_overloads(impl)), overloads)
clear_overloads(impl)
self.assertEqual(get_overloads(impl), [])

impl, overloads = self.set_up_overloads()

self.assertEqual(list(get_overloads(impl)), overloads)
clear_overloads()
self.assertEqual(get_overloads(impl), [])

def test_overload_registry_repeated(self):
for _ in range(2):
impl, overloads = self.set_up_overloads()

self.assertEqual(list(get_overloads(impl)), overloads)

def test_get_key_for_callable(self):
self.assertEqual(
typing._get_key_for_callable(len),
"builtins.len",
)
self.assertEqual(
typing._get_key_for_callable(cached_func),
f"{__name__}.cached_func",
)
self.assertEqual(
typing._get_key_for_callable(MethodHolder.clsmethod),
f"{__name__}.MethodHolder.clsmethod",
)
self.assertEqual(
typing._get_key_for_callable(MethodHolder.stmethod),
f"{__name__}.MethodHolder.stmethod",
)
self.assertEqual(
typing._get_key_for_callable(MethodHolder.method),
f"{__name__}.MethodHolder.method",
)


# Definitions needed for features introduced in Python 3.6

Expand Down
68 changes: 68 additions & 0 deletions Lib/typing.py
Original file line number Diff line number Diff line change
Expand Up @@ -121,9 +121,11 @@ def _idfunc(_, x):
'assert_type',
'assert_never',
'cast',
'clear_overloads',
'final',
'get_args',
'get_origin',
'get_overloads',
'get_type_hints',
'is_typeddict',
'Never',
Expand Down Expand Up @@ -2412,10 +2414,76 @@ def utf8(value: bytes) -> bytes: ...
def utf8(value: str) -> bytes: ...
def utf8(value):
# implementation goes here

The overloads for a function can be retrieved at runtime using the
get_overloads() function.
"""
try:
key = _get_key_for_callable(func)
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved
except TypeError:
# Not a normal function; ignore.
pass
else:
# We're inlining get_overloads() here to avoid computing the key twice.
existing = _overload_registry.setdefault(key, [])
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved
if existing:
# If we are registering a variant with a lineno below or equal to that of the
# most recent existing variant, we're probably re-creating overloads for a
# function that already exists. In that case, we clear the existing variants
# to avoid leaking memory.
firstlineno = _get_firstlineno(func)
if firstlineno is not None:
existing_lineno = _get_firstlineno(existing[-1])
if existing_lineno is not None and firstlineno <= existing_lineno:
existing.clear()
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved

existing.append(func)
return _overload_dummy


def _get_firstlineno(func):
# staticmethod, classmethod
func = getattr(func, "__func__", func)
if not hasattr(func, '__code__'):
return None
return func.__code__.co_firstlineno


# {key: [overload]}
_overload_registry = {}


def get_overloads(func):
"""Return all defined overloads for *func* as a sequence."""
key = _get_key_for_callable(func)
return _overload_registry.get(key, [])


def clear_overloads(func=None):
"""Clear all overloads for the given function (or all functions)."""
if func is None:
_overload_registry.clear()
else:
key = _get_key_for_callable(func)
_overload_registry.pop(key, None)


def _get_key_for_callable(func):
"""Return a key for the given callable.

This is used as a key in the overload registry.

If no key can be created (because the object is not of a supported type), raise
AttributeError.
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved
"""
# classmethod and staticmethod
func = getattr(func, "__func__", func)
try:
return f"{func.__module__}.{func.__qualname__}"
JelleZijlstra marked this conversation as resolved.
Show resolved Hide resolved
except AttributeError:
raise TypeError(f"Cannot create key for {func!r}") from None


def final(f):
"""A decorator to indicate final methods and final classes.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add :func:`typing.get_overloads` and :func:`typing.clear_overloads`.
Patch by Jelle Zijlstra.