[ty] Improve generic call expression inference

[ibraheemdev]

Summary

Implements astral-sh/ty#1356 and astral-sh/ty#136 (comment).

[github-actions]

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

[github-actions]

mypy_primer results

Changes were detected when running on open source projects

dulwich (https://github.com/dulwich/dulwich)
- dulwich/pack.py:1702:83: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- Found 175 diagnostics
+ Found 174 diagnostics

pydantic (https://github.com/pydantic/pydantic)
- pydantic/_internal/_validators.py:161:16: error[invalid-return-type] Return type does not match returned value: expected `Pattern[bytes]`, found `Pattern[str | bytes]`
+ pydantic/_internal/_validators.py:161:16: error[invalid-return-type] Return type does not match returned value: expected `Pattern[bytes]`, found `Pattern[bytes | str]`

pandera (https://github.com/pandera-dev/pandera)
+ tests/mypy/pandas_modules/pandas_dataframe.py:35:12: error[no-matching-overload] No overload of bound method `pipe` matches arguments
- tests/mypy/pandas_modules/pandas_dataframe.py:41:12: error[invalid-return-type] Return type does not match returned value: expected `DataFrame[SchemaOut]`, found `DataFrame[Schema] | DataFrame[SchemaOut]`
+ tests/mypy/pandas_modules/pandas_dataframe.py:41:12: error[invalid-return-type] Return type does not match returned value: expected `DataFrame[SchemaOut]`, found `DataFrame[SchemaOut] | DataFrame[Schema]`
- Found 1637 diagnostics
+ Found 1638 diagnostics

Expression (https://github.com/cognitedata/Expression)
- tests/test_result.py:491:21: error[invalid-argument-type] Argument to bound method `or_else` is incorrect: Expected `Result[Literal[42], Any]`, found `Result[Literal[0], Any]`
- tests/test_result.py:509:21: error[invalid-argument-type] Argument to bound method `or_else` is incorrect: Expected `Result[Any, Literal["original error"]]`, found `Result[Any, Literal["new error"]]`
- Found 200 diagnostics
+ Found 198 diagnostics

sphinx (https://github.com/sphinx-doc/sphinx)
- sphinx/domains/python/__init__.py:672:42: error[index-out-of-bounds] Index 0 is out of bounds for string `Literal[""]` with length 0
- Found 515 diagnostics
+ Found 514 diagnostics

vision (https://github.com/pytorch/vision)
- torchvision/prototype/transforms/_misc.py:34:13: error[invalid-assignment] Object of type `dict[Any, Sequence[int] & ~Top[dict[Unknown, Unknown]]]` is not assignable to `Sequence[int] | dict[type, Sequence[int] | None]`
- Found 1503 diagnostics
+ Found 1502 diagnostics

cloud-init (https://github.com/canonical/cloud-init)
- tests/unittests/sources/test_lxd.py:40:5: error[non-subscriptable] Cannot subscript object of type `int` with no `__getitem__` method
- tests/unittests/test_net.py:5337:41: warning[possibly-missing-attribute] Attribute `keys` may be missing on object of type `Unknown | list[Unknown | str] | dict[Unknown | str, Unknown | str | None] | dict[Unknown, Unknown]`
- tests/unittests/test_net.py:5494:41: warning[possibly-missing-attribute] Attribute `keys` may be missing on object of type `Unknown | list[Unknown | str] | dict[Unknown | str, Unknown | str | None] | dict[Unknown | str, Unknown | str]`
- Found 1134 diagnostics
+ Found 1131 diagnostics

meson (https://github.com/mesonbuild/meson)
+ mesonbuild/build.py:1293:9: error[invalid-assignment] Object of type `list[Sequence[str | Literal[False]] | Literal[False]]` is not assignable to attribute `install_dir` of type `list[str | Literal[False]]`
- Found 1664 diagnostics
+ Found 1665 diagnostics

scikit-build-core (https://github.com/scikit-build/scikit-build-core)
- src/scikit_build_core/format.py:74:16: error[invalid-return-type] Return type does not match returned value: expected `PyprojectFormatter | dict[str, str]`, found `dict[str, Literal["*"]]`
- Found 49 diagnostics
+ Found 48 diagnostics

colour (https://github.com/colour-science/colour)
- colour/recovery/otsu2018.py:1606:21: error[invalid-assignment] Method `__setitem__` of type `Unknown | (Overload[(key: SupportsIndex, value: Unknown, /) -> None, (key: slice[Any, Any, Any], value: Iterable[Unknown], /) -> None]) | (bound method dict[Unknown, Unknown].__setitem__(key: Unknown, value: Unknown, /) -> None)` cannot be called with a key of type `Node_Otsu2018` and a value of type `Unknown | list[Unknown] | dict[Unknown, Unknown] | int` on object of type `Unknown | list[Unknown] | dict[Unknown, Unknown] | int`
- colour/recovery/otsu2018.py:1607:21: error[unsupported-operator] Operator `+=` is unsupported between objects of type `list[Unknown]` and `Literal[1]`
- colour/recovery/otsu2018.py:1607:21: error[unsupported-operator] Operator `+=` is unsupported between objects of type `dict[Unknown, Unknown]` and `Literal[1]`
- colour/recovery/otsu2018.py:1610:17: error[invalid-assignment] Method `__setitem__` of type `Unknown | (Overload[(key: SupportsIndex, value: Unknown, /) -> None, (key: slice[Any, Any, Any], value: Iterable[Unknown], /) -> None]) | (bound method dict[Unknown, Unknown].__setitem__(key: Unknown, value: Unknown, /) -> None)` cannot be called with a key of type `Node_Otsu2018` and a value of type `int` on object of type `Unknown | list[Unknown] | dict[Unknown, Unknown] | int`
- colour/recovery/otsu2018.py:1610:54: error[invalid-argument-type] Argument to function `len` is incorrect: Expected `Sized`, found `Unknown | list[Unknown] | dict[Unknown, Unknown] | int`
- colour/recovery/otsu2018.py:1611:17: warning[possibly-missing-attribute] Attribute `append` may be missing on object of type `Unknown | list[Unknown] | dict[Unknown, Unknown] | int`
- Found 567 diagnostics
+ Found 561 diagnostics

static-frame (https://github.com/static-frame/static-frame)
+ static_frame/test/unit/test_series.py:4489:37: error[no-matching-overload] No overload of function `round` matches arguments
- Found 2004 diagnostics
+ Found 2005 diagnostics

core (https://github.com/home-assistant/core)
+ homeassistant/components/homekit/config_flow.py:502:9: error[invalid-assignment] Object of type `Any` is not assignable to `EntityFilterDict`
+ homeassistant/components/homekit/config_flow.py:546:9: error[invalid-assignment] Object of type `Any` is not assignable to `EntityFilterDict`
+ homeassistant/components/homekit/config_flow.py:649:9: error[invalid-assignment] Object of type `Any` is not assignable to `EntityFilterDict`
- Found 14455 diagnostics
+ Found 14458 diagnostics

No memory usage changes detected ✅

[github-actions]

ecosystem-analyzer results

Lint rule	Added	Removed	Changed
invalid-assignment	4	3	0
invalid-argument-type	0	3	0
invalid-return-type	0	1	2
possibly-missing-attribute	0	3	0
no-matching-overload	2	0	0
unsupported-operator	0	2	0
index-out-of-bounds	0	1	0
non-subscriptable	0	1	0
unused-ignore-comment	0	1	0
Total	6	15	2

Full report with detailed diff (timing results)

[codspeed-hq]

CodSpeed Performance Report

Merging #21210 will degrade performances by 7.82%

_{Comparing ibraheem/generic-call-inference (4aa5f3a) with main (d258302)}

Summary

❌ 4 regressions
✅ 18 untouched
⏩ 30 skipped¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Mode	Benchmark	BASE	HEAD	Change
❌	Simulation	hydra-zen	861.8 ms	927.5 ms	-7.08%
❌	WallTime	medium[colour-science]	15.1 s	16.3 s	-7.82%
❌	WallTime	medium[pandas]	43.1 s	45.4 s	-5.08%
❌	WallTime	medium[static-frame]	13 s	13.6 s	-4.63%

Footnotes

1. 30 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[ibraheemdev]

I imagine we could avoid the performance regression by special casing x: ... | None = f() to avoid the inference attempt using only None as the type context, but I'm a little hesitant to make that change, as there are (rare) cases where we might want to narrow to None. Most of the regressions seem relatively reasonable, other than colour.

Also note that a lot of the diagnostic improvements won't be seen in this PR directly, but #20933 (which is currently stacked on top of this PR).

[MichaReiser]

Do we understand what's causing the performance regression? Is it that we need to clone bindings or is it something else?

[ibraheemdev]

Most of the performance regressions have been resolved. There is an unavoidable regression when assigning the result of a generic call to a union, as we have to attempt inference multiple times for each element of the union.

[carljm]

Can you take a look at the new diagnostics in the ecosystem report? Just taking a quick glance, I see in HomeAssistant a few occurrences of "object of type Any is not assignable ..." which definitely doesn't look right.

[ibraheemdev]

It looks like those failures are due to TypedDict being assignable to None, which confuses overload evaluation. The weird diagnostic should be fixed by #21267, the actual type we infer (with type context) is Any | None there, not Any.

I added a failing test to track those.

[AlexWaygood]

I'm not too familiar with this machinery, but from what I can tell this all looks reasonable to me, and I definitely approve of all of the semantics changes I see in the tests! This is, as ever, very cool work

[carljm]

Oops, I had a couple draft comments from a review I started on Friday but hadn't completed yet. Nothing important, just things to consider for future.

[carljm]

Nit: should these just be another heading layer? Our usual approach is to reserve filenames for tests that actually need to be multi-file (because they are testing cross-module behavior / imports) and use headings when we are just trying to group related tests together.

[carljm]

We should probably do literal promotion here?

[AlexWaygood]

that might be fixed in #21320 :-)