Skip to content

Conversation

@dhruvmanila
Copy link
Member

@dhruvmanila dhruvmanila commented Aug 18, 2025

Summary

Closes: astral-sh/ty#669

(This turned out to be simpler that I thought :))

Test Plan

Update existing test cases.

Ecosystem report

Most of them are basically because ty has now started inferring more precise types for the return type to an overloaded call and a lot of the types are defined using type aliases, here's some examples:

Details

attrs (https://github.com/python-attrs/attrs)

  • tests/test_make.py:146:14: error[unresolved-attribute] Type Literal[42] has no attribute default
  • Found 555 diagnostics
  • Found 556 diagnostics

This is accurate now that we infer the type as Literal[42] instead of Unknown (Pyright infers it as int)

optuna (https://github.com/optuna/optuna)

  • optuna/_gp/search_space.py:181:53: error[invalid-argument-type] Argument to function _round_one_normalized_param is incorrect: Expected tuple[int | float, int | float], found tuple[Unknown | ndarray[Unknown, <class 'float'>], Unknown | ndarray[Unknown, <class 'float'>]]
  • optuna/_gp/search_space.py:181:83: error[invalid-argument-type] Argument to function _round_one_normalized_param is incorrect: Expected int | float, found Unknown | ndarray[Unknown, <class 'float'>]
  • tests/gp_tests/test_search_space.py:109:13: error[invalid-argument-type] Argument to function _unnormalize_one_param is incorrect: Expected tuple[int | float, int | float], found Unknown | ndarray[Unknown, <class 'float'>]
  • tests/gp_tests/test_search_space.py:110:13: error[invalid-argument-type] Argument to function _unnormalize_one_param is incorrect: Expected int | float, found Unknown | ndarray[Unknown, <class 'float'>]
  • Found 559 diagnostics
  • Found 563 diagnostics

Same as above where ty is now inferring a more precise type like Unknown | ndarray[tuple[int, int], <class 'float'>] instead of just Unknown as before

jinja (https://github.com/pallets/jinja)

  • src/jinja2/bccache.py:298:39: error[invalid-argument-type] Argument to bound method write_bytecode is incorrect: Expected IO[bytes], found _TemporaryFileWrapper[str]
  • Found 186 diagnostics
  • Found 187 diagnostics

This requires support for type aliases to match the correct overload.

hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)

  • src/hydra_zen/wrapper/_implementations.py:945:16: error[invalid-return-type] Return type does not match returned value: expected DataClass_ | type[@Todo(type[T] for protocols)] | ListConfig | DictConfig, found @Todo(unsupported type[X] special form) | (((...) -> Any) & dict[Unknown, Unknown]) | (DataClass_ & dict[Unknown, Unknown]) | dict[Any, Any] | (ListConfig & dict[Unknown, Unknown]) | (DictConfig & dict[Unknown, Unknown]) | (((...) -> Any) & list[Unknown]) | (DataClass_ & list[Unknown]) | list[Any] | (ListConfig & list[Unknown]) | (DictConfig & list[Unknown])
  • tests/annotations/behaviors.py:60:28: error[call-non-callable] Object of type Path is not callable
  • tests/annotations/behaviors.py:64:21: error[call-non-callable] Object of type Path is not callable
  • tests/annotations/declarations.py:167:17: error[call-non-callable] Object of type Path is not callable
  • tests/annotations/declarations.py:524:17: error[unresolved-attribute] Type <class 'int'> has no attribute _target_
  • Found 561 diagnostics
  • Found 566 diagnostics

Same as above, this requires support for type aliases to match the correct overload.

paasta (https://github.com/yelp/paasta)

  • paasta_tools/utils.py:4188:19: warning[redundant-cast] Value is already of type list[str]
  • Found 888 diagnostics
  • Found 889 diagnostics

This is correct.

colour (https://github.com/colour-science/colour)

  • colour/plotting/diagrams.py:448:13: error[invalid-argument-type] Argument to bound method __init__ is incorrect: Expected Sequence[@Todo(Support for typing.TypeAlias)], found ndarray[tuple[int, int, int], dtype[Unknown]]
  • colour/plotting/diagrams.py:462:13: error[invalid-argument-type] Argument to bound method __init__ is incorrect: Expected Sequence[@Todo(Support for typing.TypeAlias)], found ndarray[tuple[int, int, int], dtype[Unknown]]
  • colour/plotting/models.py:419:13: error[invalid-argument-type] Argument to bound method __init__ is incorrect: Expected Sequence[@Todo(Support for typing.TypeAlias)], found ndarray[tuple[int, int, int], dtype[Unknown]]
  • colour/plotting/temperature.py:230:9: error[invalid-argument-type] Argument to bound method __init__ is incorrect: Expected Sequence[@Todo(Support for typing.TypeAlias)], found ndarray[tuple[int, int, int], dtype[Unknown]]
  • colour/plotting/temperature.py:474:13: error[invalid-argument-type] Argument to bound method __init__ is incorrect: Expected Sequence[@Todo(Support for typing.TypeAlias)], found ndarray[tuple[int, int, int], dtype[Unknown]]
  • colour/plotting/temperature.py:495:17: error[invalid-argument-type] Argument to bound method __init__ is incorrect: Expected Sequence[@Todo(Support for typing.TypeAlias)], found ndarray[tuple[int, int, int], dtype[Unknown]]
  • colour/plotting/temperature.py:513:13: error[invalid-argument-type] Argument to bound method text is incorrect: Expected int | float, found ndarray[@Todo(Support for typing.TypeAlias), dtype[Unknown]]
  • colour/plotting/temperature.py:514:13: error[invalid-argument-type] Argument to bound method text is incorrect: Expected int | float, found ndarray[@Todo(Support for typing.TypeAlias), dtype[Unknown]]
  • Found 480 diagnostics
  • Found 488 diagnostics

Most of them are correct except for the last two diagnostics which I'm not sure
what's happening, it's trying to index into an np.ndarray type (which is
inferred correctly) but I think it might be picking up an incorrect overload
for the __getitem__ method.

Scipy's diagnostics also requires support for type alises to pick the correct overload.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 18, 2025

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

@github-actions
Copy link
Contributor

github-actions bot commented Aug 18, 2025

mypy_primer results

Changes were detected when running on open source projects
attrs (https://github.com/python-attrs/attrs)
+ tests/test_make.py:146:14: error[unresolved-attribute] Type `Literal[42]` has no attribute `default`
- Found 555 diagnostics
+ Found 556 diagnostics

optuna (https://github.com/optuna/optuna)
+ optuna/_gp/search_space.py:181:53: error[invalid-argument-type] Argument to function `_round_one_normalized_param` is incorrect: Expected `tuple[int | float, int | float]`, found `tuple[Unknown | ndarray[Unknown, <class 'float'>], Unknown | ndarray[Unknown, <class 'float'>]]`
+ optuna/_gp/search_space.py:181:83: error[invalid-argument-type] Argument to function `_round_one_normalized_param` is incorrect: Expected `int | float`, found `Unknown | ndarray[Unknown, <class 'float'>]`
+ tests/gp_tests/test_search_space.py:109:13: error[invalid-argument-type] Argument to function `_unnormalize_one_param` is incorrect: Expected `tuple[int | float, int | float]`, found `Unknown | ndarray[Unknown, <class 'float'>]`
+ tests/gp_tests/test_search_space.py:110:13: error[invalid-argument-type] Argument to function `_unnormalize_one_param` is incorrect: Expected `int | float`, found `Unknown | ndarray[Unknown, <class 'float'>]`
- Found 559 diagnostics
+ Found 563 diagnostics

jinja (https://github.com/pallets/jinja)
+ src/jinja2/bccache.py:298:39: error[invalid-argument-type] Argument to bound method `write_bytecode` is incorrect: Expected `IO[bytes]`, found `_TemporaryFileWrapper[str]`
- Found 186 diagnostics
+ Found 187 diagnostics

hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)
+ src/hydra_zen/wrapper/_implementations.py:945:16: error[invalid-return-type] Return type does not match returned value: expected `DataClass_ | type[@Todo(type[T] for protocols)] | ListConfig | DictConfig`, found `@Todo(unsupported type[X] special form) | (((...) -> Any) & dict[Unknown, Unknown]) | (DataClass_ & dict[Unknown, Unknown]) | dict[Any, Any] | (ListConfig & dict[Unknown, Unknown]) | (DictConfig & dict[Unknown, Unknown]) | (((...) -> Any) & list[Unknown]) | (DataClass_ & list[Unknown]) | list[Any] | (ListConfig & list[Unknown]) | (DictConfig & list[Unknown])`
+ tests/annotations/behaviors.py:60:28: error[call-non-callable] Object of type `Path` is not callable
+ tests/annotations/behaviors.py:64:21: error[call-non-callable] Object of type `Path` is not callable
+ tests/annotations/declarations.py:167:17: error[call-non-callable] Object of type `Path` is not callable
+ tests/annotations/declarations.py:524:17: error[unresolved-attribute] Type `<class 'int'>` has no attribute `_target_`
- Found 561 diagnostics
+ Found 566 diagnostics

paasta (https://github.com/yelp/paasta)
+ paasta_tools/utils.py:4188:19: warning[redundant-cast] Value is already of type `list[str]`
- Found 888 diagnostics
+ Found 889 diagnostics

colour (https://github.com/colour-science/colour)
+ colour/plotting/diagrams.py:448:13: error[invalid-argument-type] Argument to bound method `__init__` is incorrect: Expected `Sequence[@Todo(Support for `typing.TypeAlias`)]`, found `ndarray[tuple[int, int, int], dtype[Unknown]]`
+ colour/plotting/diagrams.py:462:13: error[invalid-argument-type] Argument to bound method `__init__` is incorrect: Expected `Sequence[@Todo(Support for `typing.TypeAlias`)]`, found `ndarray[tuple[int, int, int], dtype[Unknown]]`
+ colour/plotting/models.py:419:13: error[invalid-argument-type] Argument to bound method `__init__` is incorrect: Expected `Sequence[@Todo(Support for `typing.TypeAlias`)]`, found `ndarray[tuple[int, int, int], dtype[Unknown]]`
+ colour/plotting/temperature.py:230:9: error[invalid-argument-type] Argument to bound method `__init__` is incorrect: Expected `Sequence[@Todo(Support for `typing.TypeAlias`)]`, found `ndarray[tuple[int, int, int], dtype[Unknown]]`
+ colour/plotting/temperature.py:474:13: error[invalid-argument-type] Argument to bound method `__init__` is incorrect: Expected `Sequence[@Todo(Support for `typing.TypeAlias`)]`, found `ndarray[tuple[int, int, int], dtype[Unknown]]`
+ colour/plotting/temperature.py:495:17: error[invalid-argument-type] Argument to bound method `__init__` is incorrect: Expected `Sequence[@Todo(Support for `typing.TypeAlias`)]`, found `ndarray[tuple[int, int, int], dtype[Unknown]]`
+ colour/plotting/temperature.py:513:13: error[invalid-argument-type] Argument to bound method `text` is incorrect: Expected `int | float`, found `ndarray[@Todo(Support for `typing.TypeAlias`), dtype[Unknown]]`
+ colour/plotting/temperature.py:514:13: error[invalid-argument-type] Argument to bound method `text` is incorrect: Expected `int | float`, found `ndarray[@Todo(Support for `typing.TypeAlias`), dtype[Unknown]]`
- Found 480 diagnostics
+ Found 488 diagnostics

scrapy (https://github.com/scrapy/scrapy)
+ tests/test_pipeline_files.py:364:5: error[invalid-assignment] Object of type `<class 'list'>` is not assignable to `list[str]`
+ tests/test_pipeline_files.py:365:5: error[invalid-assignment] Object of type `<class 'list'>` is not assignable to `list[dict[str, str]]`
+ tests/test_pipeline_files.py:367:5: error[invalid-assignment] Object of type `<class 'list'>` is not assignable to `list[str]`
+ tests/test_pipeline_files.py:368:5: error[invalid-assignment] Object of type `<class 'list'>` is not assignable to `list[dict[str, str]]`
+ tests/test_pipeline_images.py:312:5: error[invalid-assignment] Object of type `<class 'list'>` is not assignable to `list[str]`
+ tests/test_pipeline_images.py:313:5: error[invalid-assignment] Object of type `<class 'list'>` is not assignable to `list[dict[str, str]]`
+ tests/test_pipeline_images.py:315:5: error[invalid-assignment] Object of type `<class 'list'>` is not assignable to `list[str]`
+ tests/test_pipeline_images.py:316:5: error[invalid-assignment] Object of type `<class 'list'>` is not assignable to `list[dict[str, str]]`
- Found 1058 diagnostics
+ Found 1066 diagnostics

static-frame (https://github.com/static-frame/static-frame)
- static_frame/core/bus.py:553:21: warning[possibly-unbound-implicit-call] Method `__setitem__` of type `(Unknown & ~None) | Iterable[Unknown | type[FrameDeferred]] | Unknown` is possibly unbound
+ static_frame/core/bus.py:553:21: warning[possibly-unbound-implicit-call] Method `__setitem__` of type `(Unknown & ~None) | Iterable[Unknown | type[FrameDeferred]]` is possibly unbound
- static_frame/core/bus.py:571:40: warning[possibly-unbound-attribute] Attribute `copy` on type `(Unknown & ~None) | Iterable[Unknown | type[FrameDeferred]] | Unknown` is possibly unbound
+ static_frame/core/bus.py:571:40: warning[possibly-unbound-attribute] Attribute `copy` on type `(Unknown & ~None) | Iterable[Unknown | type[FrameDeferred]]` is possibly unbound
+ static_frame/core/join.py:102:42: error[invalid-argument-type] Argument to function `nonzero_1d` is incorrect: Expected `ndarray[Unknown, Unknown]`, found `bool[bool] | @Todo(unknown type subscript)`
- Found 1778 diagnostics
+ Found 1779 diagnostics

scikit-learn (https://github.com/scikit-learn/scikit-learn)
- sklearn/_loss/loss.py:370:12: warning[possibly-unbound-attribute] Attribute `ndim` on type `Unknown | None` is possibly unbound
+ sklearn/_loss/loss.py:370:12: warning[possibly-unbound-attribute] Attribute `ndim` on type `Unknown | None | (Unknown & ~None)` is possibly unbound
- sklearn/_loss/loss.py:370:38: warning[possibly-unbound-attribute] Attribute `shape` on type `Unknown | None` is possibly unbound
+ sklearn/_loss/loss.py:370:38: warning[possibly-unbound-attribute] Attribute `shape` on type `Unknown | None | (Unknown & ~None)` is possibly unbound
- sklearn/_loss/loss.py:371:27: warning[possibly-unbound-attribute] Attribute `squeeze` on type `Unknown | None` is possibly unbound
+ sklearn/_loss/loss.py:371:27: warning[possibly-unbound-attribute] Attribute `squeeze` on type `Unknown | None | (Unknown & ~None)` is possibly unbound

sympy (https://github.com/sympy/sympy)
+ sympy/matrices/matrixbase.py:3268:16: error[missing-argument] No argument provided for required parameter 2
- Found 12993 diagnostics
+ Found 12994 diagnostics

scipy (https://github.com/scipy/scipy)
+ scipy/io/_idl.py:782:9: error[no-matching-overload] No overload of bound method `write` matches arguments
+ scipy/io/_idl.py:789:13: error[no-matching-overload] No overload of bound method `write` matches arguments
+ scipy/io/_idl.py:801:17: error[no-matching-overload] No overload of bound method `write` matches arguments
+ scipy/io/_idl.py:802:17: error[no-matching-overload] No overload of bound method `write` matches arguments
+ scipy/io/_idl.py:818:13: error[no-matching-overload] No overload of bound method `write` matches arguments
+ scipy/io/_idl.py:819:13: error[no-matching-overload] No overload of bound method `write` matches arguments
+ scipy/io/_idl.py:821:13: error[no-matching-overload] No overload of bound method `write` matches arguments
- scipy/linalg/tests/test_basic.py:1426:29: error[no-matching-overload] No overload of function `tri` matches arguments
- scipy/linalg/tests/test_basic.py:1438:29: error[no-matching-overload] No overload of function `tri` matches arguments
+ scipy/stats/_mstats_extras.py:88:13: error[invalid-assignment] Invalid assignment to data descriptor attribute `flat` on type `ndarray[tuple[int, int], Unknown]` with custom `__set__` method
- Found 6394 diagnostics
+ Found 6400 diagnostics
Memory usage changes were detected when running on open source projects
trio (https://github.com/python-trio/trio)
-     memo metadata = ~22MB
+     memo metadata = ~21MB

@dhruvmanila dhruvmanila added ty Multi-file analysis & type inference ecosystem-analyzer labels Aug 18, 2025
@dhruvmanila dhruvmanila force-pushed the dhruv/use-specialized-parameter-type branch from 628363b to 5ebe3e6 Compare August 19, 2025 05:40
@github-actions
Copy link
Contributor

ecosystem-analyzer results

Lint rule Added Removed Changed
invalid-argument-type 14 0 0
invalid-assignment 9 0 0
no-matching-overload 7 2 0
possibly-unbound-attribute 0 0 4
call-non-callable 3 0 0
unresolved-attribute 2 0 0
invalid-return-type 1 0 0
missing-argument 1 0 0
possibly-unbound-implicit-call 0 0 1
redundant-cast 1 0 0
Total 38 2 5

Full report with detailed diff

@dhruvmanila
Copy link
Member Author

dhruvmanila commented Aug 19, 2025

sympy (https://github.com/sympy/sympy)

This is interesting, it seems that a decorator that returns a callable on a method is removing the fact that the original function is an instance method:

    def __rmatmul__(self, other: MatrixBase) -> MatrixBase:
        self, other, T = _unify_with_other(self, other)

        if T != "is_matrix":
            return NotImplemented

		# This is the revealed type with the `call_highest_priority` decorator
		# ty: Revealed type: `Unknown | ((Unknown, Unknown, /) -> Unknown)` [revealed-type]
		# Pyright: Type of "self.__rmul__" is "(MatrixBase | Expr | complex) -> MatrixBase"
        return reveal_type(self.__rmul__)(other)

		# And, this is without the decorator
		# ty: Revealed type: `Unknown | (bound method MatrixBase.__rmul__(other: MatrixBase | Unknown) -> MatrixBase)` [revealed-type]


    @call_highest_priority('__mul__')
    def __rmul__(self, other: MatrixBase | SExpr) -> MatrixBase:
        return self.rmultiply(other)

This wasn't an easy before this PR because the call to an overloaded function _unify_with_other wasn't working correctly and the type of self, other and T was Unknown.

@dhruvmanila dhruvmanila marked this pull request as ready for review August 19, 2025 10:01
Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@dhruvmanila dhruvmanila merged commit f019cfd into main Aug 20, 2025
39 checks passed
@dhruvmanila dhruvmanila deleted the dhruv/use-specialized-parameter-type branch August 20, 2025 04:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ecosystem-analyzer ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use specialized parameter type for overload step 5 filtering

3 participants