Avoid Any specializations for covariant and contravariant type variables inside TypeIs returns

[AlexWaygood]

No description provided.

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

pandera (https://github.com/pandera-dev/pandera)
+ pandera/dtypes.py:561: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/pandas/test_dtypes.py:300: error: Item "object" of "object | Any" has no attribute "foo"  [union-attr]

graphql-core (https://github.com/graphql-python/graphql-core)
+ tests/execution/test_middleware.py:274: error: "object" has no attribute "data"  [attr-defined]
+ tests/execution/test_middleware.py:276: error: "object" has no attribute "data"  [attr-defined]

trio (https://github.com/python-trio/trio)
+ src/trio/_core/_tests/test_ki.py:685: error: Argument 1 to "_consume_async_generator" has incompatible type "AsyncGeneratorType[object, Never]"; expected "AsyncGenerator[None, None]"  [arg-type]
+ src/trio/_core/_tests/test_ki.py:690: error: Argument 1 to "send" of "GeneratorType" has incompatible type "None"; expected "Never"  [arg-type]
+ src/trio/_core/_tests/test_ki.py:690: error: Argument 1 to "send" of "CoroutineType" has incompatible type "None"; expected "Never"  [arg-type]

hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)
+ src/hydra_zen/third_party/beartype.py:125: error: Cannot assign to a method  [method-assign]

svcs (https://github.com/hynek/svcs)
+ src/svcs/_core.py:459: error: "Awaitable[Any]" not callable  [operator]
+ src/svcs/_core.py:492: error: "Awaitable[Any]" not callable  [operator]

psycopg (https://github.com/psycopg/psycopg)
+ psycopg_pool/psycopg_pool/_acompat.py:184: error: Incompatible types in assignment (expression has type "object", variable has type "T | Coroutine[Any, Any, T]")  [assignment]
+ psycopg_pool/psycopg_pool/_acompat.py:185: error: Incompatible return value type (got "T | Coroutine[Any, Any, T]", expected "T")  [return-value]

core (https://github.com/home-assistant/core)
+ homeassistant/core.py:596: error: Argument 1 to "HassJob" has incompatible type "Callable[[VarArg(*_Ts)], Any] | Coroutine[Any, Any, Any]"; expected "Callable[[VarArg(*_Ts)], Any]"  [arg-type]
+ homeassistant/core.py:661: error: Argument 1 to "HassJob" has incompatible type "Callable[[VarArg(*_Ts)], Coroutine[Any, Any, _R] | _R] | Coroutine[Any, Any, _R]"; expected "Callable[[VarArg(*_Ts)], Coroutine[Any, Any, _R] | _R]"  [arg-type]
+ homeassistant/core.py:661: error: Argument 1 to "HassJob" has incompatible type "Callable[[VarArg(*_Ts)], Coroutine[Any, Any, _R] | _R] | Coroutine[Any, Any, _R]"; expected "Callable[[VarArg(Any), KwArg(Any)], Coroutine[Any, Any, _R]]"  [arg-type]
+ homeassistant/core.py:988: error: Argument 1 to "HassJob" has incompatible type "Callable[[VarArg(*_Ts)], Coroutine[Any, Any, _R] | _R] | Coroutine[Any, Any, _R]"; expected "Callable[[VarArg(*_Ts)], Coroutine[Any, Any, _R] | _R]"  [arg-type]
+ homeassistant/core.py:988: error: Argument 1 to "HassJob" has incompatible type "Callable[[VarArg(*_Ts)], Coroutine[Any, Any, _R] | _R] | Coroutine[Any, Any, _R]"; expected "Callable[[VarArg(Any), KwArg(Any)], Coroutine[Any, Any, _R]]"  [arg-type]
+ homeassistant/helpers/entity_platform.py:448: error: Argument 1 to "create_eager_task" has incompatible type "Coroutine[object, Never, object]"; expected "Coroutine[Any, Any, None]"  [arg-type]

anyio (https://github.com/agronholm/anyio)
+ src/anyio/from_thread.py:267: error: Argument 1 to "set_result" of "Future" has incompatible type "object"; expected "T_Retval"  [arg-type]

prefect (https://github.com/PrefectHQ/prefect)
+ src/prefect/flow_engine.py:929: error: Incompatible types in assignment (expression has type "object", variable has type "Union[R, type[NotSet]]")  [assignment]
+ src/prefect/input/run_input.py:355: error: "object" has no attribute "value"  [attr-defined]
+ src/prefect/input/run_input.py:481: error: Incompatible return value type (got "object", expected "R")  [return-value]
+ src/prefect/input/run_input.py:500: error: "object" has no attribute "__iter__"; maybe "__dir__" or "__str__"? (not iterable)  [attr-defined]
+ src/prefect/input/run_input.py:502: error: Incompatible return value type (got "object", expected "list[FlowRunInput]")  [return-value]
+ src/prefect/input/run_input.py:586: error: "object" has no attribute "__iter__"; maybe "__dir__" or "__str__"? (not iterable)  [attr-defined]
+ src/prefect/input/run_input.py:588: error: Incompatible return value type (got "object", expected "list[FlowRunInput]")  [return-value]
- src/prefect/cli/deployment.py:292: error: R? has no attribute "model_dump"  [attr-defined]
+ src/prefect/concurrency/v1/sync.py:63: error: Argument 1 to "emit_concurrency_acquisition_events" has incompatible type "Coroutine[Any, Any, R?]"; expected "list[MinimalConcurrencyLimitResponse]"  [arg-type]
+ src/prefect/concurrency/v1/sync.py:69: error: Value of type "Coroutine[Any, Any, R?]" must be used  [unused-coroutine]
+ src/prefect/concurrency/v1/sync.py:69: note: Are you missing an await?
+ src/prefect/concurrency/v1/sync.py:72: error: Argument 1 to "emit_concurrency_release_events" has incompatible type "Coroutine[Any, Any, R?]"; expected "list[MinimalConcurrencyLimitResponse]"  [arg-type]

scrapy (https://github.com/scrapy/scrapy)
+ scrapy/utils/defer.py:560: error: Incompatible return value type (got "_T | Awaitable[_T]", expected "Awaitable[_T]")  [return-value]
+ scrapy/cmdline.py:77: error: Incompatible types in assignment (expression has type "object", target has type "ScrapyCommand")  [assignment]

strawberry (https://github.com/strawberry-graphql/strawberry)
+ strawberry/utils/await_maybe.py:13: error: Incompatible return value type (got "object", expected "T")  [return-value]
+ strawberry/types/object_type.py:306: error: Value of type variable "T" of "_inject_default_for_maybe_annotations" cannot be "object"  [type-var]
+ strawberry/types/object_type.py:307: error: Value of type variable "T" of "_wrap_dataclass" cannot be "object"  [type-var]
+ strawberry/relay/fields.py:344: error: Incompatible types in assignment (expression has type "object", variable has type "Awaitable[Any]")  [assignment]

discord.py (https://github.com/Rapptz/discord.py)
+ discord/utils.py:703: error: Incompatible return value type (got "object", expected "T")  [return-value]

aiohttp-devtools (https://github.com/aio-libs/aiohttp-devtools)
+ aiohttp_devtools/runserver/config.py:240: error: Incompatible types in assignment (expression has type "object", variable has type "Application | Awaitable[Application]")  [assignment]

pytest (https://github.com/pytest-dev/pytest)
+ testing/test_monkeypatch.py:420: error: Statement is unreachable  [unreachable]

steam.py (https://github.com/Gobot1234/steam.py)
+ steam/utils.py:894: error: Incompatible return value type (got "object", expected "_T")  [return-value]

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/web.py:307:15: error: Incompatible types in assignment (expression has type "object", variable has type "Application | Awaitable[Application]")  [assignment]

[AlexWaygood]

+ tests/pandas/test_dtypes.py:300: error: Item "object" of "object | Any" has no attribute "foo"  [union-attr]

This is inside a with pytest.raises() block; it's pretty clear that it's supposed to raise an exception

graphql-core (https://github.com/graphql-python/graphql-core)
+ tests/execution/test_middleware.py:274: error: "object" has no attribute "data"  [attr-defined]
+ tests/execution/test_middleware.py:276: error: "object" has no attribute "data"  [attr-defined]

What's happening here is that:

• the subscribe() call returns Awaitable[AsyncIterator[ExecutionResult] | ExecutionResult] | AsyncIterator[ExecutionResult] | ExecutionResult]
• the assert isasyncgen() call causes mypy to intersect that type with AsyncGeneratorType[object, Never]
• leading to a type of (AsyncGeneratorType[object, Never] & Awaitable[AsyncIterator[ExecutionResult] | ExecutionResult]) | (AsyncGeneratorType[object, Never] & AsyncIterator[ExecutionResult]) | (AsyncGeneratorType[object, Never] & ExecutionResult])

AsyncGeneratorType is @final, so I think the first and last elements of that union should both simplify to Never, meaning that type should be simplifiable to AsyncGeneratorType[object, Never] & AsyncIterator[ExecutionResult].

And that type should in turn be simplifable to AsyncGeneratorType[ExecutionResult].

All of which to is to say that the inferred type of await agen.__anext__() here should be ExecutionResult rather than object, but it seems like mypy does not have a good enough handling of intersection types to reach that conclusion.

[AlexWaygood]

trio (https://github.com/python-trio/trio)
+ src/trio/_core/_tests/test_ki.py:685: error: Argument 1 to "_consume_async_generator" has incompatible type "AsyncGeneratorType[object, Never]"; expected "AsyncGenerator[None, None]"  [arg-type]
+ src/trio/_core/_tests/test_ki.py:690: error: Argument 1 to "send" of "GeneratorType" has incompatible type "None"; expected "Never"  [arg-type]
+ src/trio/_core/_tests/test_ki.py:690: error: Argument 1 to "send" of "CoroutineType" has incompatible type "None"; expected "Never"  [arg-type]

These look like true positives. inspect.isasyncgen() returns True if an object is an async generator, but doesn't tell you anything about the yield type of that async generator. The _consume_async_generator function is annotated as requiring AsyncGenerator[None, None], and the inspect.isasyncgen() call doesn't seem like it should be sufficient to narrow result to that type here.

hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)
+ src/hydra_zen/third_party/beartype.py:125: error: Cannot assign to a method  [method-assign]

A true positive that mypy didn't catch previously because it narrowed the type to type[Any] inside the if inspect.isclass() branch.

svcs (https://github.com/hynek/svcs)
+ src/svcs/_core.py:459: error: "Awaitable[Any]" not callable  [operator]
+ src/svcs/_core.py:492: error: "Awaitable[Any]" not callable  [operator]

These again seem to be due to mypy having an imperfect understanding of intersections... the type of oc prior to line 447 is Callable[..., Any] | Awaitable[Any]. On line 456, mypy should be able to infer (Callable[..., Any] & ~Awaitable[object]) | (Awaitable[Any] & ~Awaitable[object]) -- since Awaitable is covariant, Awaitable[Any] can be understood as a subtype of Awaitable[object], meaning Awaitable[Any] & ~Awaitable[object] simplifies to Never, leading to an inferred type of Callable[..., Any] & ~Awaitable[object]. If it did that, it would conclude that oc is definitely callable on lines 459 and 492.

[AlexWaygood]

core (https://github.com/home-assistant/core)
+ homeassistant/core.py:596: error: Argument 1 to "HassJob" has incompatible type "Callable[[VarArg(*_Ts)], Any] | Coroutine[Any, Any, Any]"; expected "Callable[[VarArg(*_Ts)], Any]"  [arg-type]

Looks like another case of mypy not simplify SomeCovariantClass[Any] & ~SomeCovariantClass[object] to Never... at this point I'll abandon this; I think it's clear that this will cause too many false positives for mypy users.