Support narrowing using `TypeGuard`

[dhruvmanila]

Add support for narrowing on TypeGuard and TypeIs:

• TypeIs https://github.com/astral-sh/ruff/labels/help%20wanted
• TypeGuard

This will in turn support builtins.callable:

Add support for narrowing on builtins.callable, requires TypeIs support.

builtins.pyi:

def callable(obj: object, /) -> TypeIs[Callable[..., object]]: ...

Otherwise, it causes a few false positives, like:

def _(a: Any | None) -> None:
    # /private/tmp/mypy_primer/projects/bidict/bidict/_iter.py:50:34: Object of type `None` is not callable
    if callable(a):
        a()  # red-knot: Object of type `None` is not callable [lint:call-non-callable]
        reveal_type(a)  # revealed_type: Any | None

[AlexWaygood]

Hmm, won't this fall out naturally from TypeIs support? I'm not sure we'll need to add any special handling for builtins.callable() once we understand TypeIs.

It doesn't look like we have a dedicated issue for TypeGuard and TypeIs support right now, but we should maybe make one. It was one of the items listed in astral-sh/ruff#13694, but that issue is now closed

[dhruvmanila]

Hmm, won't this fall out naturally from TypeIs support? I'm not sure we'll need to add any special handling for builtins.callable() once we understand TypeIs.

Yeah, you're right. There's astral-sh/ruff#16314 which is in draft.

[carljm]

TypeIs at least should not be too difficult to build on top of our existing narrowing support; tagging "help wanted" for that.

TypeGuard may be somewhat more challenging, since it can fully override the previous type (like a cast), meaning we will have to extend our narrowing to support forms of narrowing that don't take the form of intersections. (Probably by changing narrowing constraints from a type to an enum where one variant is "a type to intersect with", another variant is "a typeguard type to override with", etc.)

[InSyncWithFoo]

While working on astral-sh/ruff#16314, I encountered three major problems, all listed at this comment. I'm willing to resume my work if someone could give a few hints on how to handle them, especially the second.

As a note to myself, here's a small edge case example to add to the tests:

def is_int(v: int | str) -> TypeIs[int]: ...

class C:
    a: int | str
    
    def __init__(self, a: int | str):
        self.a = a
        self.a_is_int = is_int(a)

c = C('')

reveal_type(c.a)         # int | str
reveal_type(c.a_is_int)  # TypeIs[int]
                         # Or `TypeIs[a, int]`? `TypeIs[c.a, int]`? `TypeIs[C.a, int]`?

if c.a_is_int:
    reveal_type(c.a)     # ??
else:
    reveal_type(c.a)     # ??

[carljm]

Ah, I wasn't even aware of that PR; I don't get notifications on draft PRs unless pinged. It looks like point 3 there is the same one I mentioned above: TypeGuard will require some significant additions to the current implementation of narrowing in red-knot. The differences in implementation are major enough that I would definitely suggest tackling them in separate PRs. I will try to find some time soon to review that PR and see if I can understand the other two points mentioned.

[carljm]

As a note to myself, here's a small edge case example to add to the tests

FWIW, neither pyright nor mypy support this case, and I don't think we need to either. It isn't clear in the spec, but I don't see any evidence in the spec or in the other typechecker implementations to suggest that type guards should have any effect on expressions outside the scope where the type guard is called.

[InSyncWithFoo]

@carljm How about this, then?

def f(a: int | str):
	class C:
		def __init__(self):
			self.a_is_int = is_int(a)

	reveal_type(C().a_is_int)  # TypeIs[a, int]

	if C().a_is_int:
		reveal_type(a)  # int?

In case it isn't clear, this is just a contrived example to demonstrate the second problem.

Neither Mypy nor Pyright supports this pattern, for presumably a very good reason. Something noteworthy is that the two type checkers differ in how they infer .a_is_int: Mypy considers it bool, while Pyright reveals TypeIs[int] (but the attribute isn't usable as a narrowing condition).