Type-narrowing based on x in y

[Azureblade3808]

Converted from discussion (#9337).

---

We can now have following snippet pass type-checking -

from typing_extensions import assert_type

def foo(x: float = 0.0, y: list[int] = [0]):
    if x in y:
        _ = assert_type(x, "int")  # !!!

The type-narrowing of x can be a false negative, as the default value 0.0 of x is obviously not an instance of int.

On the other hand, type-narrowing based on x == L seems to work soundly -

from typing_extensions import Literal, assert_type

def foo(x0: float = 0.0, x1: int = 0, L: Literal[0] = 0):
    if x0 == L:
        assert_type(x0, "float")  # No type-narrowing.
    if x1 == L:
        assert_type(x1, "Literal[0]")  # Safe.

My suggestion is that type-narrowing based of x in y only take effect when the element type of y is a literal type (or maybe a union of literal types that shares a same runtime type) and type of x is the runtime type of y(or some related union types).

My expected behavior would be like following examples -

from typing_extensions import Literal, assert_type

def f0(x: int, y: list[Literal[0, 1]]):
    if x in y:
        _ = assert_type(x, "Literal[0, 1]")  # Narrowed.

def f1(x: Literal[-1, 0], y: list[Literal[0, 1]]):
    if x in y:
        _ = assert_type(x, "Literal[0]")  # Narrowed.

def f2(x: Literal[0, 1, 2], y: list[Literal[0, 1]]):
    if x in y:
        _ = assert_type(x, "Literal[0, 1]")  # Narrowed.

def f3(x: float, y: list[int]):
    if x in y:
        _ = assert_type(x, "float")  # Not narrowed, because `int` is not a literal type or a union of literal types.

def f4(x: float, y: list[Literal[0, 1]]):
    if x in y:
        _ = assert_type(x, "float")  # Not narrowed, because `float` is not `int`.

def f5(x: int, y: list[Literal[0, True]]):
    if x in y:
        _ = assert_type(x, "int")  # Not narrowed, because `Literal[0]` and `Literal[True]` don't share a same runtime type.

def f6(x: bool, y: list[Literal[0, 1]]):
    if x in y:  # Assuming this is allowed.
        _ = assert_type(x, "bool")  # Not narrowed, because `bool` is not `int`.

[tusharsadhwani]

May be related, but I'm trying to do something like this:

import typing


class MyType(typing.TypedDict):
    a: int
    b: typing.NotRequired[int]


MyEnum = typing.Literal["a", "b"]


def foo(t1: MyType, t2: MyType) -> None:
    for key in t1.keys():
        if key not in t2:
            continue

        key = typing.cast(MyEnum, key)
        print(t2[key])

is it possible to do something like this currently, where pyright doesn't raise an issue?

[JodhwaniMadhur]

I want to contribute to this even though I am here for the first time. I hope that is fine.

[tusharsadhwani]

This is probably a design change, which may or may not be wanted depending on the spec, what other checkers do etc., so I'd suggest wait for confirmation from maintainers. MyPy also doesn't do this right now for example, but it has been on the roadmap for a long time:

• Narrowing types to Literal using in syntax python/mypy#12535
• Infer type from in container check python/mypy#10977

[erictraut]

@tusharsadhwani, the behavior you're seeing is not related to this issue. The OP has identified a bug in the x in y type guard form. Your code uses something closer to the S in D type guard form, although it doesn't quite match that because S must be a string literal for this form to apply. In any event, these are different cases. See this documentation for a list of supported type guard forms. Pyright is working as intended in your case. If you have questions about this, feel free to open a new discussion topic.

[wyattscarpenter]

I've just run into this myself today, I think.

Code sample in pyright playground

from typing import assert_type

def g(a: int | None):
    if a is not None:
        assert_type(a, int) #works fine

def f(a: int | None):
    if a not in [None]:
        assert_type(a, int) # "assert_type" mismatch: expected "int" but received "int | None"  (reportAssertTypeFailure)

Edit 2025-02-11: word to the wise: the above code seems to work fine if you use a tuple instead of a list; that is, for example: if a not in (None,). Probably obvious to some people but if you found this message from googling, now you know :)

[erictraut]

@wyattscarpenter, that's not related to this issue. Type narrowing in the negative case in your code sample will not eliminate None from the union. That would require extra special-casing for None. I generally don't add special cases like this unless there's a strong signal from pyright users that it's a common use case. You'll need to find a workaround if you want this to type check without errors.

[wyattscarpenter]

Ah, thanks!

[Sxderp]

I believe I recently ran into this. From the playground.

from typing import Literal

def f(a: Literal['AND', '&', 'OR', '|']):
    if a in {'AND', '&'}:
        reveal_type(a) # Type of "a" is "Literal['AND', '&', 'OR', '|']" # Seems wrong
    if a in {'OR', '|'}:
        reveal_type(a) # Type of "a" is "Literal['AND', '&', 'OR', '|']" # Seem wrong

However, if I change from set to tuple it works.

from typing import Literal

def f(a: Literal['AND', '&', 'OR', '|']):
    if a in ('AND', '&'):
        reveal_type(a) # Type of "a" is "Literal['AND', '&']"
    if a in ('OR', '|'):
        reveal_type(a) # Type of "a" is "Literal['OR', '|']"

[erictraut]

@Sxderp, the behavior you're seeing here is expected. Pyright's inference behaviors for set, list and dict expressions do not retain literals because these types are mutable. Tuples are immutable, so pyright retains literals (in most cases). For details, refer to the pyright documentation here and here.

[Sxderp]

Maybe the documentation for type-guards should be clarified or a link should be added to the type-inference page?

https://github.com/microsoft/pyright/blob/main/docs/type-concepts-advanced.md

x in y or x not in y (where y is instance of list, set, frozenset, deque, tuple, dict, defaultdict, or OrderedDict)

This is the line for my source of confusion.

[erictraut]

@Azureblade3808, thanks for the bug report. I've updated pyright's logic to remove this unsoundness. I mostly followed your suggested approach — with a couple of extensions. In addition to literals, the logic also supports None, which acts like a literal in this case. I also retained support for the case where the element type of the container is a class object that is known to represent a specific class, as opposed to a set of class objects. This allows for safe narrowing in cases like this:

def func(x: type):
    if x in (str, int):
        reveal_type(x)  # type[str] | type[int]

This change could unfortunately cause some churn (i.e. necessitate some changes) for code bases that use pyright for type checking. Here are the mypy_primer results that show examples of how it affects various code bases.

This fix will be included in the next release.

[erictraut]

This is addressed in pyright 1.1.394.

[superlopuh]

Am I correct in understanding that type narrowing is still supported if the collection is immutable?

[erictraut]

Am I correct in understanding that type narrowing is still supported if the collection is immutable?

The mutability of the collection isn't a factor here.

Narrowing for in is safe if the second operand is an iterable with literal values. For example:

def func(x: list[Literal['hi']], y: str):
    if y in x:
        reveal_type(y) # Literal['hi']

This works despite the fact that list is a mutable collection type.