Pyright does not handle overload ambiguity

[BvB93]

Describe the bug
Pyright currently does not handle overload ambiguity, always defaulting to the first overload in such a situation (see below).

After its report in numpy/numpy#20265 I initially thought this might be a similar issue as python/mypy#11347,
but the fact that it seems to affect all overloaded callables makes it quite a bit more expansive.

To Reproduce

from typing import overload, Any, List

@overload
def func(a: List[int]) -> bool: ...
@overload
def func(b: List[str]) -> float: ...

list_int: List[int]
list_str: List[str]
list_any: List[Any]

# Expected behavior
reveal_type(func(list_int))  # info: Type of "func(list_int)" is "bool"
reveal_type(func(list_str))  # info: Type of "func(list_str)" is "float"

# Uhoh, it's defaulting to the first overload (quite the assumption)
reveal_type(func(list_any))  # info: Type of "func(list_any)" is "bool"

Expected behavior
The return of either Any or, depending on the particular overloads, an Any-parametrized generic.

VS Code extension or command-line

• Command line (plain pyright command)
• pyright 1.1.184

[erictraut]

The current behavior is intended. We've discussed possibly changing the behavior, but there are downsides to doing so.

There are arguably three reasonable behaviors when faced with an ambiguity due to an Any argument:

1. Pick the first matching overload (pyright's current behavior)
2. Return an Any or Unknown result (mypy's current behavior)
3. Return a type that is the union of all applicable overloads

If pyright were just a type checker (like mypy), the argument in favor of option 2 would be stronger. However, pyright is not just a type checker. It's also the basis for a language server (pylance). For every pylance user who enables type checking, there are another twenty who do not. The majority of users value completion suggestions over type checking results. An Any result is really bad for completion suggestions.

If I were to rank the above options based on type checking alone, I'd say that option 2 is better than 1 (but only slightly so), and option 3 is by far the worst.

If I were to rank the above options based on completion suggestions, I'd say that option 3 is best followed by 1, with option 2 by far the worst.

The current behavior (1) tries to strike the right balance.

For what it's worth, TypeScript also uses option 1.

You might argue that we should have different behaviors when type checking is disabled, but we're very reluctant to do this because it creates confusion and inconsistencies when the type information changes depending on whether type errors are reported. Every time we've done this in the past, we've regretted it, so we've established a principle that type evaluations must be the same in all modes.

We will likely reconsider this decision periodically as we receive additional feedback, so we appreciate the input. For now I think we're going to stick with the current behavior.

The workaround is to avoid the use of Any types in your code when passing arguments to overloaded functions because that can lead to these ambiguities.

[BvB93]

Hi Eric, thanks for the clarification.

However, pyright is not just a type checker. It's also the basis for a language server (pylance).

Interesting, i actually wasn't aware of pyright's larger role here.

In any case, avoiding Any seems to be doable for the specific example that prompted this issue, but I do forsee similar issues (for numpy, at least) in the future. This mostly boils down to two reasons:

• The annotations are rather overload-heavy, resulting in a large percentage of code that can potentially trigger these types of false positives. In part this is intrinsic to how the functions are designed, as there are usually a decent number of arguments that affect the shape and/or dtype of the output.
• Array dtype-inference can be rather challenging, introducing plenty of opportunity for introducing Any types (e.g. NDArray[Any]). It's not too bad when dtype-like objects can be parametrized w.r.t. numpy's scalar type (dtype=np.float64), but becomes more difficult when builtin scalar types (dtype=float) or string-literals (dtype="f8") are involved.

Lastly, this might be slightly off topic but could a proper dedicated unsafe union-type (python/typing#566) change the viability of behavior (3)?

[erictraut]

That's a good point. Yes, I think an AnyOf union type would change the viability of option 3.

However, AnyOf would also present its own challenges for overload matching when an argument expression evaluates to an AnyOf type.