Infering return type as a union of all returns in type-checked functions

[Avasam]

Feature

Note: This is different than #4409 and #6646 which aim to change what is considered "annotated" and which functions are type-checked in the first place.

Mypy currently only type-check a function if its considered "annotated", ie: all parameters are annotated, or the return type is present (required for methods w/o parameters). Whilst mypy obviously can't infer a return type if it doesn't type-check the content of a function, I feel like methods that are checked could have their return type inferred as a union of whatever mypy think the type of the returns are.

Here's a very simple example comparing mypy and pyright/pylance

(same result in CLI)

Pitch

The idea would be to reduce clutter, and reduce the risk of hiding a useful true return type behind a more vague type.

Maybe there's a performance concern for libraries? (mypy having to read more code instead of stopping at a return annotation) How much is it? How much does caching help?

[sterliakov]

This would break PEP484 compliance of mypy. To quote,

For a checked function, the default annotation for arguments and for the return type is Any. An exception is the first argument of instance and class methods. If it is not annotated, then it is assumed to have the type of the containing class for instance methods, and a type object type corresponding to the containing class object for class methods.

The same from typing documentation:

Furthermore, all functions without a return type or parameter types will implicitly default to using Any:

[Avasam]

Hmm, that's kinda old and imo it doesn't make sense to force a type checker to infer any (as that would make all inference-based checking non-compliant, see pytype and pyright). I don't see anything about this in https://typing.readthedocs.io/en/latest/spec/index.html , maybe that's a discussion for the modern typing spec to rectify.

[sterliakov]

I'm actually quite against doing that, I love that all annotations are "opt in", and I do not have subtle errors and bunch of type-ignore comments when interfacing with some legacy stuff or just poorly typed codebases.

And it's right there in typing spec (here):

It is recommended but not required that checked functions have annotations for all arguments and the return type. For a checked function, the default annotation for arguments and for the return type is Any. An exception is the first argument of instance and class methods. If it is not annotated, then it is assumed to have the type of the containing class for instance methods, and a type object type corresponding to the containing class object for class methods. For example, in class A the first argument of an instance method has the implicit type A. In a class method, the precise type of the first argument cannot be represented using the available type notation.

(that's copied verbatim from current docs)

[sterliakov]

Either way, this decision cannot be made by a type checker. If you think that typing spec should be amended, then typing-sig or https://github.com/python/typing/issues is the right place to discuss that.

[Avasam]

Thanks for pointing to me where that's addressed in the spec.

It is recommended but not required that checked functions have annotations for all arguments and the return type. For a checked function, the default annotation for arguments and for the return type is Any.
[...]
Type checkers are expected to attempt to infer as much information as necessary.

So default to Any, but if you can infer it that's fine to do so. That keeps pytype, pyright, this request and #4409 compliant. So I think they can be discussed on their own merits.

If you think that typing spec should be amended

Now that I found (you showed me) the section about it, I think the spec is fine as-is. It offers guidelines, a default behaviour, and possibility for checkers to do more.

---

when interfacing with some legacy stuff or just poorly typed codebases.

When you say "interfacing with legacy stuff", do you mean libraries that are not py.typed? Which brings a clarification: this would not be run on untyped external packages (or could be opt-in like pyright's useLibraryCodeForTypes, but whether this could be allowed as an additional possibility is out of scope of my request)

About poorly typed codebases:

• I can see the argument for libraries that declare themselves as py.typed, but are missing most return types as a way to hide incorrect typing.
• As for poorly typed user code (either badly typed, or untyped "legacy stuff"). I'd personally this would help improve one's typing, finding more issues. Or force being explicit about typing that's too dynamic (ie: explicit Any return type). Note that my suggestion also only applies for functions whose parameters are annotated and that mypy already type-checks.