Idea: "Incomplete" marker

[srittau]

When creating stubs, we encourage contributors to leave items unannotated instead of using Any. This often works, but there are cases where you have to fall back on Any, for example:

class Foo:
    def bar(self, some_arg, other_arg=None):
        self.a_field = something()
        if some_cond():
            return whatever()
        else:
            return None

In this case the best we can do is:

class Foo:
    a_field: Any
    def bar(self, some_arg, other_arg: Any | None = ...) -> Any | None: ...

In this case it's not clear if Any is used, because there is no better alternative or because this needs better annotations.

Idea: We could introduce an alias for Any in _typeshed, called something like Incomplete, Unknown (my current favorite), Something, Untyped, or Unannotated. This could clearly mark that this argument type still needs work:

class Foo:
    some_arg: Unknown
    def bar(self, some_arg, other_arg: Unknown | None = ...) -> Unknown | None: ...

This has a few advantages:

• Communicates that the type is not final and will be improved (and narrowed) in the future.
• Communicates that this particular annotation needs work.
• Gives us a better overview of the completion of stubs and prevents "lies". (In fact, an idea I have is to produce graphs that shows the completion percentage of stubs over time.)
• Gives us the ability to grep for Any to find areas where the type system could be improved or where newer type system features could be used.

[srittau]

Of course, stubgen should be taught this type if we would do something like this.

[JelleZijlstra]

I like the idea of being more explicit what is meant by Any. Pyright uses a similar concept internally (https://github.com/microsoft/pyright/blob/7bb059ecbab5c0c446d4dcf5376fc5ce8bd8cd26/docs/type-inference.md#type-inference), so maybe we can help Pyright by similarly separating Any and Unknown in typeshed.

[jakebailey]

Selfishly from the pyright POV, Unknown has a specific meaning (a version of Any with special tracing and associated errors), so I'd be a bit concerned that the two would be confused and harder to distinguish, especially in the editor where we're showing the type info in tooltips.

A half idea, but I wonder if Unknown in pyright could be renamed to unknown (lowercase) to distinguish it (if it came to it), matching TS (though their unknown behaves very differently; using a value of that type requires you to explicitly cast it), and sorta-kinda matches the changes from Dict to dict, List to list, and so on.

@erictraut for his thoughts.

[srittau]

Maybe we can come up with a better name. Unknown also means something different in Typescript.

[srittau]

Unidentified matches quite well, although that's a bit long.

[erictraut]

Why not simply omit type annotations when they are not yet known? That's what most stubs do today, and it works well. I don't see a need to make this explicit. Tools like pyright can indicate when these types are omitted.

The only downside is that you wouldn't be able to specify unions that include partially-known types, like in the example above, but I see this as an edge case, and I think it's fine to omit the entire annotation in this case.

[srittau]

The edge case is quite common and it's unfortunate to lose the partial type information in this case. It's both worse for users that get less type security, and worse for type stub authors that have more work when updating the stubs. Especially since this information is easily auto generated by stubgen.

[erictraut]

OK, that argument makes sense to me.

How would you propose defining Unknown (or Incomplete or whatever)? Would it be a type alias for Any (defined within typing.pyi?) that type checkers can optionally special-case if they want to treat it differently from Any?

[srittau]

Yes, as this is basically a replacement for Any | None (or similar constructs), this would just be Unidentified = Any in _typeshed or possibly typing_extensions. I wouldn't expect type checkers to special case this, except for possibly linting purposes. This is more meant as a marker for stub authors, and for us typeshed authors as an indication which stubs need work or which stubs are nearly complete and need just a little extra push.

[srittau]

Although thinking about it, it would make some sense to include this type into "incomplete type" warnings in type checkers.

[srittau]

_typeshed.Incomplete now exists and we can start using it in stdlib stubs. We should wait with using it in third-party stubs until the next mypy release at least.