PEP 585 (built-in generics) tracker

[srittau]

This issue is supposed to track when we can start to use builtin generics (PEP 585) in typeshed. Support is needed in released versions of:

• mypy (Implement PEP 585 (Generic builtins and __future__.annotations) mypy#7907), mostly works except:
  • tuples containing ... (Unexpected '...' for homogeneous tuples using builtin tuple (PEP 585) mypy#9980)
  • generic type (type not generic when running with Python < 3.9 mypy#10303)
  • as base classes (Built-in generics as super classes mypy#10731)
• pytype
• pyright
• pyre
• PyCharm

Also, type aliases are a problem, although this might need TypeAlias support (#4913).

[srittau]

@rchen152 @mrkmndz Do pytype/pyre support PEP 585? If not, would you mind me creating a ticket in the respective projects that we can reference from here?

[sproshev]

Supported by PyCharm since 2020.2 https://youtrack.jetbrains.com/issue/PY-42418

[rchen152]

Internally, pytype has actually never distinguished between builtins.list and typing.List, etc. So we should already support this one.

[srittau]

Test case:

from __future__ import annotations

from typing import List

list1: List[str] = []
list2: list[str] = []

reveal_type(list1)
reveal_type(list2)

pyre returns:

src/pep585.py:8:0 Revealed type [-1]: Revealed type for `list1` is `List[str]`.
src/pep585.py:9:0 Revealed type [-1]: Revealed type for `list2` is `List[str]`.

🎉 This means all type checkers now support this feature and we can start using it in typeshed. 🎉

[Akuli]

Are we going to have a huge pull request to change List to list? Or is this not important enough for a huge pull request?

[srittau]

We should require future PRs to use built-in generics. I am personally in favor of a "big bang" PR after the reshuffling is done. Currently is a good time, since we have nearly no open PRs that could cause conflicts. But I am curious what the other maintainers think.

[Hultner]

Are we going to have a huge pull request to change List to list? Or is this not important enough for a huge pull request?

How would that turn out for people still using python 3.7 or 3.8? Or should they just pin an older typeshed version?

[JelleZijlstra]

I believe mypy actually does not yet support this in type aliases, only lists (x = list[int] doesn't work yet in mypy).

@Hultner this syntax should work in stubs even when it doesn't work at runtime yet. We should confirm that the type checkers actually allow this syntax in stubs when targeting older versions though.

[srittau]

@JelleZijlstra A quick test seems to work:

alias1 = "List[str]"
alias2 = List[str]
alias3 = "list[str]"
alias4 = list[str]

def foo(x: alias4):
    for i in x:
        reveal_type(i)

Correctly reveals str for both alias2 and alias4. Both alias1 and alias2 don't work, though.

[srittau]

Reopening because of python/mypy#9980.

[srittau]

The following does not work with mypy:

• tuples containing ... (Unexpected '...' for homogeneous tuples using builtin tuple (PEP 585) mypy#9980)
• generic type (type not generic when running with Python < 3.9 mypy#10303)

[srittau]

Base classes don't work either. python/mypy#10731

[JukkaL]

Mypy should be fixed on GitHub master. The fixes haven't been released yet, though.

[JelleZijlstra]

With the latest mypy release this should be mostly fixed, but as I recall there's still a few issues.

[AlexWaygood]

With the latest mypy release this should be mostly fixed, but as I recall there's still a few issues.

I am working on a PR for this.

[srittau]

Generic type didn't work yet when I tried to use it. Tuples and base classes are fixed.

[sobolevn]

@srittau do you have some opened issue to link me to? I can probably help with this one 🙂

Btw, this works for me on mypy@master:

def func(a: type[int]):
    reveal_type(a)  # N: Revealed type is "Type[builtins.int]"

func(int)  # ok
func(str)  # E: Argument 1 to "func" has incompatible type "Type[str]"; expected "Type[int]"

[srittau]

At least the following still fails with mypy 0.920. I didn't test mypy master:

class Foo(type[int]):  # invalid base class "type"
    pass

[srittau]

And here are a bunch of further errors: https://github.com/python/typeshed/runs/4604434037?check_suite_focus=true

#4820 for experiments.

[JukkaL]

Can you create mypy issues about any remaining issues and cc me? We can make a mypy point release with the fixes once everything works correctly (there should be no need to wait for a full release).

[srittau]

python/mypy#10303 seems to be still problematic, but the errors from #4820 are gone with mypy 0.930. 🥳

[AlexWaygood]

Many pytype errors were output by the CI when I tried to use imports from collections.abc everywhere 😕https://github.com/python/typeshed/runs/4629393978?check_suite_focus=true

[AlexWaygood]

Many pytype errors were output by the CI when I tried to use imports from collections.abc everywhere 😕https://github.com/python/typeshed/runs/4629393978?check_suite_focus=true

I have opened google/pytype#1096 for the pytype errors.

[srittau]

As of mypy 0.931 this is now fully supported.