TypedDict missing many dict methods

[teepark]

mypy isn't recognizing methods of TypedDict.

• mypy 0.521
• mypy_extensions 0.3.0
• typeshed 0.0.1

from mypy_extensions import TypedDict
D = TypedDict('D', {'a': int})

d = {'a': 4}  # type: D
d.copy()
d.get('a')
d.clear()
d.setdefault('b', 7)
d.has_key('a')
d.items()
d.keys()
d.values()
d.iteritems()
d.iterkeys()
d.itervalues()
d.pop('b', None)
d.popitem()
d.update({'f': 11})
d.viewitems()
d.viewkeys()
d.viewvalues()

"""
$ mypy -2 example.py
example.py:7: error: "D" has no attribute "copy"
example.py:9: error: "D" has no attribute "clear"
example.py:10: error: "D" has no attribute "setdefault"
example.py:11: error: "D" has no attribute "has_key"
example.py:18: error: "D" has no attribute "pop"
example.py:19: error: "D" has no attribute "popitem"
example.py:20: error: "D" has no attribute "update"
example.py:21: error: "D" has no attribute "viewitems"
example.py:22: error: "D" has no attribute "viewkeys"
example.py:23: error: "D" has no attribute "viewvalues"
"""

[teepark]

@douglas-treadwell

[JelleZijlstra]

This is already tracked in #3518, but you're listing more methods. A lot of these shouldn't be hard to do, so feel free to submit a PR.

I wonder if there's also a way to inherit them from the dict stubs somehow.

[teepark]

I did find that ticket, but since he marked it as a feature and not a bug I figured the "special casing" refers to adding special behaviors to the listed methods (like clear is probably currently expected to return a Dict[Any, Any] but could return an instance of the current type).

[teepark]

It seems suspiciously like a mypy bug because typeshed should be forcing it to produce a Type[dict]. I'm not sure what else the stubs could be doing to help.

[ilevkivskyi]

I think the root of the problem is that the fallback for TypedDicts is Mapping and not Dict, and the former has much fewer methods. A hacky way to fix this would be to define a special subclass of Mapping that will define Dict methods that are safe to use for TypedDicts, and use it instead for fallbacks.

[JelleZijlstra]

Why isn't the fallback for TypedDict just Dict?

[ilevkivskyi]

IIRC this is because some methods of Dict are not safe since they can change dictionary items "en masse", while __getitem__/__setitem__ are special-cased to allow only certain keys/types. Now I am not sure it was the best decision (although more safe/conservative).

[JukkaL]

A user just asked about how to .copy() a TypedDict, and apparently the most reasonable approach is to use a cast, which is unfortunate.

Note that some of the methods aren't always safe, such as clear() for a total typed dict.

[JukkaL]

Here's a quick analysis of each missing method:

• copy: Simple
• clear: Not safe, due to structural subtyping (may delete a required key)
• setdefault: Need to provide type context for the second argument; require literal key
• has_key: Python 2 only, simple
• pop: Require literal key; only valid for non-required keys
• popitem: Not safe, due to structural subtyping (may delete a required key)
• update: Should provide TypedDict type context for the positional argument; needs special casing for keyword arguments
• viewitems: Python 2 only, similar to Python 3 items()
• viewkeys: Python 2 only, similar to Python 3 keys()
• viewvalues: Python 2 only, similar to Python 3 values()

[nguyenbathanh]

@JukkaL When can we release it? I didn't found it in latest version v0.650.

Thanks.

[emmatyping]

@nguyenbathanh releases are usually done every few weeks, but due to holidays I expect it will take a bit longer between 0.650 and 0.660. For now you can use the development branch if you really need this fix. (e.g. pip3 install git+https://github.com/python/mypy.git).

[nguyenbathanh]

@ethanhs Thanks for your information! I will use the Dev version.