fix(DataFrame): #799 to_dict

[cmp0xff]

• Closes DataFrame.to_dict(orient="index") should return a nested dictionary #799 (Replace xxxx with the Github issue number)
• Tests added: Please use assert_type() to assert the type of any return value

[cmp0xff]

I do now know how to get rid of the MutableMapping here...

[twoertwein]

Is into used for the outer mapping, the inner mapping, or both at runtime?

I assume we would need somthing like _T_MUTABLE_MAPPING[Hashable, dict[Hashable, Any]], if TypeVar would support [] (it doesn't) and if the inner mapping is always a dict (I don't know).

I'm not sure if we can annotate both the correct sub-class and also the keys/values.

[cmp0xff]

Is into used for the outer mapping, the inner mapping, or both at runtime?

Good point!

pd.DataFrame({("a", "b"): [[1], [2, 3]]}).to_dict("index", into=defaultdict(list))

the result is actually

defaultdict(<class 'list'>, {0: {('a', 'b'): [1]}, 1: {('a', 'b'): [2, 3]}})

Only the outer mapping is the type specified by into. The inner mapping are dicts.

[cmp0xff]

With afb5729, now if into is defaultdict or OrderedDict, the outer mapping is of the corresponding type, and the inner mapping is correctly dict.

If into is a more generic MutableMapping, however, I do not see a way to make the outer mapping covariant and fix the inner mapping to dict at the same time. I choose to make the outer mapping just the generic MutableMapping, and the inner mapping dict.

[cmp0xff]

• Hard-coded defaultdict, OrderedDict and dict
• Otherwise, to_dict now gives MutableMapping

[cmp0xff]

This is invalid, see https://github.com/pandas-dev/pandas/blob/v2.3.1/pandas/core/common.py#L416-L417

[cmp0xff]

Made covariant

[cmp0xff]

Covariant

[cmp0xff]

The old typing was wrong, now fixed.