Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

can't structure dataclass with PEP563 postponed annotations or forward references #195

Closed
anthrotype opened this issue Dec 7, 2021 · 6 comments

Comments

@anthrotype
Copy link

  • cattrs version: 1.8.0
  • Python version: 3.10.0
  • Operating System: macOS 12.0.1

Description

I would like to use cattrs with dataclasses (for this project I can't use attrs classes). I get the following errors when I try to structure a dataclass with circular, nested definitions that contain string literals to be post-evaluated (e.g. via typing.get_type_hints).
It seems that this currently only works with attrs classes, but not dataclasses.

What I Did

from __future__ import annotations
from typing import List
from dataclasses import dataclass, field
from cattr import Converter, GenConverter


@dataclass
class Node:
    children: List[Node] = field(default_factory=list)


tree = Node([Node([Node(), Node()]), Node()])

c = GenConverter()
data = c.unstructure(tree)
# Raises: NotAnAttrsClassError: <class '__main__.Node'> is not an attrs-decorated class.
c.structure(data, Node)


c = Converter()
data = c.unstructure(tree)
# Raises: StructureHandlerNotFoundError: Unsupported type: List[Node]. Register a structure hook for it.
c.structure(data, Node)

If I remove from __future__ import annotations and type the nested children: List["Node"] I get this other error:

StructureHandlerNotFoundError: Unsupported type: ForwardRef('Node'). Register a structure hook for it.
@anthrotype
Copy link
Author

Note the same happens with the newly released cattrs 1.9.0

@Tinche
Copy link
Member

Tinche commented Dec 7, 2021

Hm, for attrs classes I use attr.resolve_types, and that doesn't exist for dataclasses. I can try using typing.get_type_hints directly, but that won't cover some of the use cases (like when you need to resolve them yourself, in a special way).

@anthrotype
Copy link
Author

using typing.get_type_hints directly

yeah, I suppose that's the way to go. You can have a resolve_types in _compat that delegates to attrs.resolve_types for attrs classes or does the same thing for dataclasses.

won't cover some of the use cases (like when you need to resolve them yourself, in a special way)

not sure what you mean by that

@Tinche
Copy link
Member

Tinche commented Dec 7, 2021

So for example cattrs doing resolve_types on a class can fail. That's why resolve_types takes some arguments so you can help it before you actually use the class.

Also resolve_types is a stateful change - it updates the fields. get_type_hints just returns the annotations. It'll probably be enough to start though.

@Tinche
Copy link
Member

Tinche commented Dec 12, 2021

I just pushed support for this to the main branch, feel free to test it out.

@anthrotype
Copy link
Author

Thanks, it works now!

mergify bot pushed a commit to aws/jsii that referenced this issue Jan 6, 2022
…1 in /packages/@jsii/python-runtime (#3315)

Updates the requirements on [cattrs](https://github.com/python-attrs/cattrs) to permit the latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/python-attrs/cattrs/blob/main/HISTORY.rst">cattrs's changelog</a>.</em></p>
<blockquote>
<h2>1.10.0 (2022-01-04)</h2>
<ul>
<li>Add PEP 563 (string annotations) support for dataclasses.
(<code>[#195](python-attrs/cattrs#195) &lt;https://github.com/python-attrs/cattrs/issues/195&gt;</code>_)</li>
<li>Fix handling of dictionaries with string Enum keys for bson, orjson, and tomlkit.</li>
<li>Rename the <code>cattr.gen.make_dict_unstructure_fn.omit_if_default</code> parameter to <code>_cattrs_omit_if_default</code>, for consistency. The <code>omit_if_default</code> parameters to <code>GenConverter</code> and <code>override</code> are unchanged.</li>
<li>Following the changes in <code>attrs</code> 21.3.0, add a <code>cattrs</code> package mirroring the existing <code>cattr</code> package. Both package names may be used as desired, and the <code>cattr</code> package isn't going away.</li>
</ul>
<h2>1.9.0 (2021-12-06)</h2>
<ul>
<li>Python 3.10 support, including support for the new union syntax (<code>A | B</code> vs <code>Union[A, B]</code>).</li>
<li>The <code>GenConverter</code> can now properly structure generic classes with generic collection fields.
(<code>[#149](python-attrs/cattrs#149) &lt;https://github.com/python-attrs/cattrs/issues/149&gt;</code>_)</li>
<li><code>omit=True</code> now also affects generated structuring functions.
(<code>[#166](python-attrs/cattrs#166) &lt;https://github.com/python-attrs/cattrs/issues/166&gt;</code>_)</li>
<li><code>cattr.gen.{make_dict_structure_fn, make_dict_unstructure_fn}</code> now resolve type annotations automatically when PEP 563 is used.
(<code>[#169](python-attrs/cattrs#169) &lt;https://github.com/python-attrs/cattrs/issues/169&gt;</code>_)</li>
<li>Protocols are now unstructured as their runtime types.
(<code>[#177](python-attrs/cattrs#177) &lt;https://github.com/python-attrs/cattrs/pull/177&gt;</code>_)</li>
<li>Fix an issue generating structuring functions with renaming and <code>_cattrs_forbid_extra_keys=True</code>.
(<code>[#190](python-attrs/cattrs#190) &lt;https://github.com/python-attrs/cattrs/issues/190&gt;</code>_)</li>
</ul>
<h2>1.8.0 (2021-08-13)</h2>
<ul>
<li>Fix <code>GenConverter</code> mapping structuring for unannotated dicts on Python 3.8.
(<code>[#151](python-attrs/cattrs#151) &lt;https://github.com/python-attrs/cattrs/issues/151&gt;</code>_)</li>
<li>The source code for generated un/structuring functions is stored in the <code>linecache</code> cache, which enables more informative stack traces when un/structuring errors happen using the <code>GenConverter</code>. This behavior can optionally be disabled to save memory.</li>
<li>Support using the attr converter callback during structure.
By default, this is a method of last resort, but it can be elevated to the default by setting <code>prefer_attrib_converters=True</code> on <code>Converter</code> or <code>GenConverter</code>.
(<code>[#138](python-attrs/cattrs#138) &lt;https://github.com/python-attrs/cattrs/issues/138&gt;</code>_)</li>
<li>Fix structuring recursive classes.
(<code>[#159](python-attrs/cattrs#159) &lt;https://github.com/python-attrs/cattrs/issues/159&gt;</code>_)</li>
<li>Converters now support un/structuring hook factories. This is the most powerful and complex venue for customizing un/structuring. This had previously been an internal feature.</li>
<li>The <code>Common Usage Examples &lt;https://cattrs.readthedocs.io/en/latest/usage.html#using-factory-hooks&gt;</code>_ documentation page now has a section on advanced hook factory usage.</li>
<li><code>cattr.override</code> now supports the <code>omit</code> parameter, which makes <code>cattrs</code> skip the atribute entirely when unstructuring.</li>
<li>The <code>cattr.preconf.bson</code> module is now tested against the <code>bson</code> module bundled with the <code>pymongo</code> package, because that package is much more popular than the standalone PyPI <code>bson</code> package.</li>
</ul>
<h2>1.7.1 (2021-05-28)</h2>
<ul>
<li><code>Literal</code> s are not supported on Python 3.9.0 (supported on 3.9.1 and later), so we skip importing them there.
(<code>[#150](python-attrs/cattrs#150) &lt;https://github.com/python-attrs/cattrs/issues/150&gt;</code>_)</li>
</ul>
<h2>1.7.0 (2021-05-26)</h2>
<ul>
<li><code>cattr.global_converter</code> (which provides <code>cattr.unstructure</code>, <code>cattr.structure</code> etc.) is now an instance of <code>cattr.GenConverter</code>.</li>
<li><code>Literal</code> s are now supported and validated when structuring.</li>
<li>Fix dependency metadata information for <code>attrs</code>.
(<code>[#147](python-attrs/cattrs#147) &lt;https://github.com/python-attrs/cattrs/issues/147&gt;</code>_)</li>
<li>Fix <code>GenConverter</code> mapping structuring for unannotated dicts.
(<code>[#148](python-attrs/cattrs#148) &lt;https://github.com/python-attrs/cattrs/issues/148&gt;</code>_)</li>
</ul>

</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/python-attrs/cattrs/commit/7d3a6ba5e0df942391349e332cb87f6871088015"><code>7d3a6ba</code></a> Bump to 1.10.0</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/22b24c28fbeb2b8ca90d568dc4939bdc98ec5902"><code>22b24c2</code></a> Tin/import cattrs (<a href="https://github-redirect.dependabot.com/python-attrs/cattrs/issues/203">#203</a>)</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/a0e56f43f061c43814d6f938833d1c325ed61525"><code>a0e56f4</code></a> Fix test with 32-bit time_t</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/bc9432e606177fe10aa3d2d11e715970c92526be"><code>bc9432e</code></a> Documentation tweaks</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/6260c58aa185200a08c76ecc99a941a26a93eeb8"><code>6260c58</code></a> Rename gen.make_dict_unstructure_fn.omit_if_default</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/bb4383c2d97aae0e8a01db64f142d07350861a17"><code>bb4383c</code></a> Remove walrus</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/81d7756541129a73b82f076b860d61c550296666"><code>81d7756</code></a> Clean up test</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/de16200c3d02d259d04960ab466f49fcf8fa47a4"><code>de16200</code></a> Fix preconf string Enum keys</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/077c9ea8521372f706346f1f39101db716ca3089"><code>077c9ea</code></a> CI tweak</li>
<li><a href="https://github.com/python-attrs/cattrs/commit/3ad74d4456c19598e7278deef62e3af85eabc99d"><code>3ad74d4</code></a> setup.cfg B gone</li>
<li>Additional commits viewable in <a href="https://github.com/python-attrs/cattrs/compare/v1.8.0...v1.10.0">compare view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
mergify bot pushed a commit to aws/jsii that referenced this issue Apr 4, 2022
…2 in /packages/@jsii/python-runtime (#3470)

Updates the requirements on [cattrs](https://github.com/python-attrs/cattrs) to permit the latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/python-attrs/cattrs/blob/main/HISTORY.rst">cattrs's changelog</a>.</em></p>
<blockquote>
<h2>22.1.0 (2022-04-03)</h2>
<ul>
<li>cattrs now uses the CalVer versioning convention.</li>
<li>cattrs now has a detailed validation mode, which is enabled by default. Learn more <code>here &lt;https://cattrs.readthedocs.io/en/latest/validation.html&gt;</code>_.
The old behavior can be restored by creating the converter with <code>detailed_validation=False</code>.</li>
<li><code>attrs</code> and dataclass structuring is now ~25% faster.</li>
<li>Fix an issue structuring bare <code>typing.List</code> s on Pythons lower than 3.9.
(<code>[#209](python-attrs/cattrs#209) &lt;https://github.com/python-attrs/cattrs/issues/209&gt;</code>_)</li>
<li>Fix structuring of non-parametrized containers like <code>list/dict/...</code> on Pythons lower than 3.9.
(<code>[#218](python-attrs/cattrs#218) &lt;https://github.com/python-attrs/cattrs/issues/218&gt;</code>_)</li>
<li>Fix structuring bare <code>typing.Tuple</code> on Pythons lower than 3.9.
(<code>[#218](python-attrs/cattrs#218) &lt;https://github.com/python-attrs/cattrs/issues/218&gt;</code>_)</li>
<li>Fix a wrong <code>AttributeError</code> of an missing <code>__parameters__</code> attribute. This could happen
when inheriting certain generic classes – for example <code>typing.*</code> classes are affected.
(<code>[#217](python-attrs/cattrs#217) &lt;https://github.com/python-attrs/cattrs/issues/217&gt;</code>_)</li>
<li>Fix structuring of <code>enum.Enum</code> instances in <code>typing.Literal</code> types.
(<code>[#231](python-attrs/cattrs#231) &lt;https://github.com/python-attrs/cattrs/pull/231&gt;</code>_)</li>
<li>Fix unstructuring all tuples - unannotated, variable-length, homogenous and heterogenous - to <code>list</code>.
(<code>[#226](python-attrs/cattrs#226) &lt;https://github.com/python-attrs/cattrs/issues/226&gt;</code>_)</li>
<li>For <code>forbid_extra_keys</code> raise custom <code>ForbiddenExtraKeyError</code> instead of generic <code>Exception</code>.
(<code>[#225](python-attrs/cattrs#225) &lt;https://github.com/python-attrs/cattrs/pull/225&gt;</code>_)</li>
<li>All preconf converters now support <code>loads</code> and <code>dumps</code> directly. See an example <code>here &lt;https://cattrs.readthedocs.io/en/latest/preconf.html&gt;</code>_.</li>
<li>Fix mappings with byte keys for the orjson, bson and tomlkit converters.
(<code>[#241](python-attrs/cattrs#241) &lt;https://github.com/python-attrs/cattrs/issues/241&gt;</code>_)</li>
</ul>
<h2>1.10.0 (2022-01-04)</h2>
<ul>
<li>Add PEP 563 (string annotations) support for dataclasses.
(<code>[#195](python-attrs/cattrs#195) &lt;https://github.com/python-attrs/cattrs/issues/195&gt;</code>_)</li>
<li>Fix handling of dictionaries with string Enum keys for bson, orjson, and tomlkit.</li>
<li>Rename the <code>cattr.gen.make_dict_unstructure_fn.omit_if_default</code> parameter to <code>_cattrs_omit_if_default</code>, for consistency. The <code>omit_if_default</code> parameters to <code>GenConverter</code> and <code>override</code> are unchanged.</li>
<li>Following the changes in <code>attrs</code> 21.3.0, add a <code>cattrs</code> package mirroring the existing <code>cattr</code> package. Both package names may be used as desired, and the <code>cattr</code> package isn't going away.</li>
</ul>
<h2>1.9.0 (2021-12-06)</h2>
<ul>
<li>Python 3.10 support, including support for the new union syntax (<code>A | B</code> vs <code>Union[A, B]</code>).</li>
<li>The <code>GenConverter</code> can now properly structure generic classes with generic collection fields.
(<code>[#149](python-attrs/cattrs#149) &lt;https://github.com/python-attrs/cattrs/issues/149&gt;</code>_)</li>
<li><code>omit=True</code> now also affects generated structuring functions.
(<code>[#166](python-attrs/cattrs#166) &lt;https://github.com/python-attrs/cattrs/issues/166&gt;</code>_)</li>
<li><code>cattr.gen.{make_dict_structure_fn, make_dict_unstructure_fn}</code> now resolve type annotations automatically when PEP 563 is used.
(<code>[#169](python-attrs/cattrs#169) &lt;https://github.com/python-attrs/cattrs/issues/169&gt;</code>_)</li>
<li>Protocols are now unstructured as their runtime types.
(<code>[#177](python-attrs/cattrs#177) &lt;https://github.com/python-attrs/cattrs/pull/177&gt;</code>_)</li>
<li>Fix an issue generating structuring functions with renaming and <code>_cattrs_forbid_extra_keys=True</code>.
(<code>[#190](python-attrs/cattrs#190) &lt;https://github.com/python-attrs/cattrs/issues/190&gt;</code>_)</li>
</ul>
<h2>1.8.0 (2021-08-13)</h2>
<ul>
<li>Fix <code>GenConverter</code> mapping structuring for unannotated dicts on Python 3.8.</li>
</ul>

</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a href="https://github.com/python-attrs/cattrs/commits">compare view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants