Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically support functools.cached_property without requiring setting dict=True #623

Open
jcrist opened this issue Jan 2, 2024 · 2 comments

Comments

@jcrist
Copy link
Owner

jcrist commented Jan 2, 2024

Description

Since msgspec.Struct types are effectively slotted classes, using functools.cached_property with structs requires the user to also set dict=True:

import functools
import msgspec

class RightTriangle(msgspec.Struct, dict=True):    # <- dict=True required for cached_property
    a: float
    b: float
    @functools.cached_property
    def c(self):
        return (self.a ** 2 + self.b ** 2) ** 0.5

This is because the cpython implementation of cached_property requires a __dict__ exist on the instance. The error raised if you fail to set dict=True happens on property access, and since it's from functools not msgspec it doesn't indicate how to resolve the issue:

TypeError: No '__dict__' attribute on 'RightTriangle' instance to cache 'c' property.

As such, users have to read the docs (or ask) to know how to mix msgspec.Struct and functools.cached_property together. It would be nice to improve this situation. A few options:

  1. Automatically set dict=True if the class includes any cached_property attributes. This lets things "just work" for most users.
  2. Warn if a class includes cached_property attributes but doesn't set dict=True. This is like a less automagic form of 1.
  3. Rewrite cached_property attributes on msgspec.Struct types if dict=False to use an alternative slots-based mechanism, similar to what attrs added here.

Of these options I'm leaning towards 1 or maybe 3.

@jcrist
Copy link
Owner Author

jcrist commented Jan 5, 2024

Upon further investigation, I'm now leaning more towards 3.

  • Some of the preprocessing we'll need to do to make this work will also be needed to support Question: computed fields #573.
  • Providing builtin support for this will avoid the need to allocate a __dict__ per instance, reducing memory usage somewhat and slightly accelerating attribute access
  • Likewise, handling this ourselves lets us backport the 3.12 fix to avoid usage of an RLock per cached property, which can provide unnecessary lock contention in threaded applications.

The main downsides are:

  • More code on our end
  • Some magic. When used on a Struct type, the cached_property implementation itself won't ever actually run, it's just a decorator flag to tell the Struct implementation to treat it as a cached property using our own implementation. Since attrs is already doing the same, I feel better following their lead.

@provinzkraut
Copy link

provinzkraut commented Jan 5, 2024

  • Some magic. When used on a Struct type, the cached_property implementation itself won't ever actually run, it's just a decorator flag to tell the Struct implementation to treat it as a cached property using our own implementation. Since attrs is already doing the same, I feel better following their lead.

What about not using functools.cached_property, but instead provide e.g. msgspec.cached_property, which would work the same way you described, but would make it more explicit that this is not the same as the functools one? That would eliminate the magic aspect of it while providing the same functionality.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants