Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempt to address memory leak described in #198 #214

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

dairiki
Copy link
Collaborator

@dairiki dairiki commented Sep 24, 2022

Here's a sort of rough and minimal crack at addressing #198.

The main change here is to avoid stack frames winding up in the LRU cache keys by passing the frames in a thread-local global.

I have a nagging feeling that there must be a cleaner way to address this, but this is the best I've come up with so far.

Suggestions welcome.

Bugs Fixed

Fixes #198
Fixes #229

@dairiki dairiki marked this pull request as draft September 25, 2022 18:00
@dairiki dairiki force-pushed the bug.198-memory-leak branch 2 times, most recently from c52cce0 to 766cd57 Compare September 25, 2022 21:56
@dairiki
Copy link
Collaborator Author

dairiki commented Jan 12, 2023

Other things that could maybe go into _SchemaContext:

  • The current dataclass field being processed (if any) e.g. for access to its metadata
  • The current base_schema for access to TYPE_MAPPING
  • In support generic dataclasses #172: generic_params_to_args which maps generic parameters to type arguments

Here we are more careful about which caller's locals we use to
resolve forward type references.  We want the callers locals
at decoration-time — not at decorator-construction time.

Consider:
```py
frozen_dataclass = marshmallow_dataclass.dataclass(frozen=True)

def f():
    @custom_dataclass
    class A:
        b: "B"

    @custom_dataclass
    class B:
        x: int
```

The locals we want in this case are the one from where the
custom_dataclass decorator is called, not from where
marshmallow_dataclass.dataclass is called.
When class_schema is called, it doesn't need the caller's whole stack
frame.  What it really wants is a `localns` to pass to
`typing.get_type_hints` to be used to resolve type references.

Here we add the ability to pass an explicit `localns` parameter to
`class_schema`.  We also add the ability to pass an explicit
`globalns`, because ... might as well — it might come in useful.
(Since we need these only to pass to `get_type_hints`, we might
as well match `get_type_hints` API as closely as possible.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant