Skip to content

Conversation

antonpirker
Copy link
Member

@antonpirker antonpirker commented Jun 5, 2025

Users reported that the DedupeIntegration can use up a lot of memory, because it keeps a full exception in memory for checking if it has seen this exception already.

Depending on the users code those exception objects can be big because they also include the traceback and local variables (which can be huge).

Idea is now to not save the whole exception but just a hash of the important parts of the exception to decide if we have seen this exception again.

fixes #3165
fixes #4327

Copy link

codecov bot commented Jun 5, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.58%. Comparing base (9001126) to head (d6ed7a8).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4446      +/-   ##
==========================================
- Coverage   84.60%   84.58%   -0.02%     
==========================================
  Files         158      158              
  Lines       16463    16463              
  Branches     2850     2850              
==========================================
- Hits        13928    13926       -2     
- Misses       1694     1696       +2     
  Partials      841      841              
Files with missing lines Coverage Δ
sentry_sdk/integrations/dedupe.py 87.50% <100.00%> (ø)

... and 1 file with indirect coverage changes

@antonpirker antonpirker changed the title Store a hash of the exc instead of full exc in DedupeIntegration Make DedupeIntegration more memory efficient. Jun 5, 2025
@sl0thentr0py
Copy link
Member

sl0thentr0py commented Jun 5, 2025

@antonpirker can't you just id(exc)?

@antonpirker
Copy link
Member Author

i thought about that too. but garbage collection moves objects around. so id(exc) could be different when we safe it and when we compare it...

@antonpirker
Copy link
Member Author

The id() of an object is guaranteed to never change during the life cycle of the object. So taking it.

@antonpirker antonpirker changed the title Make DedupeIntegration more memory efficient. fix(dedupe): Make DedupeIntegration more memory efficient. Sep 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Memory leak during exception logging Memory leak in AWS Fargate
2 participants