Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix refleak tracking in free-threaded build #115184

Closed
colesbury opened this issue Feb 8, 2024 · 0 comments
Closed

Fix refleak tracking in free-threaded build #115184

colesbury opened this issue Feb 8, 2024 · 0 comments
Assignees
@colesbury
Labels
3.13 bugs and security fixes topic-free-threading type-bug An unexpected behavior, bug, or error

Comments

@colesbury
Copy link
Contributor

colesbury commented Feb 8, 2024 •
edited by bedevere-app bot
Loading

Bug report

There are a few bugs with refleak tracking in the free-threaded build uncovered in #114824:

  • We should account for blocks in abandoned segments
  • We should stop-the-world before traversing mimalloc heaps in case there are still other threads running
  • The mimalloc heap and segment traversal must call _mi_page_free_collect(page, true); to properly account for blocks that were deferred freed by other threads. We need to call this earlier than we currently do (before computing page stats).
  • _Py_DecRefSharedDebug was missing a _Py_IncRefTotal

Linked PRs
@colesbury colesbury added type-bug An unexpected behavior, bug, or error 3.13 bugs and security fixes topic-free-threading labels Feb 8, 2024
@colesbury colesbury self-assigned this Feb 8, 2024
colesbury added a commit to colesbury/cpython that referenced this issue Feb 8, 2024
@colesbury
pythongh-115184: Fix refleak tracking issues in free-threaded build
1b17640 
- We did not account for abandoned segments, which could miss blocks in
  multithreaded runs.
- The mimalloc heap traversal needed to call "_mi_page_free_collect"
  earlier in order to get an accurate count of blocks in use.
- `_Py_DecRefSharedDebug` was missing a `_Py_IncRefTotal`, but this was
  mostly offset by a missing accounting in `_Py_ExplicitMergeRefcount`.
- get_num_global_allocated_blocks should pause other threads to ensure
  that traversing the mimalloc heaps is safe.
@bedevere-app bedevere-app bot mentioned this issue Feb 8, 2024
Merged
colesbury added a commit that referenced this issue Feb 9, 2024
@colesbury
gh-115184: Fix refleak tracking issues in free-threaded build (#115188)
31633f4 
Fixes a few issues related to refleak tracking in the free-threaded build:

- Count blocks in abandoned segments
- Call `_mi_page_free_collect` earlier during heap traversal in order to get an accurate count of blocks in use.
- Add missing refcount tracking in `_Py_DecRefSharedDebug` and `_Py_ExplicitMergeRefcount`.
- Pause threads in  `get_num_global_allocated_blocks` to ensure that traversing the mimalloc heaps is safe.
fsc-eriker pushed a commit to fsc-eriker/cpython that referenced this issue Feb 14, 2024
@colesbury @fsc-eriker
pythongh-115184: Fix refleak tracking issues in free-threaded build (p…
000f444 
…ython#115188)

Fixes a few issues related to refleak tracking in the free-threaded build:

- Count blocks in abandoned segments
- Call `_mi_page_free_collect` earlier during heap traversal in order to get an accurate count of blocks in use.
- Add missing refcount tracking in `_Py_DecRefSharedDebug` and `_Py_ExplicitMergeRefcount`.
- Pause threads in  `get_num_global_allocated_blocks` to ensure that traversing the mimalloc heaps is safe.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@colesbury colesbury

Labels
3.13 bugs and security fixes topic-free-threading type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@colesbury