smp: prefaulter: don't leave zombie worker threads #2679
Conversation
As explained in scylladb#2623 in detail, the prefaulter worker threads that have completed but not joined are left in a zombie state, which confuses gdb thread_local processing. As seastar relies on thread locals heavily, it becomes impossible to debug core dumps. Fix this by joining the threads after they complete. Use seastar::alien to ask the main reactor threads to join the completed threads when they are done, so it won't stall. Fixes scylladb#2623.
| work(ranges, page_size, huge_page_size_opt); | ||
| if (!--_active_threads) { | ||
| run_on(alien, 0, [this] () noexcept { join_threads(); }); |
There was a problem hiding this comment.
Is there no risk that this join_threads() call will happen after the memory_prefaulter is already destroyed?
There was a problem hiding this comment.
the prefaulter is nested under smp, and so are the reactors. So if the reactor is still running, the prefaulter still exists.
There was a problem hiding this comment.
Yes, but's it's not nice to have implicit lifetime assumptions like that.
Not that this case is worth caring about, though.
There was a problem hiding this comment.
In such cases it is unavoidable without strong compiler support. The setup is too hairy.
| work(ranges, page_size, huge_page_size_opt); | ||
| if (!--_active_threads) { | ||
| run_on(alien, 0, [this] () noexcept { join_threads(); }); |
There was a problem hiding this comment.
Yes, but's it's not nice to have implicit lifetime assumptions like that.
Not that this case is worth caring about, though.
As explained in #2623 in detail, the prefaulter worker threads that have completed but not joined are left in a zombie state, which confuses gdb thread_local processing. As seastar relies on thread locals heavily, it becomes impossible to debug core dumps.
Fix this by joining the threads after they complete. Use seastar::alien to ask the main reactor threads to join the completed threads when they are done, so it won't stall.
Fixes #2623.