Skip to content

Fix DbgCtl use-after-free shutdown crash via leaky singleton #12777

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
bneradt merged 1 commit into from
Jan 6, 2026
Merged

Fix DbgCtl use-after-free shutdown crash via leaky singleton #12777

bneradt merged 1 commit into from
Jan 6, 2026

Conversation

@bneradt
Copy link
Contributor

@bneradt bneradt commented Dec 31, 2025

The DbgCtl registry was experiencing use-after-free crashes during program shutdown, particularly visible in CI regression tests. The root cause was the undefined destruction order of static objects in C++.

Problem:

  • DbgCtl objects can be static or thread-local with lifetimes spanning program execution
  • When one compilation unit's DbgCtl destructs and triggers registry cleanup, other compilation units may still have DbgCtl objects with pointers into that registry
  • Thread exit order is also unpredictable, causing similar issues when thread-local DbgCtl objects destruct

This implements the "leaky singleton" pattern where the registry is:

  • Created on first use
  • Never destroyed (destructor is now = default)
  • Memory (~20KB) reclaimed by OS at process exit

Fixes: #12776

@bneradt bneradt added this to the 10.2.0 milestone Dec 31, 2025
@bneradt bneradt self-assigned this Dec 31, 2025
@bneradt bneradt force-pushed the fix_centos_dbg_ctl_destruction_crash branch 26 times, most recently from 943943e to f85493c Compare January 5, 2026 19:26
@bneradt bneradt force-pushed the fix_centos_dbg_ctl_destruction_crash branch 3 times, most recently from ddc9716 to eb15f64 Compare January 5, 2026 22:18
@bryancall bryancall requested a review from serrislew January 5, 2026 22:46
@bneradt bneradt force-pushed the fix_centos_dbg_ctl_destruction_crash branch 2 times, most recently from 324ff8a to 34d79a1 Compare January 6, 2026 16:10
@bneradt
Fix DbgCtl use-after-free shutdown crash via leaky singleton
2c68360 
The DbgCtl registry was experiencing use-after-free crashes during program
shutdown, particularly visible in CI regression tests. The root cause
was the undefined destruction order of static objects in C++.

Problem:
- DbgCtl objects can be static or thread-local with lifetimes spanning program
  execution
- When one compilation unit's DbgCtl destructs and triggers registry cleanup,
  other compilation units may still have DbgCtl objects with pointers into
  that registry
- Thread exit order is also unpredictable, causing similar issues when
  thread-local DbgCtl objects destruct

This implements the "leaky singleton" pattern where the registry is:
- Created on first use
- Never destroyed (destructor is now = default)
- Memory (~20KB) reclaimed by OS at process exit

Fixes: apache#12776
@bneradt bneradt force-pushed the fix_centos_dbg_ctl_destruction_crash branch 2 times, most recently from 623ec71 to 2c68360 Compare January 6, 2026 17:25
@bneradt bneradt merged commit 11e99cb into apache:master Jan 6, 2026
15 checks passed
@bneradt bneradt deleted the fix_centos_dbg_ctl_destruction_crash branch January 6, 2026 20:16
bneradt added a commit to bneradt/trafficserver that referenced this pull request Jan 9, 2026
@bneradt
Restore DbgCtl::_rm_reference() for ABI backward compatibility
03c24b9 
PR apache#12777 introduced a leaky singleton pattern for DbgCtl to fix
use-after-free crashes during shutdown. However, removing the
_rm_reference() method entirely broke ABI compatibility with existing
plugins that were compiled against the old header, where the destructor
called this method. This commit restores _rm_reference() as a no-op stub,
allowing old plugins to load successfully while maintaining the leaky
singleton behavior.
bneradt added a commit to bneradt/trafficserver that referenced this pull request Jan 9, 2026
@bneradt
Restore DbgCtl::_rm_reference() for ABI compatibility
f6cf6fa 
PR apache#12777 introduced a leaky singleton pattern for DbgCtl to fix
use-after-free crashes during shutdown. However, removing the
_rm_reference() method broke ABI compatibility with existing plugins
that were compiled against the old header, where the destructor called
this method. This commit restores _rm_reference() as a no-op stub,
allowing old plugins to load successfully while maintaining the leaky
singleton behavior.
@bneradt bneradt mentioned this pull request Jan 9, 2026
Merged
bneradt added a commit to bneradt/trafficserver that referenced this pull request Jan 12, 2026
@bneradt
Restore DbgCtl::_rm_reference() for ABI compatibility
0862ae0 
PR apache#12777 introduced a leaky singleton pattern for DbgCtl to fix
use-after-free crashes during shutdown. However, removing the
_rm_reference() method broke ABI compatibility with existing plugins
that were compiled against the old header, where the destructor called
this method. This commit restores _rm_reference() as a no-op stub,
allowing old plugins to load successfully while maintaining the leaky
singleton behavior.
bneradt added a commit that referenced this pull request Jan 13, 2026
@bneradt
Restore DbgCtl::_rm_reference() for ABI compatibility (#12797)
fac9502 
PR #12777 introduced a leaky singleton pattern for DbgCtl to fix
use-after-free crashes during shutdown. However, removing the
_rm_reference() method broke ABI compatibility with existing plugins
that were compiled against the old header, where the destructor called
this method. This commit restores _rm_reference() as a no-op stub,
allowing old plugins to load successfully while maintaining the leaky
singleton behavior.
bneradt added a commit to bneradt/trafficserver that referenced this pull request Jan 23, 2026
@bneradt
Fix DbgCtl use-after-free shutdown crash via leaky singleton (apache#…
ab4c65a 
…12777)

The DbgCtl registry was experiencing use-after-free crashes during program
shutdown, particularly visible in CI regression tests. The root cause
was the undefined destruction order of static objects in C++.

Problem:
- DbgCtl objects can be static or thread-local with lifetimes spanning program
  execution
- When one compilation unit's DbgCtl destructs and triggers registry cleanup,
  other compilation units may still have DbgCtl objects with pointers into
  that registry
- Thread exit order is also unpredictable, causing similar issues when
  thread-local DbgCtl objects destruct

This implements the "leaky singleton" pattern where the registry is:
- Created on first use
- Never destroyed (destructor is now = default)
- Memory (~20KB) reclaimed by OS at process exit

Fixes: apache#12776
(cherry picked from commit 11e99cb)
bneradt added a commit to bneradt/trafficserver that referenced this pull request Jan 23, 2026
@bneradt
Restore DbgCtl::_rm_reference() for ABI compatibility
f7d7778 
PR apache#12777 introduced a leaky singleton pattern for DbgCtl to fix
use-after-free crashes during shutdown. However, removing the
_rm_reference() method broke ABI compatibility with existing plugins
that were compiled against the old header, where the destructor called
this method. This commit restores _rm_reference() as a no-op stub,
allowing old plugins to load successfully while maintaining the leaky
singleton behavior.

(cherry picked from commit f6cf6fa)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@serrislew serrislew serrislew approved these changes

Assignees

@bneradt bneradt

Labels

Crash Logging

Projects

None yet

Milestone

10.2.0

Development

Successfully merging this pull request may close these issues.

CI Crashes Due to DbgCtl Use-After-Free

2 participants

@bneradt @serrislew