Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix potential deadlock during process fork #817

Merged
merged 2 commits into from
Mar 12, 2024
Merged

Fix potential deadlock during process fork #817

merged 2 commits into from
Mar 12, 2024

Conversation

vickenty
Copy link
Contributor

@vickenty vickenty commented Mar 5, 2024

What does this PR do?

Fix a potential deadlock in post_fork().

Description of the Change

close_socket acquires _socket_lock, and calling it with the lock held deadlocks.

On top of this the warning did not prove very useful, as it would also trigger when the application is sending metrics from another thread when the process forks, in which case the warning is a bit misleading.

Without the warning there is no reason to check for socket being open, so just silently call close_socket() after the fork to avoid sharing the file descriptor between processes if another thread re-opened it between pre_fork() and fork itself.

Alternate Designs

Possible Drawbacks

Verification Process

Additional Notes

Release Notes

Review checklist (to be filled by reviewers)

  • Feature or bug fix MUST have appropriate tests (unit, integration, etc...)
  • PR title must be written as a CHANGELOG entry (see why)
  • Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
  • PR must have one changelog/ label attached. If applicable it should have the backward-incompatible label attached.
  • PR should not have do-not-merge/ label attached.
  • If Applicable, issue must have kind/ and severity/ labels attached at least.

vickenty added 2 commits March 5, 2024 17:23
close_socket acquires _socket_lock, and calling it with the lock held
deadlocks.

On top of this the warning did not prove very useful, as it would also
trigger when the application is sending metrics from another thread
when the process forks, in which case the warning is a bit misleading.

Without the warning there is no reason to check for socket being open,
so just silently call close_socket() after the fork to avoid sharing
the file descriptor between processes if another thread re-opened it
between pre_fork() and fork itself.
@aweldy2
Copy link

aweldy2 commented Mar 7, 2024

+1 to this being a pretty major issue -- we encountered this exact bug earlier this week. @vickenty another option would be to change socket_lock to be a threading.RLock instead of a threading.Lock so that it's reentrant.

@vickenty vickenty changed the title Fix a potential deadlock during process fork Fix potential deadlock during process fork Mar 8, 2024
@vickenty vickenty marked this pull request as ready for review March 8, 2024 12:03
@vickenty vickenty requested review from a team as code owners March 8, 2024 12:03
@vickenty vickenty added the changelog/Fixed Fixed features results into a bug fix version bump label Mar 8, 2024
@vickenty vickenty merged commit e9d427f into master Mar 12, 2024
14 of 15 checks passed
@vickenty vickenty deleted the vickenty/lock branch March 12, 2024 17:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog/Fixed Fixed features results into a bug fix version bump
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants