-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
plugins/logs: fixed reconfiguration race condition #2948
plugins/logs: fixed reconfiguration race condition #2948
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice find! See my comment about only grabbing the mask while holding the mutex.
fb16bb6
to
551bb2a
Compare
I updated the PR. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👏 Good find, thank you!
551bb2a
to
12dc58d
Compare
@srenatus, I'm glad I could help. I squashed commits according to instructions, can you reapprove please so I can merge? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@kubaj Can you please squash the commits and update the commit message ? The sign-off line is duplicated. |
44a4906
to
3cfa8d8
Compare
@ashutosh-narkar I updated the commit message to remove duplicate sign-off. I hope it's all you need. |
@kubaj thanks for updating your commit message. I think you'll need to rebase from master and then push your single commit to your fork as currently it's showing that 11 commits need to be merged. |
Masking part of the logging plugin had incorrectly placed mutex -- only for writing the variable p.mask, but not for reading it. This caused race condition in the situation when bundle update calls plugin.Reconfigure() method between setting the p.mask variable and then evaluating it. Reconfiguration of the plugin sets p.mask to nil and therefore calling p.mask.Eval() results in panic. Signed-off-by: Jakub Kulich <jakub.kulich@exponea.com>
3cfa8d8
to
a8fd154
Compare
@ashutosh-narkar sorry about that. I fixed that, it should be ok now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! 🎉
Bug summary
Logging plugin had race condition while it was masking the log and plugin was reconfigured at the same time. This caused panic of the OPA. Fixed issue for us and also may fix #2835 .
Details
Masking part of the logging plugin had incorrectly placed mutex -- only
for writing the variable p.mask, but not for reading it. This caused
race condition in the situation when bundle update calls plugin.Reconfigure()
method between setting the p.mask variable and then evaluating it.
Reconfiguration of the plugin sets p.mask to nil and therefore calling
p.mask.Eval() results in panic.
Tests covering functionality shouldn't change, test covering the fixed issue is not delivered, because it's difficult to write a test simulating the race condition (if you know about some efficient way, feel free to guide me, I'll happily learn).