Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

time: Fix race condition in timer drop #3229

Merged
merged 5 commits into from
Dec 9, 2020

Conversation

bdonlan
Copy link
Contributor

@bdonlan bdonlan commented Dec 7, 2020

Dropping a timer on the millisecond that it was scheduled for, when it was on
the pending list, could result in a panic previously, as we did not record the
pending-list state in cached_when.

Hopefully fixes: ZcashFoundation/zebra#1452

Motivation

Solution

Dropping a timer on the millisecond that it was scheduled for, when it was on
the pending list, could result in a panic previously, as we did not record the
pending-list state in cached_when.

Hopefully fixes: ZcashFoundation/zebra#1452
@bdonlan bdonlan requested a review from carllerche December 7, 2020 23:39
Copy link
Member

@hawkw hawkw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple of very minor nits, but this looks correct to me!

use std::sync::{Arc, Mutex};
use std::task::Context;

let paniced = Arc::new(AtomicBool::new(false));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: should be panicked

Comment on lines 247 to 249
unsafe {
item.set_cached_when(u64::max_value());
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a comment a few lines above (on the match) that seems to imply that mark_pending updates cached_when; this isn't actually the case. Maybe we should update that comment to indicate that this is where cached_when is updated?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, mark_pending is probably the right place to do this update (it does this update when a timer is found to have been rescheduled)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering about that too...this is the only place we call mark_pending, so it makes sense. Then, we can remove the set_cached_when methods.

tokio/src/time/driver/entry.rs Outdated Show resolved Hide resolved
@Darksonn Darksonn added A-tokio Area: The main tokio crate C-bug Category: This is a bug. M-time Module: tokio/time labels Dec 8, 2020
Co-authored-by: Eliza Weisman <eliza@buoyant.io>
@hawkw
Copy link
Member

hawkw commented Dec 8, 2020

@carllerche did you want to take a look at this before it merges? it looks good to me but i thought you might want to sign off...

@carllerche
Copy link
Member

@hawkw I trust you 👍

@carllerche carllerche merged commit 9706ca9 into tokio-rs:master Dec 9, 2020
teor2345 added a commit to teor2345/zebra that referenced this pull request Dec 14, 2020
Updates the tokio dependency to the commit that merged
tokio-rs/tokio#3229, which should fix the time wheel panic in ZcashFoundation#1452.
dconnolly pushed a commit to ZcashFoundation/zebra that referenced this pull request Dec 14, 2020
Updates the tokio dependency to the commit that merged
tokio-rs/tokio#3229, which should fix the time wheel panic in #1452.
@teor2345
Copy link
Contributor

I've been running zebrad for about a week, and I haven't seen any of these panics. So it looks like this PR fixed the panics for us.

@carllerche
Copy link
Member

@teor2345 great news. Thanks for checking 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tokio Area: The main tokio crate C-bug Category: This is a bug. M-time Module: tokio/time
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants