Fix race condition in Arc's get_mut and make_unqiue #26610

aturon · 2015-06-26T22:08:23Z

This commit resolves the race condition in the get_mut and
make_unique functions, which arose through interaction with weak
pointers. The basic strategy is to "lock" the weak pointer count when
trying to establish uniqueness, by reusing the field as a simple
spinlock. The overhead for normal use of Arc is expected to be minimal
-- it will be none when only strong pointers are used, and only
requires a move from atomic increment to CAS for usage of weak pointers.

The commit also removes the unsafe and deprecated status of these functions.

Closes #24880

r? @alexcrichton

cc @metajack @SimonSapin @Ms2ger

alexcrichton · 2015-06-26T22:36:46Z

Could this also move the methods to Arc::{get_mut, make_unique}? Other than that I'd like to get another pair of eyes or two to check this out, but I believe that this is safe.

eefriedman · 2015-06-27T00:14:58Z

src/liballoc/arc.rs

+            let need_clone = self.inner().strong.load(SeqCst) != 1;
+            self.inner().weak.store(1, Relaxed); // release lock immediately;
+                                                 // we don't need to hold it while cloning
+            if need_clone { *self = Arc::new((**self).clone()); }
        }


We should only clone if there are other live strong references; the correct implementation is something like the following (untested, possibly buggy):

if self.inner().strong.compare_and_swap(1, 0, Release) != 1 { // Another strong pointer exists; clone. *self = Arc::new((**self).clone()); } else if self.inner().weak.load(SeqCst) != 1 { // There are no strong pointers left, but there are remaining weak // pointers; steal the contents. // FIXME: Is there a better way to write this? let weak = Weak { _ptr: self._ptr }; self._ptr = 0; let stolen_data = ptr::read(&(*weak._ptr).data); *self = Arc::new(stolen_data); } else { // No other strong or weak pointers; just revive this Arc. self.inner().strong = 1; }

No spinlock necessary.

This would not work for get_mut though, so the spinlock would still need to exist, just not for make_unique?

Yes. Granted, we could change the semantics of get_mut so it kills weak pointers rather than failing, which would allow removing the spinlock, but that's probably not a decision we should make based on implementation details.

talchas · 2015-06-27T00:30:13Z

Clone/Drop for Weak and Arc::weak_count also need to be CAS loops. I would also suggest using weak.CAS(acquire); strong.load(Relaxed); weak.store(Release);, as I believe that is sufficient and will remove an unnecessary fence on x86 (and maybe pull it out into a private is_unique() to reduce code duplication?)

eefriedman · 2015-06-27T00:34:28Z

An argument could be made that get_mut() on an Arc with no other strong references should kill all weak pointers and memcpy the data if necessary instead of failing. Not sure how get_mut() is used in practice. Either way, it needs documentation which explicitly states what happens in the presence of weak pointers.

IIRC, we need to move from atomic increment to CAS anyway to avoid unsafety from overflowing the counter, so any overhead involved in that is irrelevant.

eddyb · 2015-06-27T09:25:41Z

src/liballoc/arc.rs

+
+            // NOTE: this code currently ignores the possibility of overflow
+            // into usize::MAX; in general both Rc and Arc need to be adjusted
+            // to deal with overflow.


Since references take up memory, I don't think usize can be overflowed by them... except we now have safe mem::forget which makes it trivial. RefCell is also affected.

There's a loong discussion about it (internals)

Actually, scratch that, RefCell overflows into a "deadlock" state - mutably locked by too many immutable borrows, which denies any further borrows.

aturon · 2015-06-30T18:50:11Z

Thanks everyone for the feedback, especially @eefriedman for the insight about the semantics of make_unique with respect to weak refs. I've updated the impl and am testing now.

nagisa · 2015-06-30T18:57:09Z

src/liballoc/arc.rs

+    /// ```
+    #[inline]
+    #[unstable(feature = "arc_unique")]
+    pub fn get_mut(this: &mut Arc<T>) -> Option<&mut T> {


Why not &mut self?

That could interfere with any get_mut() on T. (The doc comment still thinks it's a free function, though.)

I don’t see how that’s a problem given we have UFCS.

It follows the new convention for smart pointer methods. I guess it can be written as this: &mut Self though.

aturon · 2015-06-30T23:29:09Z

Additional nits addressed. I switched to a swap + mem::forget to avoid manual zeroing.

alexcrichton · 2015-07-01T05:10:31Z

src/liballoc/arc.rs

+        // Note that we hold both a strong reference and a weak reference.
+        // Thus, releasing our strong reference only will not, by itself, cause
+        // the memory to be deallocated.
+        if this.inner().strong.compare_and_swap(1, 0, Release) != 1 {


Can you add some comments here to the effect of the memory orderings? It's not immediately clear to me why this is Release and the one below is SeqCst. Also, can't the one at the bottom be relaxed b/c this is the only pointer in existence? (e.g. single threaded)

alexcrichton · 2015-07-01T05:17:51Z

A few minor nits here and there, but otherwise looks good to me. Tricky stuff!

r=me

cuzbog · 2015-07-02T17:49:36Z

Any news on this? It would be awesome to get rid of those Servo warnings :)

aturon · 2015-07-02T20:35:45Z

@boghison Sorry, got tied up with other things after sorting out the memory orderings with @alexcrichton. Just running tests locally, and then it's ready for bors.

This commit resolves the race condition in the `get_mut` and `make_unique` functions, which arose through interaction with weak pointers. The basic strategy is to "lock" the weak pointer count when trying to establish uniqueness, by reusing the field as a simple spinlock. The overhead for normal use of `Arc` is expected to be minimal -- it will be *none* when only strong pointers are used, and only requires a move from atomic increment to CAS for usage of weak pointers. The commit also removes the `unsafe` and deprecated status of these functions. Along the way, the commit also improves several memory orderings, and adds commentary about why various orderings suffice.

alexcrichton · 2015-07-02T23:55:52Z

src/liballoc/arc.rs

+
+    // the value usize::MAX acts as a sentinel for temporarily "locking" the
+    // ability to upgrade weak pointers or downgrade strong ones; this is used
+    // to avoid races in `make_unique` and `get_mut`.


I think this comment is slightly out of date now, I believe the only race is between get_mut and downgrade, right?

alexcrichton · 2015-07-03T00:18:18Z

Some final nits about comments, but I have no objections to any literal code written down, so I'm fine landing soon to help servo and following up with a comment heavy PR.

@bors: r+ d77c4b0

bors · 2015-07-03T01:00:32Z

⌛ Testing commit d77c4b0 with merge 4a21775...

@alexcrichton

This commit resolves the race condition in the `get_mut` and `make_unique` functions, which arose through interaction with weak pointers. The basic strategy is to "lock" the weak pointer count when trying to establish uniqueness, by reusing the field as a simple spinlock. The overhead for normal use of `Arc` is expected to be minimal -- it will be *none* when only strong pointers are used, and only requires a move from atomic increment to CAS for usage of weak pointers. The commit also removes the `unsafe` and deprecated status of these functions. Closes #24880 r? @alexcrichton cc @metajack @SimonSapin @Ms2ger

bors · 2015-07-03T02:32:26Z

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-x-android-t, auto-mac-32-opt, auto-mac-64-nopt-t, auto-mac-64-opt, auto-win-gnu-32-nopt-t, auto-win-gnu-32-opt, auto-win-gnu-64-nopt-t, auto-win-gnu-64-opt

rust-highfive assigned alexcrichton Jun 26, 2015

aturon force-pushed the fix_make_unique branch from 87c4643 to 0acddef Compare June 26, 2015 22:14

aturon mentioned this pull request Jun 26, 2015

arc::get_mut/Arc::make_unique are racy #24880

Closed

eefriedman reviewed Jun 27, 2015
View reviewed changes

eddyb reviewed Jun 27, 2015
View reviewed changes

aturon force-pushed the fix_make_unique branch 2 times, most recently from 0d56386 to d264b0e Compare June 30, 2015 17:43

nagisa reviewed Jun 30, 2015
View reviewed changes

aturon force-pushed the fix_make_unique branch 4 times, most recently from 9726a81 to d733e2b Compare June 30, 2015 23:28

alexcrichton reviewed Jul 1, 2015
View reviewed changes

aturon force-pushed the fix_make_unique branch 2 times, most recently from 8566f31 to 71b496c Compare July 2, 2015 00:45

aturon force-pushed the fix_make_unique branch from 71b496c to 1c8e43c Compare July 2, 2015 20:34

aturon force-pushed the fix_make_unique branch from 1c8e43c to d77c4b0 Compare July 2, 2015 20:58

alexcrichton reviewed Jul 2, 2015
View reviewed changes

bors merged commit d77c4b0 into rust-lang:master Jul 3, 2015

SimonSapin mentioned this pull request Jul 15, 2015

Remove the implementation of DerefMut for FlowRef. servo/servo#6503

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix race condition in Arc's get_mut and make_unqiue #26610

Fix race condition in Arc's get_mut and make_unqiue #26610

aturon commented Jun 26, 2015

alexcrichton commented Jun 26, 2015

eefriedman Jun 27, 2015

talchas Jun 27, 2015

eefriedman Jun 27, 2015

talchas commented Jun 27, 2015

eefriedman commented Jun 27, 2015

eddyb Jun 27, 2015

bluss Jun 27, 2015

eddyb Jun 29, 2015

aturon commented Jun 30, 2015

nagisa Jun 30, 2015

Ms2ger Jun 30, 2015

nagisa Jun 30, 2015

bluss Jun 30, 2015

aturon commented Jun 30, 2015

alexcrichton Jul 1, 2015

alexcrichton commented Jul 1, 2015

cuzbog commented Jul 2, 2015

aturon commented Jul 2, 2015

alexcrichton Jul 2, 2015

alexcrichton commented Jul 3, 2015

bors commented Jul 3, 2015

bors commented Jul 3, 2015

Fix race condition in Arc's get_mut and make_unqiue #26610

Fix race condition in Arc's get_mut and make_unqiue #26610

Conversation

aturon commented Jun 26, 2015

alexcrichton commented Jun 26, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

talchas commented Jun 27, 2015

eefriedman commented Jun 27, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aturon commented Jun 30, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aturon commented Jun 30, 2015

Choose a reason for hiding this comment

alexcrichton commented Jul 1, 2015

cuzbog commented Jul 2, 2015

aturon commented Jul 2, 2015

Choose a reason for hiding this comment

alexcrichton commented Jul 3, 2015

bors commented Jul 3, 2015

bors commented Jul 3, 2015