Introduce crossbeam-deque

text/2017-11-05-deque.md

@@ -0,0 +1,165 @@
+# Summary
+
+Introduce crate `crossbeam-deque` with an implementation of the Chase-Lev deque.
+
+# Motivation
+
+Work-stealing deque is a common building block in schedulers, for example in
+[Rayon](https://github.com/nikomatsakis/rayon/) and
+[futures-pool](https://github.com/carllerche/futures-pool).
+Currently, these crates are using the deque from [Coco](https://github.com/stjepang/coco),
+which is an experimental crate that will soon be deprecated in favor of Crossbeam.
+
+`crossbeam-deque` will use the new Crossbeam's epoch-based garbage collector.
+
+# Detailed design
+
+The basis for implementation is the deque from Coco that already had a good run in
+Rayon and consequently in Stylo. However, there will be a few differences:
+
+1. Memory management is done using `crossbeam-epoch`.
+2. The `Worker`/`Stealer` interface is redesigned as `Deque`/`Stealer`.
+3. Several new convenience methods are introduced.
+
+Other than that, the crucial (and the most tricky to implement) methods like
+`push`, `pop`, and `steal` do not depart from the original implementation
+(except for the changes associated with porting it to Crossbeam's new epoch-based GC).
+
+## The interface
+
+```rust
+/// The "worker" side of the deque.
+pub struct Deque<T>;
+
+unsafe impl<T: Send> Send for Deque<T> {}
+
+impl<T> Deque<T> {
+    /// Creates a new deque.
+    pub fn new() -> Deque<T>;
+
+    /// Creates a new deque with specified minimum capacity.
+    pub fn with_min_capacity(min_cap: usize) -> Deque<T>;
+
+    /// Returns `true` if the deque is empty.
+    pub fn is_empty(&self) -> bool;
+
+    /// Returns the number of elements in the deque.
+    pub fn len(&self) -> usize;
+
+    /// Pushes an element onto the bottom of the deque.
+    pub fn push(&self, value: T);
+
+    /// Pops an element from the bottom of the deque.
+    pub fn pop(&self) -> Option<T>;
+
+    /// Steals an element from the top of the deque.
+    pub fn steal(&self) -> Steal<T>;
+
+    /// Creates a new stealer for the deque.
+    pub fn stealer(&self) -> Stealer<T>;
+}
+
+/// The "stealer" side of the deque.
+pub struct Stealer<T>;
+
+unsafe impl<T: Send> Send for Stealer<T> {}
+unsafe impl<T: Send> Sync for Stealer<T> {}
+
+impl<T> Stealer<T> {
+    /// Returns `true` if the deque is empty.
+    pub fn is_empty(&self) -> bool;
+
+    /// Returns the number of elements in the deque.
+    pub fn len(&self) -> usize;
+
+    /// Steals an element from the top of the deque.
+    pub fn steal(&self) -> Steal<T>;
+}
+
+impl<T> Clone for Stealer<T> { ... }
+
+/// Possible outcomes of a steal operation.
+pub enum Steal<T> {
+    /// The deque was empty.
+    Empty,
+    /// Some data was stolen.
+    Data(T),
+    /// Lost the race for stealing data to another concurrent operation. Try again.
+    Retry,
+}
+```
+
+An interesting difference from Coco's API is that a deque is now
+constructed using `Deque::new()` rather than using a global function returning
+a `(Worker<T>, Stealer<T>)` pair.
+Also, there are two ways of creating multiple stealers: you can either create each of them
+with `Deque::stealer` or by clone an existing one - whatever works best for you.
+
+Another addition is the `with_min_capacity` constructor. Deques dynamically grow and
+shrink as elements are inserted and removed. By specifying a large minimum capacity
+it is possible to reduce the frequency of reallocations.
+
+The `steal` method returns a `Steal<T>`. Instead of retrying on a failed CAS operation, the
+method returns immediately with `Steal::Inconsistent`. This gives schedulers
+fine control over what to do in case of contention.
+
+There is an open [pull request](https://github.com/crossbeam-rs/crossbeam-deque/pull/1)
+implemented according to this RFC.
+
+# Drawbacks
+
+None.
+
+# Alternatives
+
+### Hazard pointers
+
+Ultimately, hazard pointers would probably be a better fit for the deque than
+epoch-based garbage collection.
+
+In theory, epoch-based GC may leak memory indefinitely if a pinned thread is preempted,
+while hazard pointers give stricter guarantees. HP-based GC guarantees that there
+is always a bounded number of still-unreclaimed garbage objects.
+On the other hand, the advantage of epoch-based GC is that it allows fast
+traversal through linked data structures. The Chase-Lev deque isn't a linked data
+structure, so here we don't gain anything by choosing epoch-based GC over HP-based GC.
+
+Moreover, HP-based GC would allow us to force perfectly eager destruction of
+buffers: just like `Arc`, but without the overhead of contended reference counting.
+The idea is that when a thread wants to destroy a buffer, it would check
+all hazard pointers to see if it is still used. If there is a thread using it,
+we'd CAS the hazard pointer using the buffer and set it to null. When that thread
+notices that someone has set its hazard pointer to null, it would take over the
+responsibility of destroying the buffer. Then it would check all hazard pointers to see if
+anyone is still using it and continue in the same vein.
+
+At some point in the future, we should definitely experiment with HP-based
+garbage collection.
+
+### Signature of `Stealer::steal`
+
+Instead of returning an explicit enum `Steal<T>`, the `steal` function could also
+return a `Result`, in one of the following two forms:
+
+1. `Result<Option<T>, StealError>`, where `StealError` is equivalent to `Steal::Retry`.
+2. `Result<T, StealError>`, where `StealError` is an enumeration of `Empty` and `Retry`.
+
+Returning a `Result` would allow one to use it with the `?` operator and all
+the other commonly used combinators. However, since stealing is a rather unusual
+and rarely used operation, ergonomics are not of high priority in this situation.
+
+# Unresolved questions
+
+When building `futures-pool`, **[@carllerche](https://github.com/carllerche)**
+wanted a few additional methods:
+
+1. [A method that steals more than one value at a time.](https://github.com/stjepang/coco/issues/11#issuecomment-339785208)
+2. [A `steal_when_greater` method.](https://github.com/stjepang/coco/issues/10#issuecomment-339785563)
+
+While the lack of those is not a deal-breaker, it would still be nice to have them.
+
+At the moment, I'd probably prefer to push forward with the current minimal design,
+and then later on discuss how exactly these methods would work, possibly after (if?)
+we switch to HP-based GC.
+
+All that said, suggestions are always welcome.