Simpler and faster lock-free linked list

Lock-free list implementation is considerably simplified, taking into
account a limited number of operations that it needs to support.

* prev pointers in the list are not marked for removal, since we
  don't need to support linearizable backwards iteration.
* helpDelete method is completely removed. All "delete-helping" is
  performed only by correctPrev method.
* correctPrev method bails out when the node it works on is removed to
  reduce contention during concurrent removals.
* Special open methods "isRemoved" and "nextIfRemoved" are introduced
  and are overridden in list head class (which is never removed).
  This ensures that on long list "removeFist" operation (touching head)
  does not interfere with "addLast" (touch tail). There is still
  sharing of cache-lines in this case, but no helping between them.

All in all, this improvement reduces the size of implementation code
and makes it considerably faster. Operations on LinkedListChannel are
now much faster (see timings of ChannelSendReceiveStressTest).

Additionally, change also reduces amount of wasteful helping and
starvation of ConflatedChannel receiver with many senders:

* removeFirstIfIsInstanceOfOrPeekIf that is used to take the first
  element from a channel does not need help every time when
  it finds a removed element, but quickly tries the next element
  to see if it can remove that one instead for a few times.
* conflatePreviousSendBuffered does not help with removal when its own
  node was removed.

Author: elizarov

kotlinx-coroutines-core/common/src/channels/ConflatedChannel.kt

@@ -4,8 +4,8 @@
 
 package kotlinx.coroutines.channels
 
-import kotlinx.coroutines.selects.*
 import kotlinx.coroutines.internal.*
+import kotlinx.coroutines.selects.*
 
 /**
  * Channel that buffers at most one element and conflates all subsequent `send` and `offer` invocations,
@@ -46,13 +46,14 @@ internal open class ConflatedChannel<E> : AbstractChannel<E>() {
     }
 
     private fun conflatePreviousSendBuffered(node: SendBuffered<E>) {
-        // Conflate all previous SendBuffered, helping other sends to conflate
-        var prev = node.prevNode
+        // Conflate all previous SendBuffered, helping other sends to conflate, but only as long
+        // as this node itself is not removed
+        var prev = node.prevNodeIfNotRemoved ?: return
         while (prev is SendBuffered<*>) {
             if (!prev.remove()) {
                 prev.helpRemove()
             }
-            prev = prev.prevNode
+            prev = prev.prevNodeIfNotRemoved ?: return
         }
     }
 

kotlinx-coroutines-core/common/src/internal/LockFreeLinkedList.common.kt

@@ -11,6 +11,7 @@ public expect open class LockFreeLinkedListNode() {
     public val isRemoved: Boolean
     public val nextNode: LockFreeLinkedListNode
     public val prevNode: LockFreeLinkedListNode
+    public val prevNodeIfNotRemoved: LockFreeLinkedListNode?
     public fun addLast(node: LockFreeLinkedListNode)
     public fun addOneIfEmpty(node: LockFreeLinkedListNode): Boolean
     public inline fun addLastIf(node: LockFreeLinkedListNode, crossinline condition: () -> Boolean): Boolean

kotlinx-coroutines-core/js/src/internal/LinkedList.kt

@@ -23,6 +23,7 @@ public open class LinkedListNode {
 
     public inline val nextNode get() = _next
     public inline val prevNode get() = _prev
+    public inline val prevNodeIfNotRemoved: LockFreeLinkedListNode? get() = _prev.takeIf { !_removed }
     public inline val isRemoved get() = _removed
 
     public fun addLast(node: Node) {