Tracking Issue for mpmc

[obeis]

Feature gate: #![feature(mpmc_channel)]

This is a tracking issue for mpmc_channel as described in ACP.

History

• ACP
• Implementation: Add multi-producer, multi-consumer channel (mpmc) #126839
• Final comment period (FCP)
• Stabilization PR

[ora-0]

The sync::mpsc module's sync_channel function returned a SyncSender and SyncReceiver type, but the new mpmc module's sync_channel returns just a normal Sender and Receiver type. Is this symmetry something we want to preserve?

This shouldn't affect any usage of them, except for the slightly different documentation provided.

[ibraheemdev]

I wonder if there's a downside to SyncSender and Sender being separate types, and you would ever need to create an enum (which is unnecessary because they are internally enum) to switch between them dynamically. This seems unlikely as they have different semantics, but theoretically if you are only using try_send you may want to. I think this is less of a concern than keeping the modules consistent, but the SyncSender naming is itself confusing, so I'm not sure.

[andrew-otiv]

Does this channel deliver each value to all receivers? Or is it intended for building work queues? This is unclear from the existing documentation.

A multi-consumer channel with backpressure like this one would be most welcome in our application! Thanks everyone who is pushing this forward!

[glyn]

Does this channel deliver each value to all receivers? Or is it intended for building work queues? This is unclear from the existing documentation.

Great question! I've been digging around in the channels code recently trying to understand the precise behaviour and I can't see any evidence of each value being delivered to all receivers. In fact, there are a few indications that delivering each value to all receivers is not the intended behaviour:

1. ACP (referenced in the OP) has a section "Motivating examples or use cases" which implies each value is delivered to at most one receiver.
2. The MPMC tests do not clone any receivers.
3. The crossbeam-channel crate, from which the MPMC code was derived, contains this example, which may settle the matter:

use crossbeam_channel::unbounded;

let (s1, r1) = unbounded();
let (s2, r2) = (s1.clone(), r1.clone());
let (s3, r3) = (s2.clone(), r2.clone());

s1.send(10).unwrap();
s2.send(20).unwrap();
s3.send(30).unwrap();

assert_eq!(r3.recv(), Ok(10));
assert_eq!(r1.recv(), Ok(20));
assert_eq!(r2.recv(), Ok(30));

Comments:

1. I'd prefer ACP to state the intended behaviour explicitly.
2. The lack of receiver cloning in the MPMC tests seems like a serious omission, unless this is tested somewhere else.

[glyn]

@andrew-otiv I can confirm the following behaviour:

use std::sync::mpmc::*;

#[test]
fn receiver_cloning() {
    let (tx, rx) = channel::<i32>();
    let rx2 = rx.clone();

    tx.send(1).unwrap();
    tx.send(2).unwrap();

    assert_eq!(rx2.recv(), Ok(1));
    assert_eq!(rx.recv(), Ok(2));
}

Notes:

1. Basic tests of MPMC receiver cloning #139836 passes the above test in CI (and should be merged in due course).
2. DO NOT MERGE - Checking MPMC tests run by CI #139844 confirms the MPMC tests are actually run by CI.

[andrew-otiv]

Thanks for clarifying that! I was hoping for a broadcast channel with try_send so we can panic when the buffer fills. In our application deadlocking or blowing up an unbounded buffer until OOM are both worse than crashing. Every multi consumer channel in both tokio and std can either block the sender, silently "leak" until oom, or drop values.

[Skgland]

I am not sure I quite understand your problem.

Every multi consumer channel in both tokio and std can either block the sender, silently "leak" until oom, or drop values.

Sender has a try_send, send_timeout and send_deadline methods for non-blocking behaviour and there is sync_channel to limit the amount of values in-flight which should resolve the silent "leak" until oom problem.

So the problem with this is that it drops in-flight values if the last receiver is dropped before all values are received?

Edit: Could there be a miscommunication of what it means that each value is delivered to only one receiver instead of being delivered to all receivers? My understanding is the following:

• The only time a value is dropped by the channel is if the channel is closed (last receiver is dropped)
  before all in-flight values are received.
• Otherwise all values send will be received by one of the receivers (the values isn't copied/cloned to all receivers).