JRFC 34 - Streamer - A Stream Multiplexing Interface

[jbenet]

Streamer - A Stream Multiplexing Interface

There are many ways to do stream muxing (multiplexing). Many transports offer their own stream multiplexing, for example:

• SSH
• QUIC
• HTTP/2
• WebRTC

These often differ in how to handle flow control, congestion, priorities, stream hierarchies, and so on. The interfaces different and are often incomplete. They requires knowing much about the specific transport in question, which forces applications to understand the stream muxing library.

This is a great moment to standardize a common API. We should have

• a standard, cross-platform interface for stream-multiplexors
• a default implementation that works over any reliable stream (e.g. TCP)
• native implementation wrappers
• polyfill for missing features
• interfaces for both simple streams (no config) and advanced features (priorities, etc)

---

Links:

• Muxado: https://github.com/inconshreveable/muxado/tree/wip
• HTTP2: https://http2.github.io/http2-spec/#StreamsLayer
• WebRTC: http://w3c.github.io/webrtc-pc/#peer-to-peer-data-api
• QUIC: https://docs.google.com/document/d/1RNHkx_VvKWyWg6Lr8SZ-saqsQx7rFV-ev2jRFUoVD34/mobilebasic#h.zd2x76sign3y

[jbenet]

cc @inconshreveable @substack @whyrusleeping @mafintosh @maxogden

[max-mapper]

in dat we are using https://github.com/maxogden/multiplex

which is a varint length prefixed stream of protocol buffers

e.g. each message is simply:

[varint][protobuf]

where varint is a varint of the length of the protobuf. it's the simplest design we could come up with

[mafintosh]

The protobuf used in multiplex looks like this:

enum TYPE {
  DATA = 1;
  END = 2;
  ERROR = 3;
  OPEN = 4;
}

message Frame {
  required sint32 channel = 1;
  optional string name = 2;
  optional TYPE type = 3 [default=DATA];
  optional bytes data  = 4;
}

The person instantiating the connection chooses a local channel id and sends that the frame to the remote. If the remote wants to reply with data on the same channel he uses -channel so the when the instantiator receives this frame he can match it to his local channel without there being clashes if both parties use the same channel id for two channels.

Optionally you can set name on the first frame to associate a string name with a channel

[mafintosh]

Also channel ids can be reused after a channel ends to keep the channel numbers as low as possible

[mafintosh]

I should note that we designed our multiplex protocol to be as simple as possible (no per channel watermarks etc) since this fits 99% of our use-cases so far.

[max-mapper]

we might wanna think about doing doing varints in the protobuf style, where we add 3 bits to the end for a 'wire type' ID https://developers.google.com/protocol-buffers/docs/encoding#structure

if we use wire type 2 (length prefixed) then our format would be 100% protobufs all the way down, with this schema describing the outermost container format:

message Stream {
  repeated bytes data = 1;
}

[jbenet]

Does it have any flow control? (iirc, no). We need flow control because otherwise sending lots of data makes our data flows starve the control flows (things like dht). We already ran into this :( 

May want to catch up on the discussion @inconshreveable @substack and I were having in #ipfs. I'll find a link and send. The goal is to try and use native streaming as much as possible when present (say on WebRTC, http2, QUIC), mapping with a common interface. But have one library that we can always fall back on if there isn't any native stuff.

Think of it like an abstract-stream-mux module