Finish media streaming (session start, stats)

index.bs

@@ -63,6 +63,7 @@ url: https://tools.ietf.org/html/rfc8122#section-5; type: dfn; spec: RFC8122; te
 url: https://tools.ietf.org/html/rfc8122#section-5; type: dfn; spec: RFC8122; text: sha-512
 url: https://tools.ietf.org/html/rfc8122#section-5; type: dfn; spec: RFC8122; text: md2
 url: https://tools.ietf.org/html/rfc8122#section-5; type: dfn; spec: RFC8122; text: md5
+url: https://tools.ietf.org/html/rfc6381#section-3; type: dfn; spec: RFC6381; text: codecs parameter
 </pre>
 
 <h2 class='no-num no-toc no-ref' id='document-status'>Status of this document</h2>
@@ -1409,17 +1410,12 @@ following properties:
 : receive-video (required)
 :: A list of capabilities for receiving video. For an explanation of fields, see below.
 
-: receive-data (required)
-:: A list of arbitrary data formats the device supports
-    for receiving data.
 
-The format type is used as the basis for audio, video, and data capabilities.
+The format type is used as the basis for audio and video capabilities.
 Formats are composed of the following properties:
 
-<!-- TODO, specify where names are defined. -->
-
 : name (required)
-:: The name of the format. Expected values include "vp8", "h264", "opus."
+:: The name of the codec.   This must be a single-codec RFC 6381 [=codecs parameter=].
 
 <!-- TODO, specify where codec-specific parameters are defined. -->
 
@@ -1484,8 +1480,163 @@ following additional fields:
 Sessions {#streaming-sessions}
 ------------------------------------
 
-TODO
+To start a streaming session, a sender may send a
+streaming-session-start-request message with the following properties:
+
+: streaming-session-id
+:: Identifies the streaming session.  Must be unique for the (sender,
+    receiver) pair.  Can be used later to modify or terminate a
+    streaming session.
+
+: desired-stats-interval
+:: Indicates the frequency the receiver should send stats messages to
+    the sender.
+
+: stream-offers
+:: Indicates the streams that the receiver can request from the sender.
+
+Each stream offer contains the following properties:
+
+: media-stream-id
+:: Identifies the media stream being offered.  Must be unique within
+    the streaming session.  Can be used by the receiver to request the
+    media session.
+
+: display-name
+:: An optional name intended to be shown to a user, such that the
+    receiver may allow the user to choose which media streams to
+    receive, or if they are received automatically by the receiver,
+    give the user some information about what th e media stream is.
+
+: audio
+:: A list of audio encodings offered.  An audio encoding is a series
+    of encoded audio frames.  Encodings define properties needed by
+    the receiver to know how to decode the encoding, such as codec and
+    sample rate.  They can differ by codec and related properties,
+    but should be different encodings of the same audio.
+
+: video
+:: A list of video encodings offered.  A video encoding is a series of
+    encoded video frames.  Encodings define properties needed by the
+    receiver to know how to decode the encoding, such as codec and
+    default duration.  They can differ by codec and potentially other
+    properties, but should be different encodings of the same video.
+
+: data
+:: A list of data encodings offered.  A data encoding is a series of
+    data frames.  Encodings define properties needed by the
+    receiver to know how to interpret the encoding, such as data type and
+    default duration.  They can differ by data type and potentially other
+    properties, but should be different encodings of the same data.
+    (For encodings of different data, use distinct media streams,
+     not distinct encodints with the same media stream).
+
+
+Each audio encoding offered defines the following properties:
+
+: encoding-id
+:: Identifies the audio encoding being offered.  Must be unique within
+    the media stream.
+
+: codec-name
+:: The name of the codec used by the encoding.  This must be a single-codec
+   RFC 6381 [=codecs parameter=].
+
+: time-scale
+:: The time scale used by all audio frames.  This allows senders to
+    make audio-frame messages smaller by not including the time scale
+    in each one.
+
+: default-duration:
+:: The duration of an audio frame .  This allows senders to make
+    audio-frame messages smaller by not including the duration for
+    audio-frame messages that have the default duration.
+
+Each video encoding offered defines the following properties:
+
+: encoding-id
+:: Identifies the video encoding being offered.  Must be unique within
+    the media stream.
+
+: codec-name
+:: The name of the codec used by the encoding.
+
+: time-scale
+:: The time scale used by all video frames.  This allows senders to
+    make video-frame messages smaller by not including the time scale
+    in each one.
+
+: default-duration:
+:: The default duration of a video frame .  This allows senders to make
+    video-frame messages smaller by not including the duration for
+    video-frame messages that have the default duration.
+
+: default-rotation:
+:: The default rotation of a video frame .  This allows senders to make
+    video-frame messages smaller by not including the rotation for
+    video-frame messages that have the default rotation.
+
+Each data encoding offered defines the following properties:
+
+: encoding-id
+:: Identifies the data encoding being offered.  Must be unique within
+    the media stream.
+
+: data-type-name
+:: The name of the data type used by the encoding.
+
+: time-scale
+:: The time scale used by all data frames.  This allows senders to
+    make data-frame messages smaller by not including the time scale
+    in each one.
+
+: default-duration:
+:: The duration of an data frame .  This allows senders to make
+    data-frame messages smaller by not including the duration for
+    data-frame messages that have the default duration.
+
+After receiving a streaming-session-start-request message, a receiver
+should send back a streaming-session-start-response message with the
+following properties:
+
+: desired-stats-interval
+:: Indicates the frequency the sender should send stats messages to
+    the receiver.
+
+: stream-requests
+:: Indicates which media streams the receiver would like to receiver
+    from the sender.
+
+Each stream request contains the following properties:
+
+: media-stream-id
+:: The ID of the stream reqeusted.
+
+: audio (optional)
+:: The requested audio encoding, by encoding ID
+
+: video (optional)
+:: The requested video encoding, by encoding ID.  It may
+    include a target resolution and maximum frame rate.  The sender
+    should not exceed the maximum frame rate and should attempt to
+    send at the target bitrate, possibly exceeding it by a small amount.
+
+: data (optional)
+:: The requested data encoding, by encoding ID
+
 
+During a streaming session, the receiver can modify the requests it
+made for encodings by sending a streaming-session-modify-request
+containing a modify list of stream-requests.  When the sender receives
+a streaming-session-modify-request, it should send back a
+streaming-session-modify-response.
+
+Finally, the sender may terminate the streaming session by sending
+a streaming-session-terminate-request command.  When the receiver
+receives the streaming-session-terminate-request, it should send back
+a streaming-session-terminate-response.  The receiver can terminate at
+any point and notify the sender by sending a
+streaming-session-terminate-event message.
 
 Audio {#streaming-audio}
 ------------------------------
@@ -1518,13 +1669,13 @@ separate QUIC streams.
     helps to avoid sending duplicate information in every frame.
 
 : start-time
-:: Identifies the beginning of the time range of the audio frame.  The time
+:: Identifies the beginning of the time range of the audio frame. The time
     scale is inferred from the properties of the encoding (from the
     audio-encoding-offer).  The end time can be inferred from the
     start time and duration.
 
 : duration
-:: If present, the duration of the audio frame.  The time
+:: If present, the duration of the audio frame. The time
     scale is inferred from the properties of the encoding.  Likewise, if not
     present, the duration is inferred from the properties of the encoding.
 
@@ -1573,14 +1724,14 @@ ending at the last dependent frame.
     If not present, the default value is [-1].
 
 : start-time
-:: Identifies the beginning of the time range of the video frame.  The time
+:: Identifies the beginning of the time range of the video frame. The time
     scale is inferred from the properties of the encoding (from the
     video-encoding-offer).  The end time can be inferred from the
     start time and duration.
 
 : duration
-:: If present, the duration of the video frame.  The time
-    scale is inferred from the properties of the encoding.  If not
+:: If present, the duration of the video frame. The time
+    scale is inferred from the properties of the encoding. If not
     present, that means duration is unknown.
 
 : sync-time
@@ -1598,24 +1749,21 @@ ending at the last dependent frame.
     inferred from the properties of the encoding.
 
 
-Data {#media-streaming-data}
+Data {#streaming-data}
 ------------------------------------
 
 Media senders may send timed data to media receivers by sending [=data-frame=] messages (see
 [[#appendix-a]]) with the following keys and values.  A data frame message
-contains an arbitrary payload that can be synchronized with and video, such as
-text track data.  A series of data frames that share a data type and timeline
+contains an arbitrary payload that can be synchronized with audio and video.
+A series of data frames that share a data type and timeline
 form a data encoding.
 
 To allow for data frames to be sent out of order, they may be sent in separate
 QUIC streams, but more than one data frame may be sent in one QUIC stream if
 that makes sense for a specific type of data.
 
-Text track data uses a payload type of text, a default duration of unknown, and
-a timescale of 1000000 (microseconds).
-
 : encoding-id
-:: Identifies the media encoding to which this data frame belongs.  This can be
+:: Identifies the data encoding to which this data frame belongs.  This can be
     used to reference properties of the encoding such as the type of data and
     time scale.  Referencing properties of the encoding through the encoding id
     helps to avoid sending duplicate information in every frame.
@@ -1672,7 +1820,116 @@ QUIC streams.
 Stats {#streaming-stats}
 ------------------------------
 
-TODO
+During a streaming session, the sender should send stats with the
+streaming-session-sender-stats-event at the interval the receiver requested.  It
+should send all of the following stats for all of the media streams it is
+sending. The streaming-session-sender-stats-event message contains the following
+properties:
+
+: streaming-session-id
+:: The ID of the streaming session these stats apply to.
+
+: system-time
+:: The time when the stats were calculated, using a monotonic system
+    clock.
+
+: audio
+:: Stats specific to audio.  Stats for multiple encodings can be sent
+    at once, but encodings need not be included if the stats haven't
+    changed.  See below.
+
+: video
+:: Stats specific to video.  Stats for multiple encodings can be sent
+    at once, but encodings need not be included if the stats haven't
+    changed.  See below.
+
+Audio encoding sender stats include the following properties:
+
+: encoding-id
+:: The ID of the encoding for which the stats apply.
+
+: cumulative-sent-frames
+:: The total number of frames sent.
+
+: cumulative-encode-delay
+:: The sum of the time spent encoding frames sent.
+
+Video encoding sender stats include the following properties:
+
+: encoding-id
+:: The ID of the encoding for which the stats apply.
+
+: cumulative-sent-duration
+:: The sum of all of the durations of all of the audio frames sent.
+
+: cumulative-encode-delay
+:: The sum of the time spent encoding frames sent.
+
+: cumulative-dropped frames
+:: The total number of frames that were not sent due to network, CPU,
+    or other contraints.
+
+
+Durings a streaming session, the receiver should send stats with the
+streaming-session-receiver-stats-event at the interval the sender requested.
+It should send all of the following stats for all of the media streams it is
+receiving.  The streaming-session-receiver-stats-event message contains the
+following properties:
+
+: streaming-session-id
+:: The ID of the streaming session these stats apply to.
+
+: system-time
+:: The time when the stats were calculated, using a monotonic system
+    clock.
+
+: audio
+:: Stats specific to audio.  Stats for multiple encodings can be sent
+    at once, but encodings need not be included if the stats haven't
+    changed.  See below.
+
+: video
+:: Stats specific to video.  Stats for multiple encodings can be sent
+    at once, but encodings need not be included if the stats haven't
+    changed.  See below.
+
+Audio encoding receiver stats include the following properties:
+
+: encoding-id
+:: The ID of the encoding for which the stats apply.
+
+: cumulative-decoded-frames
+:: The total number of audio frames received and decoded.
+
+: cumulative-received-duration
+:: The sum of all of the durations of all of the audio frames received.
+
+: cumulative-lost-duration
+:: The sum of all of the durations of all of the audio frames detected as lost.
+
+: cumulative-buffer-delay
+:: The sum of the time frames spent buffered between receipt and playout.
+
+: cumulative-decode-delay
+:: The sum of the time spent decoding frames received.
+
+Video encoding receiver stats include the following properties:
+
+: encoding-id
+:: The ID of the encoding for which the stats apply.
+
+: cumulative-decoded-frames
+:: The total number of video frames received and decoded.
+
+: cumulative-lost-frames
+:: The total number of video frames detected as lost.
+
+: cumulative-buffer-delay
+:: The sum of the time frames spent buffered between receipt and render.
+
+: cumulative-decode-delay
+:: The sum of the time spent decoding frames received.
+
 
 Requests, Responses, and Watches {#requests-responses--watches}
 ===============================================================