-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rewrite the Priority section to improve clarity #518
Changes from all commits
885030d
e40027d
f538b4b
3b197f0
22a9a62
bf9b2c0
ddafdd0
e4b452f
46dbf7e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -617,56 +617,70 @@ MoQ priorities allow a subscriber and original publisher to influence | |
the transmission order of Objects within a session in the presence of | ||
congestion. | ||
|
||
## Definitions | ||
|
||
MoQT maintains priorities between different _schedulable objects_. | ||
A schedulable object in MoQT is either: | ||
|
||
1. An object that belongs to a peep where that object would be the next object | ||
to be sent in that peep. | ||
1. An object that belongs to a track with delivery preference Datagram. | ||
vasilvv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Since a single peep or datagram has a single publisher priority, it can be | ||
useful to conceptualize this process as scheduling peeps or datagrams | ||
instead of individual objects on them. | ||
|
||
A _priority number_ is an unsigned integer with a value between 0 and 255. | ||
ianswett marked this conversation as resolved.
Show resolved
Hide resolved
|
||
A lower priority number indicates higher priority; the highest priority is 0. | ||
|
||
_Subscriber Priority_ is a priority number associated with an individual | ||
subscription. It is specified in the SUBSCRIBE message, and can be updated via | ||
SUBSCRIBE_UPDATE message. The subscriber priority of an individual schedulable | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would add here something along lines of "when it is changed, in a relay it is implementation dependent on how it will apply to object that the relay has already received". Clearly if a something had been pushed into the QUIC stack but not yet pushed to the wire, it would be pretty hard to change. I think it is fine to leave this very implementation dependent because this is something that is happening for a short time on the edge of the change and weather a given object gets the old or new priority during the switch over does not matter much to apps. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I tried to add a clarifying sentence, PTAL |
||
object is the subscriber priority of the subscription that caused that object | ||
to be sent. When subscriber priority is changed, a best effort SHOULD be made | ||
to change the apply that to all objects that have not been sent, but it is | ||
implementation dependent what happens to objects that have already been | ||
received and possibly scheduled. | ||
|
||
_Publisher Priority_ is a priority number associated with an indiviaul | ||
schedulable object. It is specified in the header of the respective peep or | ||
datagram, and is the same for all objects in a single peep. | ||
|
||
_Group Order_ is a property of an invidual subscription. It can be either | ||
'Ascending' (groups with lower group ID are sent first), or 'Descending' | ||
(groups with higher group ID are sent first). The publisher communicates its | ||
group order in the SUBSCRIBE_OK message; the subscriber can override it in its | ||
SUBSCRIBE message. The group order of an existing subscription cannot be | ||
changed. | ||
|
||
## Scheduling Algorithm | ||
|
||
When an MoQT publisher has multiple schedulable objects it can choose between, | ||
the objects SHOULD be selected as follows: | ||
|
||
1. If two objects have a different subscriber priority associated with them, | ||
the one with **the highest subscriber priority** is sent first. | ||
1. If two objects have the same subscriber priority, but a different publisher | ||
priority, the one with **the highest publisher priority** is sent first. | ||
Comment on lines
+663
to
+664
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This implies that publisher priority trumps subscriber group order. That was not how I read the previous algorithm. Higher publisher priority would select a subscription to send from, then you go through to group order. This is how we got min. I like the property that says group order is ascending or descending only, and priority doesn't influence group order. Trying to use the object priority as both intra and inter track is weird. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I agree that it might feel a little bit weird, but if priority is applied before group order, this makes sense, since this should be more expressive than just having two of those. As to publisher priority vs group order, I feel like for the use cases that I can think of, the group order should always apply after the publisher priority. E.g. for the active vs non-active speaker case, you'd always want to prioritize the more-useful audio over less-useful regardless of stream position; or for SVC, you'd probably rather have all of the base layer frames than have the newest group in the best quality and a missing group right before it. (Of course, even if that is not true, you can choose the opposite behavior by setting publisher priority as the same for all track, and then using peep IDs to prioritize those within the group.) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The problem that I see if that the publisher gets the final say on group order here -- if the subscriber asks for one track with pri=0 and order=asc, but the publisher sets the peep priorities for each group in descending order, the data will be delivered dsc. I'd find it more consistent and easier to think about if we separate publisher track priority from peep/object priority (Martin's Option 3). Then it's In any case, this PR doesn't feel like a clarification so much as a change to the algorithm. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Adding an explicit track priority is completely reasonable, though it'd be good to understand what cases it was necessary? For SVC, I'm quite sure you'd want the peep priority to be more important than group order. We only have 256 priority values, so your example won't last that long for most use cases. Also, your example is somewhat adversarial, so I'm not sure how realistic it is? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I created #519 to discuss the question of whether publishers should specify a priority in addition to the subscriber priority. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Looking at wills slides from seattle interim it is clear that publisher priority override group id other wise nearly all the uses cases that use publisher priority don't work. I think if we just step back and look at uses cases, we can reason our way though the conversation we have had several times and end up back at the space spot of what Victor has in the PR now There are not enough priorities codes for a publisher to even do the example Alan raised at top of this thread. I view that sort of use cases as just a broken application if the publisher and subscriber are trying to do totally conflicting things and does not bother me much that weird things happen in that case. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Individual Comment: I agree the Seattle proposal was: pick a track by sub priority, then min(pub-priority-of-pending-objects), select a group by group order, order by object ID. There's a complaint that min priority is too hard to implement. Sounds like the new proposal is: Pick a track by sub priority, then pub priority, then group by group order, order by peep/object ID. Here's what I think is a weird case: The subscriber asked for ascending groups. The new algorithm will deliver group 2 pri=90 objects first and that seems wrong. The Seattle algorithm would send group 1 object in obj ID order, then group 2 pri=90 objects, then group 2 pri=100 objects. This seemed more correct, if harder to implement. You can sort of fix this by incrementing the base priority of objects in every group, but you will run out of priority levels after 256 groups. There seemed to be general consensus that groups would only be delivered ascending or descending and I think we should try to preserve that property -- do people care less about this now? I'm inclined to think we need to set aside some time to discuss at the hybrid interim next month. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. One observation about ascending / descending order is that it seems the use case for it are all times where it is a fetch not a subscribe. It also seems that for flow control reason fetch will likely flow over one stream. I wonder if we could just remove descending order for subscribe. I've pointed out in the past it is totally useless and in most cases where the things are not congested and the subscribe is on the live edge, the subscribe will get things in the order the publisher published them regardless of how any of the priorities and orders are set. This will mean a subscriber asks for things in reverse order and will then get most things in forward order with perhaps a few object in reverse order then there was a bit of congestion. I don't object to it being here if others want it because it is all trivial to implement but I remain of the opinion order is unless for subscribe and critical for fetch. |
||
1. If two objects have the same subscriber and publisher priority, but belong | ||
to two different groups of the same track received through the same | ||
subscription, **the group order** of the associated subscription is used to | ||
decide the one that is sent first. | ||
1. If two objects belong to the same group of the same track received through | ||
the same subscription, the one with **the lowest Peep ID** (for tracks with | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not a big deal but I would have a preference of limiting peep ID to 8 bits if they are going to be used as one of the priority fields. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I do not see any reason to add peep into this. If the publisher wanted the object in one peep to be delivered ahead of another peep, it should have set the publisher priority. I can probably live with this but I'd like to understand why we need it. |
||
delivery preference Peep), or **the lowest Object ID** (for tracks with | ||
delivery preference Datagram) is sent first. | ||
|
||
This algorithm does not provide a well-defined ordering for objects that belong | ||
to different subscriptions, but have the same subscriber and publisher | ||
priority. The ordering in those cases is implementation-defined, though the | ||
expectation is that all subscriptions will be able to send some data. | ||
|
||
Given the critical nature of control messages and their relatively | ||
small size, the control stream SHOULD be prioritized higher than all | ||
subscribed Objects. | ||
|
||
Both SUBSCRIBE and FETCH messages indicate a subscriber priority and | ||
group order, so the following text applies equally to both types of | ||
subscriptions. | ||
|
||
The subscriber indicates the priority of a subscription via the | ||
Subscriber Priority field and the original publisher indicates priority | ||
in every stream or datagram header. As such, the subscriber's priority is a | ||
property of the subscription and the original publisher's priority is a | ||
property of the Track and the Objects it contains. In both cases, a lower | ||
value indicates a higher priority, with 0 being the highest priority. | ||
|
||
When Objects are contained in Subgroups, all Objects in the Subgroup have the same | ||
priority. | ||
|
||
The Subscriber Priority is considered first when selecting a subscription | ||
to send data on within a given session. When two or more subscriptions | ||
have equal subscriber priority, the original publisher priority is considered | ||
next and can change within the track, so subscriptions are prioritized based | ||
on the highest priority data available to send. For example, if the subscription | ||
had data at priority 6 and priority 10 to send, the subscription priority would | ||
be 6. When both the subscriber and original publisher priorities for a | ||
subscription are equal, how much data to send from each subscription is | ||
implementation-dependent, but the expectation is that all subscriptions will | ||
be able to send some data. | ||
|
||
The subscriber's priority can be changed via a SUBSCRIBE_UPDATE message. | ||
This updates the priority of all unsent data within the subscription, | ||
though the details of the reprioritization are implementation-specific. | ||
|
||
Subscriptions have a Group Order of either 'Ascending' or 'Descending', | ||
which indicates whether the lowest or highest Group Id SHOULD be sent first | ||
when multiple Groups are available to send. A subscriber can specify either | ||
'Ascending' or 'Descending' in the SUBSCRIBE message or they can specify they | ||
want to use the Original Publisher's Group Order, which is indicated in | ||
the corresponding SUBSCRIBE_OK. | ||
|
||
Within the same Group, and the same priority level, | ||
Objects with a lower Object Id are always sent before objects with a | ||
higher Object Id, regardless of the specified Group Order. If the group | ||
contains more than one Subgroup and the priority varies between these Subgroups, | ||
higher priority Subgroups are sent before lower priority Subgroups. If the specified | ||
priority of two Subgroups in a Group are equal, the lower Subgroup ID has priority. | ||
Within a Subgroup, Objects MUST be sent in increasing Object ID order. | ||
|
||
The Group Order cannot be changed via a SUBSCRIBE_UPDATE message, and | ||
instead an UNSUBSCRIBE and SUBSCRIBE can be used. | ||
## Considerations for Setting Priorities | ||
|
||
Relays SHOULD respect the subscriber and original publisher's priorities. | ||
Relays SHOULD NOT directly use Subscriber Priority or Group Order | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let me ask a questions first. If every object was a schedulable object, does it change the algorithm in the rest of this in any way ? When I read the rest of this I think the answer is No but I feel like others might feel the answer is yes and that I am missing something that is a key issue for other people.
The way I would implement this is put all the object on a priority queue where the priority on the q was an integer basically formed by concatenating subscriber priority, publisher priority, and either the group id or reverse group id based on direction, then peep id, then object id. Then when there is space available send an object, you grab the next object out of queue and and send it. I want to understand if this is wrong. If it is wrong, I am probably fine with what people want but I think it will help me get to understanding why people keep saying this is complicated. If this is right, then I don't think we need the concept of a schedulable object.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@vasilvv - This question here is the only question I have of significance on this PR. Love the PR - thanks.