Routing tracks correctly in an SFU

[srcrip]

Hello everyone,

I was wondering if someone could take a look at this and let me know if they have any idea what I'm doing wrong: https://github.com/srcrip/sfu

It's a simple little SFU demo. You can run it with iex, and connect to port 7001 in multiple tabs.

I think my signalling is correct, I have the correct number of tracks on the peers I expect, and I'm writing the packets the way I would expect, but I'm experiencing weird issues like:

• when a second peer connects, sometimes they get the track from peer 1 but it never gets video
• sometimes when a second peer connects it gets no video for like a minute, then suddenly starts getting it

I've looked at the sdp and it seems right to me. I feel like the problem is in writing the packets but I just don't know. I feel like I'm using the API correctly.

[LVala]

I'll try to take a look tomorrow

[LVala]

You don't seem to send a keyframe request when a new peer joins, which would explain why the joining peer cannot decode video from already present peers (or it starts to work fine after a while, when the browser decides to encode a new keyframe for unrelated reasons, probably). Read this if you don't know what a keyframe is.

When a new peer is added, you should prompt other peers to encode a keyframe by sending a FIR or PLI RTCP feedback to them (both will result in a new keyframe, although FIR is probably more semantically correct in this case).

I've added both FIR and PLI to ex_rtcp, but it hasn't been released yet so you have to use

{:ex_rtcp, github: "elixir-webrtc/ex_rtcp", override: true}

That's what I would start with.

[srcrip]

Thank you, I will look into that further. I knew the key frame thing was a problem from reading other SFUs but I didn't fully understand why the relaying streams needed to generate them. I assumed they'd just 'pass through' from one client to the other.

[srcrip]

Ah after thinking about it some more I think I get it. The video stream just consists of those non key frames for large portions so that's why the other end isn't getting usable video by just joining at a random time.

[srcrip]

Makes a lot more sense after I thought about it some more 😂

[LVala]

Closing, as I answered your question (hopefully), and the discussion has been inactive for a while.