Significant delay in "live stream"

[gurupras]

If I compare the output from the Ring Android app and custom code written with this library, there is at least a 5-10 second delay in the video received via camera.streamVideo(). Is this expected?

I tried the stream-example and also tried a custom UDP version with ffplay:

const output = [
  '-f', 'mpegts',  // Format for ffplay
  '-codec:v', 'libx264',  // Video codec
  '-r', '30',  // Frame rate
  '-b:v', '1500k',  // Bitrate
  'udp://127.0.0.1:5455'
]
const session = await camera.streamVideo({
  output
})

ffplay udp://127.0.0.1:5455

Edit: I am referring to the delay in the stream and not the time taken to run the script or to initially fetch and render the stream.
In other words, once the stream is up and running, the delay between the content shown in the Ring app and the content picked up by the example code is significant in my case - in the 5-10s range.

[tsightler]

I'm not sure what you mean by delay, do you mean stream latency, or do you mean startup time? Overall, comparing a Javascript WebRTC implementation with mulitple hops to a native WebRTC stream is of course going to provide some additional stream latency overall, however, the code is typically with 1 second of native WebRTC in my testing, and stream startup time is generally 2-3 seconds total, vs about 1-2 seconds for native WebRTC.

If you are talking about startup time, well, we'd need more data on exactly what your are comparing. If you are comparing running a script, which includes starting Node, initialization Javascript and all the dependencies, authenticating to the API, create the stream control channel, starting the WebRTC stream, firing up ffmeg, etc. to an app that already has all of this done.

[gurupras]

That's fair. I will update the original question, but I am referring to delay in the stream.

If I walk outside my house, wave my hand in front of the camera and walk back in and wait a couple of seconds, I can see my hand show up in front of my camera. Meanwhile, it shows up instantaneously in the Ring app that I am holding in my hand.

[tsightler]

No reason for this on the ring-client-api side. I notice you are re-transcoding, which will of course add delay, especially if you don't add options to reduce this. As the video stream is already H.264, best latency is achieved via ffmpeg copy mode. If you need to transcode for some reason, you should use standard tunes for live streams to reduce latency.

[gurupras]

I tried with a simpler ffmpeg command-line:

const output = [
  '-f', 'mpegts',  // Format for ffplay
  'udp://127.0.0.1:5455'
]
const session = await camera.streamVideo({
  output
})

Still no luck. I mesured the delay. It's about ~7 seconds. I am hoping for something that's as close to live as possible.

Another option seems to be camera.createSimpleWebRtcSession(), but there are no examples for this. Is this supported? Any chance there is some documentation or quick example that you (or someone else) can cook up that uses this?

[tsightler]

But with that you are still transcoding the stream, since you did not specify to copy (the default for ffmpeg is to transcode). I would suggest you look at the streaming code in both homebridge-ring and ring-mqtt, since both of those work fine with fairly low latency. If not, things like return audio would not be possible as 7 second latency is a nightmare.

But mostly I'm convinced you need to use copy mode with ffmpeg, that's likely the most important part. Transcoding with default settings will always add latency, likely 2-4 seconds with defaults, but varies with hardware, and then ffplay and others have their own buffers.