Frame Drops per Second

[a-m-p-m]

How can it be that RealSense Viewer shows stable fps close to 90 for 848x480@90 fps stream and simultaneously shows "Frame Drops per Second" on the level of 42%?

RealSense Viewer 2.38.1 is shown in the image, but the question is valid for many other RSV versions too.

[MartyG-RealSense]

Hi @a-m-p-m I tested on my laptop with Radeon graphics and 20 GB memory and got 60% to 100% frame drop from the Frame Drops Per Second panel at 90 FPS. However, FPS remained constant and the stream did not stutter at all.

I would therefore recommend using FPS and the visual feedback of the stream on-screen as a primary benchmark of the performance stability of active streams.

[a-m-p-m]

How can one rely on the fps shown by RealSense Viewer (RSV)?  The way fps is calculated in RSV does not capture any frame drops and will always result in the fps that is equal to the one set by the user.  

The reason is that RSV uses timestamps and frame numbers generated for the frames by the camera to calculate fps.  

RSV calculates fps by getting, from the frame metadata, timestamps and frame numbers for two frames received from the camera, calculating a difference between the timestamps and dividing it by a difference between the frame numbers.

Suppose the camera generated 1000 frames at a certain constant fps, FPS1.  The camera assigned frame numbers from 1 to 1000 to those frames.  Suppose  frames 2-999 were lost on their way from the camera to a computer or in the RSV itself.  RSV would receive frames 1 and 1000, perform fps calculation outlined above and, lo and behold, would obtain the perfect value of FPS1.  The information which in no way reflects that 998 frames have been lost somewhere.  

Perhaps, RSV should assign frame numbers to the frames it receives itself and use those numbers in the fps calculation.  At least that way fps would reflect frame drops.  

[MartyG-RealSense]

Historically, the realtime-updating FPS value is proven as a highly reliable indicator of a stream's performance from moment to moment. The FPS certainly can vary - down or up - depending on the conditions that are currently active. For example, manual exposure values can affect FPS, as described in the link below.

#1957

Having said that, an apparently stable FPS value does not guarantee that frames will not be dropped. The link below discusses the balance of latency versus performance, where favoring performance may reduce latency but increase the chance of dropped frames.

https://github.com/IntelRealSense/librealsense/wiki/Frame-Buffering-Management-in-RealSense-SDK-2.0#latency-vs-performance

[a-m-p-m]

It looks like I didn't get my point about the fps in RSV across. In a nutshell, the fps value that RSV shows is meaningless and misleading. Because of the way it is calculated, it can't in principle capture any fps variation on the RSV's end due to frame drops, for example.

"an apparently stable FPS value does not guarantee that frames will not be dropped"
Wouldn't that be a problem when something shows you an "apparently stable fps" while, in reality, frames are dropping and you are unaware about that happening?

And it is exactly the problem with RSV that it shows absolutely stable fps while frames are dropping. All because of the way RSV calculates fps values, as I explained in my previous comment.

[MartyG-RealSense]

I am not on the RealSense developer team so I cannot comment on the reasons for the way in which the SDK implements FPS. One of the developer team members such as @dorodnic or @ev-mp will be better able to explain why the SDK implements FPS in the way that it does.

[a-m-p-m]

Thanks. It would be nice to hear what @dorodnic and @ev-mp can say about the way RealSense Viewer calculates fps.

[dorodnic]

Hi @a-m-p-m
Frame drops are calculated here:
https://github.com/IntelRealSense/librealsense/blob/master/common/output-model.cpp#L1099-L1108
FPS is calculated using:
https://github.com/IntelRealSense/librealsense/blob/master/common/rendering.h#L46-L99

Since FPS is normalised by frame-number, it basically reports FPS that camera is generating. If frames are lost at USB / OS level, or if Actual FPS metadata is not in line with real actual FPS, this will be reported in frame-drops calculator.

Also Frame drops are more strict with regards to latency. If frames do not arrive at regular intervals, it will be reflected as drops but not impact FPS.

[MartyG-RealSense]

Hi @a-m-p-m Do you still require assistance with this case, please? Thanks!

[a-m-p-m]

@MartyG-RealSense, @dorodnic, thank you for the information.

[MartyG-RealSense]

Case closed due to no further comments received.