Current best practice re: RS2_OPTION_FRAMES_QUEUE_SIZE

[evercoast-smm]

Camera Model: D415
Firmware Version: 05.11.15.00
OS: Ubuntu 18.04 with kernel 4.18.20-041820-generic
Platform: PC
SDK Version: librealsense-2.29.0
Segment: Telecommunications

Question:

What's understood to the be the best practice relating to RS2_OPTION_FRAMES_QUEUE_SIZE, noting that there various documents indicate different things. For instance:

#2637

^^ The above states:

on(RS2_OPTION_FRAMES_QUEUE_SIZE, 1);.
In LabView app, @agrunnet is configuring each sensor directly with an output queue. In this case, you want to make sure you don't allow any unnecessary queuing. Frame queue size = 1 is default for C++ today (rs_processing.hpp:131)
I believe that using pipeline does not introduce noticeable extra-latency, so I think you should be better of with the default settings.

https://github.com/IntelRealSense/librealsense/wiki/Frame-Buffering-Management-in-RealSense-SDK-2.0

^^ While here we have (as stated in the white-paper as well):

To minimize latency in the SDK when streaming is configured, it is best to set the queue size (the “capacity” parameter) to 1 if enabling depth-only, and 2 if enabling both depth and color streams.

It's very important for us to have certainty that we're aligning to established best-practices.

Thanks!

[MartyG-RealSense]

A further example of use of the frame queue size can be found in the official documentation link below, under the 'librealsense2' heading. It suggests a value of '5', representing a maximum latency of 5 frames.

https://github.com/IntelRealSense/librealsense/wiki/API-How-To#do-processing-on-a-background-thread

As far as I understand it, the smaller the value used, the more performance you may get but the higher the risk of dropping frames.

[evercoast-smm]

Thank you, Marty. You've pointed out a fundamental flaw in my original question. Please allow me to be more precise:

In the case where color + depth streams are enabled, what is the best setting for RS2_OPTION_FRAMES_QUEUE_SIZE if the desired outcome is to not drop frames?

Also, would this answer change if there were multiple sensors using hardware-synchronization?

Please understand that the nature of my question is to ensure that we're adhering to current [late-2019] best practices, noting that there is conflicting documentation on the ideal setting for this option.

[MartyG-RealSense]

The suggestion of the frame buffering documentation is to use a value of '2' if you are using two streams. Ultimately though, I believe that this should be taken as guidance rather than an unbreakable rule though, since the frame queue size option encourages you to choose your own value based on your personal priorities - whether performance (low value) or greater stability through added latency (higher value) is important to you.

If you want to find a good balance between two-stream stability and minimizing frame drop, '3' might be a good compromise value, adding a little extra wait into the process without slowing it down too much. Try '2' first and see what kind of frame drop you get, and go to '3' if you think extra stability is needed.

Regarding multiple camera sync, my interpretation of the advice in the multicam white paper on this subject (section 3B) is that you should stick with the recommended values if possible - '1' for a single stream and '2' for two streams - as using a larger buffer value may introduce problems.

[dorodnic]

I believe this should be fairly accurate advise:

1. When using rs2::frame_queue directly (this is what @agrunnet is doing in the white-papers) pass some small number to restrict the queue from accumulating latency. Usually one queue per stream type is recommended, and then the value of 1 is sufficient. Depth sensor, however, may produce multiple frames (depth and infrared) and if a queue must be shared it should be initialised with larger value (2,3).

2. We would strongly recommend not to change RS2_OPTION_FRAMES_QUEUE_SIZE on the sensor object. This option does not strictly configures any particular queue but rather limits the total number of frames in circulation (limiting total queue size of all queues involved). The thing is, all internal queues are already fined-tuned for min latency, and getting it wrong can introduce stream-breaking bug.

[evercoast-smm]

Thank you both. Part 2 of @dorodnic 's above answer satisfies my question. Closing.