ROS nodes delay

[tsampazk]

General Description

Some ROS nodes exhibit delay between the current frame and the frame that is being published.

Cause

This is because in most (all?) nodes the subscriber which uses the callback that performs inference, has a queue_size of 10. If the callback inference is slow, the subscriber keeps the incoming frames and processes them all one-by-one. When processing of one frame ends, the callback grabs an old frame from inside the queue and so on, causing the delay in the output.

Expected behaviour and solution

The expected behaviour would be to process the newest, most update-to-date frame every time the callback grabs a frame for inference and not use older frames. This is done by setting the queue_size to 1, but to properly work in ROS 1 you also need to set the buffer_size accordingly. I found that a value of 5000000 for the buffer size works well on object detection 2D. The delay is minimized to whatever time it takes for one frame to process which is expected.

Edit: Some All nodes require a larger buffer size to minimize delay, e.g. 10000000.

Edit: For ROS2 it is not required to set the buffer size, just add 1 as the queue size as seen here.

[tsampazk]

I'll document my understanding below for future reference. After the fixes you should expect a delay of 2 frames. This is because, as far as i can tell, we have to work with a minimum buffer of size 1, plus the frame that was already being processed. In more detail:

At times t, t + 1, etc. processed frames are being published. Keep in mind that frames are coming in as fast as the camera being used allows. Irrelevant frames are omitted.

1. A reference visible movement starts at tm and is completed before t + 1, e.g. a subject person raised their hand quickly (movement is complete before t + 1) and holds it in position until t + 1.
2. The critical frame Fm that shows the person with their hand raised is captured at t+1 and is stored in the buffer (that's when F2 is taken from the buffer to process and the buffer is empty). At the same time F1 is published.
3. F2 is published at t+2 and processing of Fm begins.
4. Finally, Fm is published at t+3 with a total delay of the time it takes for 2 frames to process.

[passalis]

I think this explains the expected behavior in a very nice way. We can perhaps mention this issue in the documentation (or even move the diagram and explanation there).