RGB rolling shutter artifacts coupled with FPS

[LostCanadian]

Camera Model: D415
Firmware Version: 5.11.01
OS: Win 10
SDK Version: 2.19.0

This issue has been reported before (see closed issues #2461 and #3169), but there seems to be a bug here that has not been addressed. Or, it's a fundamental hardware design issue; either way, it would be nice to know if this is something that can be fixed.

Essentially, the RGB camera's rolling shutter artifacts appear to be coupled with the FPS of the camera, when one would actually expect them to depend on the exposure. This is most obvious when setting the camera to 6 FPS and waving the sensor around, but is still an issue at 30 FPS albeit slight less visible. When doing the same test with the IR cameras (emitter disabled), the images do not exhibit rolling shutter artifacts even at 6 FPS and with equivalent effective exposure results. The units for the IR exposure do not seem to be directly comparable to the RGB exposure (why are they different??), so I had to eyeball it. Note that auto-exposure is disabled in both cases.

Here I am waving my hand around while capturing at 6 FPS. The RGB image has some major rolling shutter artifacts, while the IR image is just slightly blurry but otherwise OK:

For our use case, we map the RGB images on top of the depth data while capturing from a handheld device. We would normally try and resolve this type of issue by reducing the exposure time on the RGB cameras and increasing the ambient lighting, but as mentioned, this does not work.

So far, the "best" workaround I have found is to set the frame rate to 60 FPS, which effectively reduces the rolling shutter artifacts. However, we were aiming to capture RGB at 1920x1080, which is not possible at that frame rate. Right now, we are stuck with bad rolling shutter artifacts with 1920x1080 @ 30 FPS, or greatly reducing the resolution (848x480) so we can use 60 FPS.

To sum up, rolling shutter artifacts are tied to FPS with the RGB camera, but not the IR cameras. If possible, we would like to see the RGB camera behave the same way as the IR cameras, which is to decouple the frame rate from the exposure, at least in cases where auto-exposure is disabled.

[RealSenseCustomerSupport]

---

Hi

Yes, you statement is correct.
Under 6FPS, the horizontal and vertical timing configuration between Depth imager and RGB imager are different. The RGB rolling speed is configured at ~10x slower than depth camera.
However, when you increase frame rate to 30FPS, the RGB rolling speed is configured at 2x slower than Depth camera. This will improve a lot.
Please raise the bar of Gain to compensate the brightness.

Thanks

[LostCanadian]

Thanks for the response.

Under 6FPS, the horizontal and vertical timing configuration between Depth imager and RGB imager are different.

Can I make a feature request to have the RGB imager use the same timing configuration as the depth imager? This seems like it would greatly improve upon the rolling shutter issues seen with the RGB imager.

[RealSenseCustomerSupport]

---

I definitely understood your request, but due to some limitation (i.e counter size) inside of the chip, it is not able to make same timing for both cases.
I agree, there is some room to push though, but not too much.

Thanks

[suiye921]

I definitely understood your request, but due to some limitation (i.e counter size) inside of the chip, it is not able to make same timing for both cases.
I agree, there is some room to push though, but not too much.

Thanks

I have the same issue (RGB camera's rolling shutter artifacts coupled with FPS) in D435i. Does D435i has the same hardware limitation?
Thanks!

[szx0112]

Similar issue. See motion blur at 30 HZ when use d435 for SLAM applications. It seems that the default exposure (166) is set for 60 HZ. But 60 HZ seems to consume too much space.