HW time stamping: exact time relation between input/output trigger and exposure event

[m0oz]

Required Info
Camera Model	D435
Firmware Version	05.10.03.00
Operating System & Version	Linux (Ubuntu 16)
Kernel Version (Linux Only)	4.15.0
Platform	PC
SDK Version	2.16.0
Language	C++
Segment	Robot

Issue Description

What do I want to do?

I want to measure hw timestamps for each rs frame. E.g. I want to use an external platform (in my case a pixhawk) to measure the exact moment of exposure. I want to do this in order to prevent using the timestamps provided by the frames metadata as they do not have the same time reference as other sensors in the setup (e.g. an IMU) and are subject to drift.

What is the exact issue?

Since FW version 5.10 hw syncing multiple D435s is supported. Therefore, you can either trigger a D435 when in slave mode or use it to produce ouput triggers when in master mode. I do understand that this feature is merely designed to sync multiple cams and then use the timestamps of the master cam but I do not want to use this software timestamp for several reasons. So I would need the following info and could not find it in any issue or piece of documentation:

• When listening to output triggers: What is the time relation between when a D435 generates an output trigger and when the exposure of the related frame began/ended?

• When triggering externally: What is the time relation between when an external pulse is fed to a D435 and when the exposure of the related frame starts/ends?

• Is there any buffering going on that would cause an offset or delay between the end of exposure and the execution of the functor that is provided as sensor callback?

If you could help me with this timing issue I'd be more than glad.

Thank you!

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment]
1.The trigger was generated when the image frame was read out from the image sensor. in another word, the end of the exposure.

2. When triggering externally, the trigger pulse aligns with the frame start

3. There is no buffer. The delay is about 1~2 ms, due to the image data traveling time inside the ASIC.

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment]
Hi m0oz,

Do you have further questions?

[m0oz]

Thank you for the info!

Indeed, I have another question:
How should we setup the realsense so that frames are send out directly after the end of integration?
You said there is no buffer, but when I used the infra1 stream and the timestamps at which the output trigger is produced I always observed an offset of about one 1/fps seconds.

So what is my setup:
I am feeding the image stream as well as an IMU-track to kalibr. When enabling time-calibration this tool gives the time-shift between image stream and imu data stream.

At 30fps we observed about 33ms delay between output trigger and when the frame callback was executed.
At 15fps about 66ms,
At 6fps about 166ms and so forth.

Do you have and idea why frames could be held back before they are sent to the frame callback function that was set when calling sensor.start(callback)?