IMU initialization at high velocity

[tigerschweiger]

When i ran the ORBSLAM3 in a city-center scenario, it occured the following problems:

Fail to track local map!
IMU is not or recently initialized. Reseting active map...
LM: Active map reset recieved
LM: Active map reset, waiting...
LM: Reseting current map in Local Mapping...
LM: End reseting Local Mapping...
LM: Reset free the mutex
LM: Active map reset, Done!!!
mnFirstFrameId = 1
mnInitialFrameId = 4913
0 Frames set to lost
not IMU meas
First KF:1269; Map init KF:1268
New Map created with 112 points

I guess the system crashed and need to reset, but at that moment the car ran at a high velocity so the IMU had not enough acceleration that it could not be successfully initialized. Were there any solutions to solve this problem?

[jdtardos]

Please, read the conclusions in ORB-SLAM3 paper. IMU initialization requires acceleration and tilting the sensor w.r.t. gravity. For a car we recommend using stereo.

[YanhaoZhang]

Dear Prof. @jdtardos, thanks a lot for this reply. I find this is very useful for my case. Could you please give me more details on why, for a monocular-inertial system, the IMU initialization becomes difficult for a car moving in a flat area? I was wondering if there are any papers, or any potential ways to solve this (other than using a depth prediction network)? Thanks a lot. I am very interested in this because this is very common for autonomous driving when we only have one camera or multiple cameras, but without any overlapping among each of them. :)

[makolele12]

@YanhaoZhang I have been working with ORB SLAM for several months now. I am also testing car-sequences in Monocular-Inertial Mode.

As far as I can tell, the main problem with the IMU initialization is motion. If you take a look closely to the inertial optimizations, you will notice they always fail after a couple of frames. TrackLocalMap is where the failure occurs.

My guess is that this happens because of two main reasons:

• When driving straight, the Gyroscope readings in the 3 axes are almost 0 (as we are not turning).
• When driving in a flat street, the accelerometer captures mostly readings in one axis but if you maintain a constant speed, the accelerometer readings will be pretty small as well. On the Y-axis the accelerometer will be capturing the gravity force, but it remains also constant.

In my experience, only when the car is turning, I could achieve the IMU initialization. And it makes sense because in that case the gyroscope captures some motion. However, after turning and driving straight again, you get again the same motion problem, and I noticed how localization accuracy degrades over time.

[YanhaoZhang]

Hi @makolele12, sorry I just saw your reply. What you write is very helpful for me. Many thanks! :)

Do you mean, based on your experience, even IMU is successfully initialized when the car is tuning, it also will introduce some problems when the motion becomes straight or flat later? That is interesting. This means IMU is not a good sensor for autonomous driving, right?

Besides, orb-slam3 suggests using stereo-inertial for autonomous driving. But I do not know how stereo benefits the robustness. My guess is that the global scale from stereo helps. But I do not know the reason. May I ask you for any opinions on this? Thank you. :)

[mascomen4]

Thanks for the discussion! I've been struggling for a week now with Complex Urban Dataset trying to find the mistake in .yaml file. Apparently, it's the ORB-SLAM3s' issue

[makolele12]

Hi @makolele12, sorry I just saw your reply. What you write is very helpful for me. Many thanks! :)

Do you mean, based on your experience, even IMU is successfully initialized when the car is tuning, it also will introduce some problems when the motion becomes straight or flat later? That is interesting. This means IMU is not a good sensor for autonomous driving, right?

Besides, orb-slam3 suggests using stereo-inertial for autonomous driving. But I do not know how stereo benefits the robustness. My guess is that the global scale from stereo helps. But I do not know the reason. May I ask you for any opinions on this? Thank you. :)

Hi @YanhaoZhang, yes based on my experience even if the IMU got initialized while turning, if you keep on driving straight, the accuracy will degrade over time and eventually TrackLocalMap will fail again.

Yes, I also believe that the IMU alone is not appropriate for autonomous driving. I think it could help if other sensors are used, maybe when combining it with some kind of odometry sensors. But just Monocular+IMU seems to be difficult because of the driving motions.

I have also tested it in stereo-inertial and yes you are right, the optimizations leave the scale out, so with one less variable to optimize, it works better. However, after also testing it, I've found out that the map gets initialized longer, but it eventually fails after a while.