[bug] Initializer::ReconstructH wrong

[joyousrabbit]

Hello, I'm not very good at maths. Just after studied < Deeper understanding of the homography decomposition for vision-based control > (Ezio Malis, Manuel Vargas,2007), and also did some tests, I think that Initializer::ReconstructH cannot work as expected.

As in the code, one of the conditions to return true from ReconstructH is: secondBestGood<0.75*bestGood

However,among the 8 solutions of Homography decomposition, 2 of them are completely different but both give a good physical and numerical meaning. Without prior knowledge or the 3rd image, we cannot tell which is the true one.

The consequence is clear: secondBestGood always equals bestGood. And ReconstructH will always return false. Only when the planar is very noised (not a planer any more), it can have the chance to return true.

Simply add inside the cycle " cout<<nGood<<endl; ", by dataset freiburg1_xyz, we can see:

rgb/1305031102.275326.png
rgb/1305031102.311267.png
H
157
3
157
82
0
0
0
0

rgb/1305031102.343233.png
H
63
153
0
153
0
0
0
0

rgb/1305031102.375329.png
H
138
0
138
45
0
0
0
0

rgb/1305031102.411258.png
H
133
0
133
20
0
0
0
0

rgb/1305031102.443271.png
H
107
0
107
5
0
0
0
0

rgb/1305031102.475318.png
H
114
0
114
4
0
0
0
0

rgb/1305031102.511219.png
H
2
104
0
104
0
0
0
0

rgb/1305031102.543220.png
H
2
93
0
93
0
0
0
0

rgb/1305031102.575286.png
H
1
83
0
83
0
0
0
0

rgb/1305031102.611233.png
H
90
0
90
3
0
0
0
0

rgb/1305031102.643265.png
rgb/1305031102.675285.png
rgb/1305031102.711263.png
H
198
47
198
87
0
0
0
0

rgb/1305031102.743234.png
H
168
0
168
26
0
0
0
0

rgb/1305031102.775472.png
H
145
0
145
17
0
0
0
0

rgb/1305031102.811232.png
H
134
4
134
0
0
0
0
0

rgb/1305031102.843290.png
H
3
127
0
127
0
0
0
0

rgb/1305031102.875363.png
H
1
98
0
98
0
0
0
0

rgb/1305031102.911185.png
H
105
0
105
2
0
0
0
0

rgb/1305031102.943229.png
H
3
83
0
83
0
0
0
0

rgb/1305031102.975203.png
rgb/1305031103.011215.png
rgb/1305031103.043227.png
H
168
0
168
161
0
0
0
0

rgb/1305031103.075319.png
H
128
0
128
54
0
0
0
0

rgb/1305031103.111240.png
H
0
151
7
151
0
0
0
0

rgb/1305031103.143318.png
H
138
2
138
0
0
0
0
0

rgb/1305031103.175452.png
H
119
0
119
1
0
0
0
0

rgb/1305031103.211220.png
H
0
80
0
80
0
0
0
0

rgb/1305031103.243216.png
H
0
81
0
81
0
0
0
0

rgb/1305031103.275370.png
rgb/1305031103.311210.png
rgb/1305031103.343223.png
H
138
125
138
137
3
2
2
3

rgb/1305031103.375327.png
H
127
119
133
134
5
3
3
5

rgb/1305031103.411260.png
H
21
137
137
137
0
2
2
0

rgb/1305031103.443280.png
H
0
132
140
140
0
0
0
0

rgb/1305031103.475274.png
H
121
125
20
107
1
0
0
1

rgb/1305031103.511333.png
H
0
83
121
121
0
3
3
0

rgb/1305031103.543444.png
H
0
43
91
91
0
0
0
0

rgb/1305031103.575474.png
H
102
102
7
0
0
0
0
0

rgb/1305031103.611223.png
H
86
86
7
0
0
0
0
0

rgb/1305031103.643445.png
H
88
88
2
0
0
0
0
0

rgb/1305031103.675523.png
H
79
79
1
0
0
0
0
0

rgb/1305031103.711610.png
H
0
2
74
74
0
0
0
0

rgb/1305031103.743326.png
rgb/1305031103.775342.png
rgb/1305031103.811242.png
H
175
175
144
0
0
0
0
0

rgb/1305031103.843251.png
H
0
50
119
119
0
0
0
0

rgb/1305031103.875361.png
H
119
119
7
0
0
0
0
0

rgb/1305031103.911221.png
H
0
0
112
112
0
0
0
0

rgb/1305031103.943211.png
H
117
117
1
0
0
0
0
0

rgb/1305031103.975373.png
H
0
1
99
99
0
0
0
0

rgb/1305031104.011232.png
rgb/1305031104.043249.png
rgb/1305031104.075425.png
H
146
146
129
0
0
0
0
0

rgb/1305031104.111235.png
H
140
140
0
81
0
0
0
0

rgb/1305031104.143230.png
H
152
152
11
0
0
0
0
0

rgb/1305031104.175424.png
H
117
117
14
0
0
0
0
0

rgb/1305031104.211283.png
H
127
127
2
0
0
0
0
0

rgb/1305031104.243196.png
H
120
120
1
0
0
0
0
0

rgb/1305031104.275546.png
H
104
104
0
0
0
0
0
0

rgb/1305031104.311219.png
H
92
92
1
0
0
0
0
0

rgb/1305031104.343342.png
H
90
90
1
0
0
0
0
0

rgb/1305031104.375837.png
rgb/1305031104.411509.png
rgb/1305031104.443288.png
H
192
192
110
112
0
0
0
0

rgb/1305031104.475456.png
H
168
168
62
0
0
0
0
0

rgb/1305031104.511329.png
H
0
21
140
140
0
0
0
0

rgb/1305031104.543368.png
H
134
134
20
0
0
0
0
0

rgb/1305031104.575343.png
H
94
94
0
3
0
0
0
0

rgb/1305031104.611336.png
rgb/1305031104.643243.png
rgb/1305031104.675525.png
F
rgb/1305031104.711277.png
H
100
100
1
43
0
0
0
0

rgb/1305031104.743280.png
H
99
99
63
0
0
0
0
0

rgb/1305031104.775327.png
rgb/1305031104.811465.png
rgb/1305031104.843258.png
H
39
111
115
115
0
1
1
0

rgb/1305031104.875350.png
F
rgb/1305031104.911534.png
rgb/1305031104.943262.png
rgb/1305031104.975202.png
H
168
162
172
172
1
0
0
1

rgb/1305031105.011290.png
H
31
127
124
128
0
1
1
0

rgb/1305031105.043373.png
H
124
33
124
28
0
0
0
0

rgb/1305031105.075320.png
H
0
110
16
110
0
0
0
0

rgb/1305031105.111299.png
H
60
109
0
109
0
0
0
0

rgb/1305031105.143106.png
rgb/1305031105.175159.png
rgb/1305031105.211268.png
H
58
93
0
93
0
0
0
0

rgb/1305031105.243270.png
F
rgb/1305031105.275288.png
H
77
0
77
22
0
0
0
0

rgb/1305031105.311290.png
rgb/1305031105.343302.png
rgb/1305031105.375338.png
H
132
144
0
144
0
0
0
0

rgb/1305031105.411286.png
H
40
136
0
136
0
0
0
0

rgb/1305031105.443316.png
H
128
0
128
55
0
0
0
0

rgb/1305031105.475280.png
H
113
0
113
13
0
0
0
0

rgb/1305031105.511332.png
H
117
14
117
0
0
0
0
0

rgb/1305031105.543282.png
H
95
0
95
7
0
0
0
0

rgb/1305031105.575449.png
H
91
0
91
5
0
0
0
0

rgb/1305031105.611378.png
H
2
84
0
84
0
0
0
0

rgb/1305031105.643273.png
H
82
0
82
2
0
0
0
0

rgb/1305031105.675166.png
rgb/1305031105.711309.png
rgb/1305031105.743312.png
H
153
19
153
72
0
0
0
0

rgb/1305031105.775339.png
H
156
43
156
0
0
0
0
0

rgb/1305031105.811283.png
H
146
0
146
16
0
0
0
0

rgb/1305031105.843271.png
H
125
2
125
0
0
0
0
0

rgb/1305031105.875337.png
H
0
83
2
83
0
0
0
0

rgb/1305031105.911262.png
H
83
0
83
0
0
0
0
0

rgb/1305031105.943272.png
rgb/1305031105.975329.png
rgb/1305031106.011285.png
H
97
164
0
164
0
0
0
0

rgb/1305031106.043355.png
H
142
0
142
12
0
0
0
0

rgb/1305031106.075330.png
H
28
139
0
139
0
0
0
0

rgb/1305031106.111327.png
H
5
104
0
104
0
0
0
0

rgb/1305031106.143355.png
H
10
109
0
109
0
0
0
0

rgb/1305031106.175534.png
H
5
81
0
81
0
0
0
0

rgb/1305031106.211275.png
H
70
0
70
3
0
0
0
0

rgb/1305031106.243267.png
rgb/1305031106.276385.png
rgb/1305031106.311238.png
H
24
152
128
152
0
0
0
0

rgb/1305031106.343258.png
F
rgb/1305031106.375388.png
H
84
21
84
0
0
0
0
0

rgb/1305031106.411320.png
H
117
59
117
0
0
0
0
0

rgb/1305031106.443278.png
H
129
1
129
63
0
0
0
0

rgb/1305031106.475345.png
H
68
131
18
131
0
0
0
0

rgb/1305031106.511129.png
H
109
12
109
58
0
0
0
0

rgb/1305031106.543302.png
H
140
140
0
98
1
0
0
1

rgb/1305031106.575282.png
H
0
19
131
131
0
0
0
0

rgb/1305031106.611151.png
H
114
114
9
0
0
0
0
0

rgb/1305031106.643207.png
H
0
2
113
113
0
0
0
0

rgb/1305031106.675279.png
H
0
4
77
77
0
0
0
0

rgb/1305031106.711508.png
rgb/1305031106.743341.png
rgb/1305031106.775390.png
H
0
95
149
149
0
0
0
0

rgb/1305031106.811289.png
H
0
46
111
111
0
0
0
0

rgb/1305031106.843416.png
H
0
5
120
120
0
0
0
0

rgb/1305031106.875905.png
H
0
11
90
90
0
0
0
0

rgb/1305031106.911243.png
rgb/1305031106.943439.png
rgb/1305031106.975547.png
H
139
139
0
105
0
0
0
0

rgb/1305031107.011576.png
H
0
40
134
134
0
0
0
0

rgb/1305031107.043281.png
H
137
137
10
0
0
0
0
0

rgb/1305031107.075432.png
H
102
102
0
0
0
0
0
0

rgb/1305031107.111229.png
H
0
1
123
123
0
0
0
0

rgb/1305031107.143260.png
H
91
91
2
0
0
0
0
0

rgb/1305031107.175399.png
rgb/1305031107.211358.png
rgb/1305031107.243378.png
H
0
102
142
142
0
0
0
0

rgb/1305031107.275398.png
H
154
154
0
111
0
0
0
0

rgb/1305031107.311226.png
H
0
16
115
115
0
0
0
0

rgb/1305031107.343509.png
H
0
1
99
99
0
0
0
0

rgb/1305031107.375413.png
H
0
15
115
115
0
0
0
0

rgb/1305031107.411271.png
H
97
97
12
0
0
0
0
0

rgb/1305031107.443419.png
H
9
0
87
87
0
0
0
0

rgb/1305031107.475377.png
H
87
87
0
9
0
0
0
0

rgb/1305031107.511352.png
H
90
90
10
0
0
0
0
0

rgb/1305031107.543605.png
rgb/1305031107.575454.png
rgb/1305031107.611271.png
H
195
195
114
121
0
0
0
0

rgb/1305031107.643323.png
H
60
141
144
144
0
0
0
0

rgb/1305031107.675568.png
H
150
149
150
3
0
2
2
0

rgb/1305031107.711307.png
H
26
149
159
160
0
2
2
0

rgb/1305031107.743538.png
H
31
156
140
162
0
2
2
0

rgb/1305031107.775802.png
H
181
157
177
0
0
3
3
0

rgb/1305031107.811596.png
H
0
147
110
147
0
1
1
0

rgb/1305031107.843332.png
H
0
147
111
147
0
0
0
0

rgb/1305031107.875358.png
H
109
143
0
143
1
0
0
1

rgb/1305031107.911541.png
H
130
0
130
54
0
0
0
0

rgb/1305031107.943122.png
H
0
114
0
114
0
0
0
0

rgb/1305031107.975807.png
H
102
9
102
0
0
0
0
0

rgb/1305031108.011320.png
F
rgb/1305031108.043418.png
rgb/1305031108.075352.png
rgb/1305031108.111378.png
H
112
14
112
98
2
0
0
2

rgb/1305031108.143334.png
F
rgb/1305031108.176058.png
rgb/1305031108.211475.png
rgb/1305031108.243347.png
H
85
40
85
52
0
0
0
0

rgb/1305031108.275358.png
H
105
45
105
30
0
0
0
0

rgb/1305031108.311332.png
H
25
79
10
79
0
0
0
0

rgb/1305031108.343278.png
rgb/1305031108.375410.png
rgb/1305031108.411361.png
H
154
119
154
103
1
1
1
1

rgb/1305031108.443610.png
H
16
97
38
97
0
3
3
0

rgb/1305031108.475471.png
rgb/1305031108.511378.png
rgb/1305031108.543737.png
H
97
49
97
26
0
0
0
0

rgb/1305031108.575414.png
F
rgb/1305031108.611407.png
F
rgb/1305031108.643303.png
rgb/1305031108.675375.png
rgb/1305031108.711411.png
H
0
107
103
107
0
0
0
0

rgb/1305031108.743502.png
H
0
65
30
65
0
0
0
0

rgb/1305031108.775493.png
F
rgb/1305031108.811244.png
rgb/1305031108.843264.png
rgb/1305031108.876515.png
H
148
148
148
148
0
0
0
0

rgb/1305031108.911364.png
H
108
114
77
114
1
0
0
1

rgb/1305031108.943243.png
H
80
80
77
1
0
1
1
0

rgb/1305031108.975268.png
H
78
78
60
0
0
1
1
0

rgb/1305031109.011269.png
H
104
104
88
0
0
0
0
0

rgb/1305031109.043277.png
H
93
93
63
0
0
1
1
0

rgb/1305031109.075410.png
H
75
75
36
0
0
3
3
0

rgb/1305031109.111282.png
H
2
6
75
75
0
0
0
0

[raulmur]

In our T-RO paper: http://webdiis.unizar.es/~raulmur/MurMontielTardosTRO15.pdf
and in: " O. D. Faugeras and F. Lustman, “Motion and structure from motion in a piecewise lanar environment,” International Journal of Pattern Recognition and Artificial Intelligence
, vol. 2, no. 03, pp. 485–508, 1988.", you will find the answer.

[joyousrabbit]

Hello Raulmur. Thank you for response. I read your excellent paper and the related ones. In your paper it mentions "waiting until significant parallax to eliminate the twofold ambiguity". However, according to my test, even the parallax is quite significant, the twofold ambiguity is still there. What's more, it's very long, must wait hundreds of frames to get such significant parallax.

Why not use 3 frames to do it? Even remove the condition "secondBestGood<0.75*bestGood" and let the following LOST check can initialize the system in several frames instead of hundreds.

[joyousrabbit]

Or maybe at Initializer::checkRT : 890, it should be:
if(cosParallax<0.99998){
vbGood[vMatches12[i].first]=true;
nGood++;
}

[rikpires]

Hi @joyousrabbit ,
I'm not quite sure about the twofold ambiguity, so I cannot answer your question directly. But in my understanding, in ORB-SLAM, the author prefers to retrieve the pose from fundamental matrix which should be success in a significant parallax and sufficient non-coplanar feature matches. In my practice, the initialization is pretty fast in a complex scene.

[joyousrabbit]

Hi, @rikpires
That's because your scene is complex and always trigger the Fundmental step. Could you test a planar scene for example against the wall or ground? If I'm right it will never be initialized.

[joyousrabbit]

Hi, @raulmur
Why a significant parallax check can tell the true one from the false? The parallax depends the degree of the camera to that planar. If the Z-axis is nearly parallel to the test plane, the little parallax should win (because most points are quite far). Instead, if the z-axis is perpendicular to the test plane, the bigger parallax wins.

[odiseo123]

Hi @joyousrabbit,
I am working on sequences where a large parallax is difficult to achieve (fulcrum effect). Do you have an update on the changes you made to the initialization step?

Thanks a lot,