Test 82 failing under Windows in Debug configuration

[claudiofantacci]

Here is the error:

82: Test command: C:\Users\cfantacci\GitHub\pcl\buildd\bin\test_recognition_cg.exe "C:/Users/cfantacci/GitHub/pcl/test/milk.pcd" "C:/Users/cfantacci/GitHub/pcl/test/milk_cartoon_all_small_clorox.pcd"
82: Test timeout computed to be: 9.99988e+06
82: User Error 1001:
82: argument to num_threads clause must be positive
82:
82: User Error 1001:
82: argument to num_threads clause must be positive
82:
82: User Error 1001:
82: argument to num_threads clause must be positive
82:
82: User Error 1001:
82: argument to num_threads clause must be positive
82:
82: User Error 1001:
82: argument to num_threads clause must be positive
82:
82: User Error 1001:
82: argument to num_threads clause must be positive
82:
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6812
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6822
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6836
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6874
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 5978
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 5986
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 5988
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7091
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7173
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 4189
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7205
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6205
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6234
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7253
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6240
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7273
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7281
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7293
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7318
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7321
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7331
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7343
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 5340
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7345
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7349
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7350
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7355
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7364
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7373
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7390
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7401
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7420
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7422
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7436
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7440
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7449
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 5400
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 5410
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6403
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7468
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7493
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 5447
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7507
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7513
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6463
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6465
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7532
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7559
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7588
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7603
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7605
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7629
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7636
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7641
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7646
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7664
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 5575
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7683
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7692
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7703
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7704
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6606
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 5606
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7720
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7729
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7732
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7735
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7736
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7737
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7745
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7751
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7752
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7753
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 7755
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6673
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 5720
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6756
82: [pcl::SHOTEstimation::computeFeature] The local reference frame is not valid! Aborting description of point with index 6772
82: [==========] Running 2 tests from 1 test case.
82: [----------] Global test environment set-up.
82: [----------] 2 tests from PCL
82: [ RUN      ] PCL.Hough3DGrouping
82: ..\test\recognition\test_recognition_cg.cpp(134): error: Value of: clusterer.recognize (rototranslations)
82:   Actual: false
82: Expected: true
82: ..\test\recognition\test_recognition_cg.cpp(137): error: Expected: (rototranslations.size ()) >= (1), actual: 0 vs 1
82: [  FAILED  ] PCL.Hough3DGrouping (17562 ms)
82: [ RUN      ] PCL.GeometricConsistencyGrouping
82: [       OK ] PCL.GeometricConsistencyGrouping (59199 ms)
82: [----------] 2 tests from PCL (76761 ms total)
82:
82: [----------] Global test environment tear-down
82: [==========] 2 tests from 1 test case ran. (76761 ms total)
82: [  PASSED  ] 1 test.
82: [  FAILED  ] 1 test, listed below:
82: [  FAILED  ] PCL.Hough3DGrouping
82:
82:  1 FAILED TEST
5/7 Test  #82: a_recognition_cg_test ............***Failed  142.73 sec

User Error 1001 does not interfere with the test. This error can be fixed by checking whether the number of threads assigned to OpenMP is 0 (which is the default if no number is specified) or not and by setting them to a number greater than 0.

Unfortunately, however, the real reasons to which this test fails cannot be easily fixed.
This line

pcl/test/recognition/test_recognition_cg.cpp

Line 134 in a602b19

EXPECT_TRUE (clusterer.recognize (rototranslations));

ends up calling this function

pcl/recognition/include/pcl/recognition/impl/cg/hough_3d.hpp

Line 272 in a602b19

hough_space_->findMaxima (hough_threshold_, max_values, max_ids);

that finds the maximum in hough_space_ given a threshold. hough_space_ differs a lot between Release and Debug configuration and may be due to the use of NormalEstimation class that provides different results as well, thus conditioning the results provided by SHOTEstimation, see

pcl/test/recognition/test_recognition_cg.cpp

Lines 206 to 217 in a602b19

	//Descriptor
	SHOTEstimationOMP<PointType, NormalType, DescriptorType> descr_est;
	descr_est.setRadiusSearch (0.015);
	descr_est.setInputCloud (model_downsampled_);
	descr_est.setInputNormals (model_normals_);
	descr_est.setSearchSurface (model_);
	descr_est.compute (*model_descriptors_);
	descr_est.setInputCloud (scene_downsampled_);
	descr_est.setInputNormals (scene_normals_);
	descr_est.setSearchSurface (scene_);
	descr_est.compute (*scene_descriptors_);

By having a look at the numbers that comes out from hough_space_, it seems that by changing the threshold from 13 to 12 the test passes also in Debug. Please note that this is a workaround, but from previous discussion it seems that normal estimation repeatability is undergoing investigation per se.

See further discussions in #2399

[SergioRAgostinho]

Thanks for digging into it 👍 That procedure of lowering the threshold was employed before in #2507. We can do it again, but it would be good to stress test it over a big amount of test runs. Just to ensure that the new threshold will be enough to solve the problem

[claudiofantacci]

Thanks for digging into it 👍 That procedure of lowering the threshold was employed before in #2507. We can do it again, but it would be good to stress test it over a big amount of test runs. Just to ensure that the new threshold will be enough to solve the problem

I tried several run on my machine and found out that in Release I have two over-threshold correspondences of about 19 and 25 (which indeed was the first threshold changed in #2507), while in Debug the only seemingly good correspondence is above 12. I also checked whether there were other numbers around 10 and there were none. If we want to be "confident" for the test to pass, we can set it to 10. Just let me know about it 👍

[SergioRAgostinho]

Sounds good by me. Let's go with that 👍