Something is (still) wrong with the binning indices in OUR-CVFH feature computation

[translunar]

I'm getting an odd memory corruption error when training OUR-CVFH under certain circumstances. This is with the 3d rec framework, and when I turn off MLS, I get a SIGABRT:

*** Error in `/home/jwoods/pose/build/pose': double free or corruption (out): 0x0000000000b34000 ***
Program received signal SIGABRT, Aborted.

0x00007fffef26af89 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56  ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007fffef26af89 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007fffef26e398 in __GI_abort () at abort.c:89
#2  0x00007fffef2a81e4 in __libc_message (do_abort=do_abort@entry=1, fmt=fmt@entry=0x7fffef3b6ab0 "*** Error in `%s': %s: 0x%s ***\n")
    at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007fffef2b44be in malloc_printerr (ptr=<optimized out>, str=0x7fffef3b6be0 "double free or corruption (out)", action=1) at malloc.c:4996
#4  _int_free (av=<optimized out>, p=<optimized out>, have_lock=0) at malloc.c:3840
#5  0x00007ffff336d26d in aligned_free (ptr=<optimized out>) at /usr/include/eigen3/Eigen/src/Core/util/Memory.h:243
#6  conditional_aligned_free<true> (ptr=<optimized out>) at /usr/include/eigen3/Eigen/src/Core/util/Memory.h:317
#7  conditional_aligned_delete_auto<float, true> (size=<optimized out>, ptr=<optimized out>)
    at /usr/include/eigen3/Eigen/src/Core/util/Memory.h:443
#8  ~DenseStorage (this=0xb32c90, __in_chrg=<optimized out>) at /usr/include/eigen3/Eigen/src/Core/DenseStorage.h:303
#9  ~PlainObjectBase (this=0xb32c90, __in_chrg=<optimized out>) at /usr/include/eigen3/Eigen/src/Core/PlainObjectBase.h:85
#10 ~Matrix (this=0xb32c90, __in_chrg=<optimized out>) at /usr/include/eigen3/Eigen/src/Core/Matrix.h:127
#11 _Destroy<Eigen::Matrix<float, -1, 1> > (__pointer=0xb32c90) at /usr/include/c++/4.8/bits/stl_construct.h:93
#12 __destroy<Eigen::Matrix<float, -1, 1>*> (__last=<optimized out>, __first=0xb32c90) at /usr/include/c++/4.8/bits/stl_construct.h:103
#13 _Destroy<Eigen::Matrix<float, -1, 1>*> (__last=<optimized out>, __first=<optimized out>) at /usr/include/c++/4.8/bits/stl_construct.h:126
#14 _Destroy<Eigen::Matrix<float, -1, 1>*, Eigen::Matrix<float, -1, 1> > (__last=0xb32d10, __first=<optimized out>)
    at /usr/include/c++/4.8/bits/stl_construct.h:151
#15 ~vector (this=0x7fffffffa030, __in_chrg=<optimized out>) at /usr/include/c++/4.8/bits/stl_vector.h:415
#16 pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::computeRFAndShapeDistribution (this=this@entry=0x7fffffffb5d0, 
    processed=..., output=..., cluster_indices=...) at /home/jwoods/pcl/features/include/pcl/features/impl/our_cvfh.hpp:393
#17 0x00007ffff336eb6f in pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::computeFeature (this=0x7fffffffb5d0, 
    output=...) at /home/jwoods/pcl/features/include/pcl/features/impl/our_cvfh.hpp:713
#18 0x00007ffff33680b1 in pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::compute (this=0x7fffffffb5d0, output=...)
    at /home/jwoods/pcl/features/include/pcl/features/impl/our_cvfh.hpp:68
#19 0x00000000006ab3ee in pcl::rec_3d_framework::OURCVFHEstimator<pcl::PointXYZ, pcl::VFHSignature308>::estimate (this=0xdacda0, in=..., 
    processed=..., signatures=..., centroids=...) at /home/jwoods/pose/rec/ourcvfh_estimator.h:349
#20 0x00000000006983c2 in pcl::rec_3d_framework::GlobalNNCVFHRecognizer<flann::ChiSquareDistance, pcl::PointXYZ, pcl::VFHSignature308>::initialize (this=0x7fffffffdb70, retrain_missing=true, force_retrain=false) at /home/jwoods/pose/rec/global_nn_recognizer_cvfh.hpp:700
#21 0x0000000000686174 in main (argc=37, argv=0x7fffffffe098) at /home/jwoods/pose/pose.cpp:416

I've gone through to look at a bunch of these and here's where it gets weird:

(gdb) frame 16
#16 pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::computeRFAndShapeDistribution (this=this@entry=0x7fffffffb5d0, 
    processed=..., output=..., cluster_indices=...) at /home/jwoods/pcl/features/include/pcl/features/impl/our_cvfh.hpp:393
393     for (size_t t = 0; t < transformations.size (); t++)
(gdb) list
388     sgurf (centroids_dominant_orientations_[i], dominant_normals_[i], processed, transformations, grid, cluster_indices[i]);
389 
390     // Make a note of how many transformations correspond to each cluster
391     cluster_axes_[i] = transformations.size ();
392     
393     for (size_t t = 0; t < transformations.size (); t++)
394     {
395 
396       pcl::transformPointCloud (*processed, *grid, transformations[t]);
397       transforms_.push_back (transformations[t]);

transformations is correctly allocated, or seems to be. Its size is 1. I can even get the element:

(gdb) p transformations[0]
$8 = (Eigen::Matrix<float, 4, 4, 0, 4, 4> &) @0xb2b060: {<Eigen::PlainObjectBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >> = {<Eigen::MatrixBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >> = {<Eigen::DenseBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >> = {<Eigen::internal::special_scalar_op_base<Eigen::Matrix<float, 4, 4, 0, 4, 4>, float, float, false>> = {<Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 0, 4, 4>, 3>> = {<Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 0, 4, 4>, 1>> = {<Eigen::DenseCoeffsBase<Eigen::Matrix<float, 4, 4, 0, 4, 4>, 0>> = {<Eigen::EigenBase<Eigen::Matrix<float, 4, 4, 0, 4, 4> >> = {<No data fields>}, <No data fields>}, <No data fields>}, <No data fields>}, <No data fields>}, <No data fields>}, <No data fields>}, m_storage = {m_data = {array = {0.99495858, 7.63832431e-05, 0.100286491, 0, 7.63832431e-05, 0.999998868, -0.00151946174, 0, 
          0.100286491, -0.00151946174, -0.994957447, 0, 3.81469727e-06, 0, 434.818634, 1}}}}, <No data fields>}

What I can't figure out is why this line causes ~vector to be called. Perhaps it's because a vector is allocated within this loop, and at loop termination time, it tries to de-allocate it? If so, this could have something to do with an earlier bug I fixed:

translunar@0b5e5a5 (#791 #792)

I'd love to get a second pair of eyes on this.

Valgrind output

As a side note, it works just fine when I run it with valgrind, but there are tons of libflann errors. But I am seeing an invalid read and an invalid write on the previously fixed line:

==27037== Invalid read of size 4
==27037==    at 0x9346FF2: pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::computeRFAndShapeDistribution(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZ> >&, pcl::PointCloud<pcl::VFHSignature308>&, std::vector<pcl::PointIndices, std::allocator<pcl::PointIndices> >&) (our_cvfh.hpp:501)
==27037==    by 0x9348B6E: pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::computeFeature(pcl::PointCloud<pcl::VFHSignature308>&) (our_cvfh.hpp:713)
==27037==    by 0x93420B0: pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::compute(pcl::PointCloud<pcl::VFHSignature308>&) (our_cvfh.hpp:68)
==27037==    by 0x6AB3ED: pcl::rec_3d_framework::OURCVFHEstimator<pcl::PointXYZ, pcl::VFHSignature308>::estimate(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZ> >&, boost::shared_ptr<pcl::PointCloud<pcl::PointXYZ> >&, std::vector<pcl::PointCloud<pcl::VFHSignature308>, Eigen::aligned_allocator<pcl::PointCloud<pcl::VFHSignature308> > >&, std::vector<Eigen::Matrix<float, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<float, 3, 1, 0, 3, 1> > >&) (ourcvfh_estimator.h:349)
==27037==    by 0x6983C1: pcl::rec_3d_framework::GlobalNNCVFHRecognizer<flann::ChiSquareDistance, pcl::PointXYZ, pcl::VFHSignature308>::initialize(bool, bool) (global_nn_recognizer_cvfh.hpp:700)
==27037==    by 0x686173: main (pose.cpp:416)
==27037==  Address 0x1ae33bfc is 4 bytes before a block of size 52 alloc'd
==27037==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27037==    by 0x68906A: Eigen::internal::aligned_malloc(unsigned long) (Memory.h:220)
==27037==    by 0x9346B61: pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::computeRFAndShapeDistribution(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZ> >&, pcl::PointCloud<pcl::VFHSignature308>&, std::vector<pcl::PointIndices, std::allocator<pcl::PointIndices> >&) (Memory.h:301)
==27037==    by 0x9348B6E: pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::computeFeature(pcl::PointCloud<pcl::VFHSignature308>&) (our_cvfh.hpp:713)
==27037==    by 0x93420B0: pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::compute(pcl::PointCloud<pcl::VFHSignature308>&) (our_cvfh.hpp:68)
==27037==    by 0x6AB3ED: pcl::rec_3d_framework::OURCVFHEstimator<pcl::PointXYZ, pcl::VFHSignature308>::estimate(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZ> >&, boost::shared_ptr<pcl::PointCloud<pcl::PointXYZ> >&, std::vector<pcl::PointCloud<pcl::VFHSignature308>, Eigen::aligned_allocator<pcl::PointCloud<pcl::VFHSignature308> > >&, std::vector<Eigen::Matrix<float, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<float, 3, 1, 0, 3, 1> > >&) (ourcvfh_estimator.h:349)
==27037==    by 0x6983C1: pcl::rec_3d_framework::GlobalNNCVFHRecognizer<flann::ChiSquareDistance, pcl::PointXYZ, pcl::VFHSignature308>::initialize(bool, bool) (global_nn_recognizer_cvfh.hpp:700)
==27037==    by 0x686173: main (pose.cpp:416)
==27037== 
==27037== Invalid write of size 4
==27037==    at 0x9346FF6: pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::computeRFAndShapeDistribution(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZ> >&, pcl::PointCloud<pcl::VFHSignature308>&, std::vector<pcl::PointIndices, std::allocator<pcl::PointIndices> >&) (our_cvfh.hpp:501)
==27037==    by 0x9348B6E: pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::computeFeature(pcl::PointCloud<pcl::VFHSignature308>&) (our_cvfh.hpp:713)
==27037==    by 0x93420B0: pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::compute(pcl::PointCloud<pcl::VFHSignature308>&) (our_cvfh.hpp:68)
==27037==    by 0x6AB3ED: pcl::rec_3d_framework::OURCVFHEstimator<pcl::PointXYZ, pcl::VFHSignature308>::estimate(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZ> >&, boost::shared_ptr<pcl::PointCloud<pcl::PointXYZ> >&, std::vector<pcl::PointCloud<pcl::VFHSignature308>, Eigen::aligned_allocator<pcl::PointCloud<pcl::VFHSignature308> > >&, std::vector<Eigen::Matrix<float, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<float, 3, 1, 0, 3, 1> > >&) (ourcvfh_estimator.h:349)
==27037==    by 0x6983C1: pcl::rec_3d_framework::GlobalNNCVFHRecognizer<flann::ChiSquareDistance, pcl::PointXYZ, pcl::VFHSignature308>::initialize(bool, bool) (global_nn_recognizer_cvfh.hpp:700)
==27037==    by 0x686173: main (pose.cpp:416)
==27037==  Address 0x1ae33bfc is 4 bytes before a block of size 52 alloc'd
==27037==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27037==    by 0x68906A: Eigen::internal::aligned_malloc(unsigned long) (Memory.h:220)
==27037==    by 0x9346B61: pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::computeRFAndShapeDistribution(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZ> >&, pcl::PointCloud<pcl::VFHSignature308>&, std::vector<pcl::PointIndices, std::allocator<pcl::PointIndices> >&) (Memory.h:301)
==27037==    by 0x9348B6E: pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::computeFeature(pcl::PointCloud<pcl::VFHSignature308>&) (our_cvfh.hpp:713)
==27037==    by 0x93420B0: pcl::OURCVFHEstimation<pcl::PointXYZ, pcl::Normal, pcl::VFHSignature308>::compute(pcl::PointCloud<pcl::VFHSignature308>&) (our_cvfh.hpp:68)
==27037==    by 0x6AB3ED: pcl::rec_3d_framework::OURCVFHEstimator<pcl::PointXYZ, pcl::VFHSignature308>::estimate(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZ> >&, boost::shared_ptr<pcl::PointCloud<pcl::PointXYZ> >&, std::vector<pcl::PointCloud<pcl::VFHSignature308>, Eigen::aligned_allocator<pcl::PointCloud<pcl::VFHSignature308> > >&, std::vector<Eigen::Matrix<float, 3, 1, 0, 3, 1>, std::allocator<Eigen::Matrix<float, 3, 1, 0, 3, 1> > >&) (ourcvfh_estimator.h:349)
==27037==    by 0x6983C1: pcl::rec_3d_framework::GlobalNNCVFHRecognizer<flann::ChiSquareDistance, pcl::PointXYZ, pcl::VFHSignature308>::initialize(bool, bool) (global_nn_recognizer_cvfh.hpp:700)
==27037==    by 0x686173: main (pose.cpp:416)
==27037== 

[taketwo]

Fixed by #875.