Improved concurrency for multi-threaded indexing

[nigels-com]

Concurrent tree building is faster on my 12 core laptop for ~130M point clouds, in this arrangement.

The intuition is to avoid having threads wait for both the left and right subtrees if they are both asynchronous.
In this arrangment we first recurse to left on the same thread (depth-first to the left) with an async recursion of the right subtree concurrently.
Once the left is all visited we'll recurse to the right with the current thread.

What this is aiming to avoid is having too many threads blocked (doing no work) waiting for async processing to complete on both left and right.

I see around a 2x speedup for 12 cores and ~130M, around 30 seconds rather than 60 seconds.

[jlblancoc]

Great contribution! Let me check this against valgrind and do some minimal benchmarking before merging.

[nigels-com]

Happy indeed for a second opinion on this.

[jlblancoc]

I updated nanoflann benchmark to make some quick benchmarking with this PR. The results are good but, as usual, the optimal approach depends on the data distribution and dataset size:

Build index profiling for the KITTI 04 sequence pointclouds:

Method	1 thread	4 threads	8 threads	auto # threads (=0)
Baseline	8.993637 ms	6.922916 ms	6.759240 ms	7.775914 ms
This PR	8.999447 ms	6.235389 ms	7.164032 ms	8.716205 ms

Some graphs:

For 4 threads vs 1 thread

For 8 threads vs 1 thread

The advantage of this PR is clearer for 4 threads than for 8 threads... for this particular pointcloud.

Anyway, I feel positive about the logic of the change, and valgrind (including helgrind) is happy, so I'm merging it...
Thanks!

[nigels-com]

Interesting results. Modest speedup for these (small?) point clouds.
Our typical workload is 100M to 1B points.

[jlblancoc]

Yes, they are small in comparison. The horizontal axis is number of points...

But the trend is clear in that multi threading is better for larger clouds.

[nigels-com]

I'm a bit intrigued about four threads being faster than eight threads.
I'd like to dig into that a little more.

[jlblancoc]

@nigels-com The exact place to force a fixed number of threads in our benchmark is here:
https://github.com/MRPT/nanoflann-benchmark/blob/master/benchmarkTool/realTests/benchmark_nanoflann_real.cpp#L169-L177

feel free of using that in your project/dataset just to see what happens...

My guess is that results may depend a lot depending on how "random" the distribution of consecutive points are. In 3D LIDARs (as KITTI), point coordinates have a strong correlation with their neighbors in the sequence.

[nigels-com]

Thanks for the details.

Our points are probably less coherent because the Hovermap lidar is both spinning and moving through space.
We do see better performance sorting them upstream of nanoflann.
So my number and graphs are likely to be quite different to this benchmark.

https://emesent.com/hovermap-series/