Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Bonxai as Map representation #22

Open
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

tizianoGuadagnino
Copy link
Collaborator

@tizianoGuadagnino tizianoGuadagnino commented Nov 29, 2024

Motivation

It is finally time to introduce a proper data structure for representing 3D space instead of a simple hash table. By having a more appropriate and flexible data structure, we could extend the system further, for example, by using some TSDF registration or similar. Of course, this should come at a low cost in terms of performance. Our VoxelHashMap has done wonders for us in KISS, but I intend to move on.

Why Bonxai?

The project is well maintained, does not add additional dependencies, and is header-only. In terms of the build system, it is exactly the same as fossil. The project is already integrated with the ROS ecosystem, so we are also covered on that side. That is a nice cherry on top. When an RViz visualization of the grid is introduced, Kinematic-ICP will immediately benefit.

Why not OpenVDB

I don't want to deal with 20 dependencies and an ugly Cmake script that does madness with FetchContent, Bonxai is much smaller and does precisely what we need it for.

This PR

I keep the same API as the VoxelHashMap but change the type name and the internal map representation. Everything else will be the same for the user.

Please give me feedback on this @benemer @nachovizzo

Copy link
Collaborator

@nachovizzo nachovizzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind switching to Bonxai. But did you run some benchmarks? I would expect that it uses much more memory than a simple hash table, as each point would be allocated in a leaf node of 512 voxels, no matter how isolated is this point, which doesn't happen for pain hash tables.

That said I also think it might be slightly slower in terms of runtime. I would probably give it a try with kiss icp, just to try on more datasets, and check the impact on the registration system

@nachovizzo
Copy link
Collaborator

And another comment on

the project is already integrated with the ROS ecosystem,

It is as much integrated as Tessil and OpenVDB are 🤣 Actually, OpenVDB is more integrated to ROS due to Steve's https://github.com/SteveMacenski/spatio_temporal_voxel_layer, where we went through big pain making sure OpenVDB is it available as an installable package: https://index.ros.org/p/openvdb_vendor/github-SteveMacenski-spatio_temporal_voxel_layer/#jazzy

@tizianoGuadagnino
Copy link
Collaborator Author

I don't mind switching to Bonxai. But did you run some benchmarks? I would expect that it uses much more memory than a simple hash table, as each point would be allocated in a leaf node of 512 voxels, no matter how isolated is this point, which doesn't happen for pain hash tables.

That said I also think it might be slightly slower in terms of runtime. I would probably give it a try with kiss icp, just to try on more datasets, and check the impact on the registration system

We can port it to KISS to test a broader range of datasets. At least on the "Kinematic" sequences, I didn't notice a significant difference in runtime or memory consumption, but as you said, the sample size of our datasets with Kinematic is kind of limited (just 2 sensor configurations).

Remember, although we allocate N voxels in the leaf node, each voxel contains a dynamic data structure (a.k.a. std::vector, so the memory overhead shouldn't be significant.

PS: In terms of "accuracy" it is literally the same number up to the 4th digit, but I guess this doesn't surprise nobody

@tizianoGuadagnino
Copy link
Collaborator Author

tizianoGuadagnino commented Nov 29, 2024

And another comment on

the project is already integrated with the ROS ecosystem,

It is as much integrated as Tessil and OpenVDB are 🤣 Actually, OpenVDB is more integrated to ROS due to Steve's https://github.com/SteveMacenski/spatio_temporal_voxel_layer, where we went through big pain making sure OpenVDB is it available as an installable package: https://index.ros.org/p/openvdb_vendor/github-SteveMacenski-spatio_temporal_voxel_layer/#jazzy

This was an additional motivation for me. The reason I chose Bonxai is not the ROS integration; this is more of a cherry on top. For me, it needs to be easily installable for the C++ API and ROS so that we can keep these two aspects separate.

@facontidavide
Copy link

Thanks.

Please run benchmarks.

@tizianoGuadagnino
Copy link
Collaborator Author

tizianoGuadagnino commented Dec 2, 2024

Thanks.

Please run benchmarks.

image

Here there is. For this benchmark, I generate a point cloud out of the surface of a cube with a side of 100m. I randomly sampled 10.000 points for each face, so the total points for the point cloud were 60.000. I then simulate a LiDAR 16x1024 moving inside this "box" and benchmark both the GetClosestPoint, AddPoints and Update functions. The code for the benchmark is in the branch benchmark if you are curious (which I will probably merge to master after this).

GetClosestPoint benchmark: I use the full cube point cloud as a map for both Bonxai and VoxelHash and query neighbors for a downsampled version of the scan (the same preprocessing as the original pipeline).

AddPoints: Add the all cube points to the map.

Update benchmark: The map is initialized with all cube points. I simulate a circular trajectory inside the cube, and for each location, I add the points from the downsampled LiDAR and remove the points that are too far away.

Note: We could probably further improve the performance of the Update function by checking a single voxel per leaf grid instead of all the cells.

I will be working on the memory analysis next ;)

@tizianoGuadagnino
Copy link
Collaborator Author

Note: We could probably further improve the performance of the Update function by checking a single voxel per leaf grid instead of all the cells.

image

But here I am also removing the dangling keys from the root map once all the leafs from an inner_grid are empty.

@tizianoGuadagnino
Copy link
Collaborator Author

tizianoGuadagnino commented Dec 3, 2024

Memory Analysis

For this experiment, I follow the God commandment and use heaptrack to perform the measurement. I report two different leaf/inner grid sizes, default (inner_grid_log2_dim=2, leaf_grid_log2_dim=3) and small leaf (inner_grid_log2_dim=2, leaf_grid_log2_dim=2).

Sampling 6.000.000 points from the surface of a cube

VoxelHashMap

voxelhash_1e6

BonxaiGrid (default)

bonxai_default_1e6

BonxaiGrid (small leaf)

bonxai_new_1e6

Sampling 60.000 points from the surface of a cube

VoxelHashMap

voxelhash_10000

BonxaiGrid (default)

bonxai_default_10000

BonxaiGrid (small leaf)

bonxai_new_10000

Comments

While using Bonxai increases memory consumption, this overhead seems proportional to the sparsity of the data. This is clearly due to the dense allocation of the leafs as foretold by @nachovizzo. This makes a difference in scenarios with some "random point measurements" due to reflections, occlusions, or sparse structures. In structured environments, the difference in memory consumption might be acceptable, considering that the system halved the time of the nearest neighbor search, which has been one of the bottlenecks for a long time, especially single-threaded.

Give me your thoughts guys ;)

@tizianoGuadagnino
Copy link
Collaborator Author

tizianoGuadagnino commented Dec 3, 2024

For completeness, I also report the results of the benchmark for the BonxaiGrid (small leaf)

image

@tizianoGuadagnino
Copy link
Collaborator Author

tizianoGuadagnino commented Dec 3, 2024

Running the online node on some real data between main and this PR results in the following:

image

This is a comparison on Heaptrack, it seems VoxelHashMap::GetClosestNeighbor if generating ~18 more allocations per second. I upload here the data so that you guys can have a look.

heaptrack_data.zip

@benemer
Copy link
Member

benemer commented Dec 13, 2024

To summarize the analysis: At least with the simulated points from the tests, Bonxai is faster for NN query and removing points, whereas the voxel hash map adds points faster. Hard to say how this affects the total runtime because it depends on the actual data rather than the simulated one. I agree we should port this to KISS and run more tests on actual data besides these benchmarks.

Regarding memory consumption, I don't know how severe this increase is. Also, it's again not easy to draw conclusions from simulated data because now we sample from a cube, resulting in more dense surfaces compared to a real-world environment.

But in general, I also like the results so far, and the time we save on the NN search is the most critical.

# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
# Silence timestamp warning
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed? I think it's coming from the find_dependencies.cmake from KISS, we could move it to the top-level .cmake

#include "bonxai/grid_coord.hpp"

namespace {
static constexpr std::array<Bonxai::CoordT, 27> shifts{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be discussed after resolving PRBonn/kiss-icp#410

})) {
return;
}
voxel_points->reserve(max_points_per_voxel_);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There will be duplicated calls to reserve, but since max_points_per_voxel_ is fixed, this should not matter, right?

std::vector<Eigen::Vector3d> SparseVoxelGrid::Pointcloud() const {
std::vector<Eigen::Vector3d> point_cloud;
point_cloud.reserve(map_.activeCellsCount() * max_points_per_voxel_);
map_.forEachCell([&point_cloud, this](VoxelBlock &block, const auto &) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
map_.forEachCell([&point_cloud, this](VoxelBlock &block, const auto &) {
map_.forEachCell([&point_cloud, this](const VoxelBlock &block, const auto &) {

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants