Remove redundant Modulus operation for Voxel Hash function

[saurabh1002]

1. Remove the modulus operation (1 << 20) - 1 & <hash> from the custom Hashing operator.

2. Initialize the robin map with a higher bucket size, change the default load factors. #359

Description by Gupta

The modulus operation in the hashing operator of the custom hash function is actually not required by the Robin Map implementation. They perform that operation internally based on the current bucket size, which keeps on changing based on current load_factor of the hash table. See the figure below where I backtrace to that modulus operation in Robin Map library.

In fact, the current bucket size that we have, through the hash operator, is 2^20 buckets. This can cause an issue, when internally the robin map grows to more than 2^20 buckets, in which case our hash function will limit the robin map bucket computation. Otherwise, when the internal bucket size is below 2^20, our additional modulus operation is just redundant.

[benemer]

Can you please give some pointers to the documentation where this can be found? Especially that the modulus operation is done on the hash map side

[saurabh1002]

The Hashing function

The modulus operation in the hashing operator of the custom hash function is actually not required by the Robin Map implementation. They perform that operation internally based on the current bucket size, which keeps on changing based on current load_factor of the hash table. See the figure below where I backtrace to that modulus operation in Robin Map library.

In fact, the current bucket size that we have, through the hash operator, is 2^20 buckets. This can cause an issue, when internally the robin map grows to more than 2^20 buckets, in which case our hash function will limit the robin map bucket computation. Otherwise, when the internal bucket size is below 2^20, our additional modulus operation is just redundant.

[saurabh1002]

Default load factor params

The robin map internally resizes the hash table based on the load_factor thresholds.

1. It upscales the bucket size by a factor of 2 when load_factor goes above 0.5, which is an expensive operation as it recomputes all the hashes and the buckets they are assigned to.
2. It downscales the bucket size by a factor of 2 again, when the load_factor goes below 0.0, which means basically never.

It is better if we already initialize the robin map with a sufficient bucket size, along with increasing the maximum load factor threshold for upscaling to 0.75. This will avoid frequent rehashing of the map, especially at the beginning of any sequence.
Also, we can set the minimum load factor threshold for downscaling to 0.05 to have some sort of downscaling when operating in narrow regions with lower FOV, and avoid costly iterations through the map, for example when querying for nearest neighbors, or generating pointcloud from local map.

[saurabh1002]

[saurabh1002]

Test on MulRAN Sequences

[saurabh1002]

Apollo Dataset

[nachovizzo]

Fantastic @saurabh1002 thanks !