This repo is the official implementation of NIPS24 paper Voxel Mamba: Group-Free State Space Models for Point Cloud based 3D Object Detection. Our Voxel Mamba achieves state-of-the-art performance on Waymo and nuScene datasets. It abandons the grouping operation and serializes voxels into one single sequence, enabling better efficiency.
-[24-06-18] Voxel Mamba released on arxiv
-[24-07-01] Code of Waymo is released (SOTA).
-[24-09-26] Voxel Mamba is accepted by NIPS24 Spotlight!
- Release the arXiv version.
- Clean up and release the code.
- Release code of Waymo.
- Release code of NuScenes.
- Release code of ERFs visualization.
- Merge Voxel Mamba to OpenPCDet.
Validation set
| Model | mAPH_L1 | mAPH_L2 | Veh_L1 | Veh_L2 | Ped_L1 | Ped_L2 | Cyc_L1 | Cyc_L2 | Log |
|---|---|---|---|---|---|---|---|---|---|
| Voxel Mamba | 79.6 | 73.6 | 80.8/80.3 | 72.6/72.2 | 85.0/80.8 | 77.7/73.6 | 78.6/77.6 | 75.7/74.8 | Log |
Test set
| Model | mAPH_L1 | mAPH_L2 | Veh_L1 | Veh_L2 | Ped_L1 | Ped_L2 | Cyc_L1 | Cyc_L2 | Leaderboard |
|---|---|---|---|---|---|---|---|---|---|
| Voxel Mamba | 79.6 | 74.3 | 84.4/84.0 | 77.0/76.6 | 84.8/80.6 | 79.0/74.9 | 75.4/74.3 | 72.6/71.5 | leaderboard |
Validation set
| Model | mAP | NDS | mATE | mASE | mAOE | mAVE | mAAE | ckpt | Log |
|---|---|---|---|---|---|---|---|---|---|
| Voxel Mamba | 67.5 | 71.9 | 26.7 | 25.0 | 25.8 | 21.8 | 18.9 | ckpt | Log |
Test set
| Model | mAP | NDS | mATE | mASE | mAOE | mAVE | mAAE | Leaderboard | Submission |
|---|---|---|---|---|---|---|---|---|---|
| Voxel Mamba | 69.0 | 73.0 | 24.3 | 23.0 | 30.9 | 23.7 | 13.3 | leaderboard | Submission |
Voxel Mamba's result on Waymo compared with other leading methods. All the experiments are evaluated on an NVIDIA A100 GPU with the same environment. We hope that our Voxel Mamba can provide a potential group-free solution for efficiently handling sparse point clouds for 3D tasks.
Please refer to INSTALL.md for installation.
Please follow the instructions from OpenPCDet. We adopt the same data generation process.
cd data
mkdir hilbert
python ./tools/hilbert_curves/create_hilbert_curve_template.py
You can also download Hilbert Template files from Google Drive or BaiduYun(code: kubf).
# multi-gpu training
cd tools
bash scripts/dist_train.sh 8 --cfg_file ./cfgs/voxel_mamba_models/voxel_mamba_waymo.yaml # for waymo
# multi-gpu testing
cd tools
bash scripts/dist_test.sh 8 --cfg_file <CONFIG_FILE> --ckpt <CHECKPOINT_FILE>
Please consider citing our work as follows if it is helpful.
@article{zhang2024voxel,
title={Voxel Mamba: Group-Free State Space Models for Point Cloud based 3D Object Detection},
author={Zhang, Guowen and Fan, Lue and He, Chenhang and Lei, Zhen and Zhang, Zhaoxiang and Zhang, Lei},
journal={arXiv preprint arXiv:2406.10700},
year={2024}
}
Voxel Mamba is based on OpenPCDet and DSVT.
We also thank the CenterPoint, TransFusion, OctFormer, Mamba, and HEDNet authors for their efforts.