README.md

PowerBEV

This is the official PyTorch implementation of the paper:

PowerBEV: A Powerful yet Lightweight Framework for Instance Prediction in Bird's-Eye View
Peizheng Li, Shuxiao Ding, Xieyuanli Chen, Niklas Hanselmann, Marius Cordts, Jürgen Gall

📃 Contents

• PowerBEV
  • 📃 Contents
  • 📰 News
  • ⚙️ Setup
  • 📁 Dataset
  • 🔥 Pre-trained models
  • 🏊 Training
  • 🏄 Prediction
    • Evaluation
    • Visualisation
  • 📜 License
  • 🔗 Citation

📰 News

• PowerBEV has been accepted by the 32nd International Joint Conference on Artificial Intelligence.
• PowerBEV has been included in ROAD++: The Second Workshop & Challenge on Event Detection for Situation Awareness in Autonomous Driving @ ICCV 2023.

⚙️ Setup

Create the conda environment by running

conda env create -f environment.yml

📁 Dataset

• Download the full NuScenes dataset (v1.0), which includes the Mini dataset (metadata and sensor file blobs) and the Trainval dataset (metadata and file blobs part 1-10).
• Extract the tar files to the default nuscenes/ or to YOUR_NUSCENES_DATAROOT. The files should be organized in the following structure:

  nuscenes/
  ├──── trainval/
  │     ├──── maps/
  │     ├──── samples/
  │     ├──── sweeps/
  │     └──── v1.0-trainval/
  └──── mini/
        ├──── maps/
        ├──── samples/
        ├──── sweeps/
        └──── v1.0-mini/
  

🔥 Pre-trained models

The config file can be found in powerbev/configs

Config	Weights	Dataset	Past Context	Future Horizon	BEV Size	IoU	VPQ
powerbev.yml	PowerBEV_long.ckpt	NuScenes	1.0s	2.0s	100m x 100m (50cm res.)	39.3	33.8
powerbev.yml	PowerBEV_short.ckpt	NuScenes	1.0s	2.0s	30m x 30m (15cm res.)	62.5	55.5

Note: All metrics above are obtained by training based on pre-trained static weights (static long/static short).

🏊 Training

To train the model from scratch on NuScenes, run

python train.py --config powerbev/configs/powerbev.yml

To train the model from the pre-trained static checkpoint on NuScenes, download pre-trained static weights (static long/static short) to YOUR_PRETRAINED_STATIC_WEIGHTS_PATH and run

python train.py --config powerbev/configs/powerbev.yml \
                PRETRAINED.LOAD_WEIGHTS True \
                PRETRAINED.PATH $YOUR_PRETRAINED_STATIC_WEIGHTS_PATH

Note: These will train the model on 4 GPUs, each with a batch of size 2.

To set your configs, please run

python train.py --config powerbev/configs/powerbev.yml \
                DATASET.DATAROOT $YOUR_NUSCENES_DATAROOT \
                LOG_DIR $YOUR_OUTPUT_PATH \
                GPUS [0] \
                BATCHSIZE $YOUR_DESIRED_BATCHSIZE

The above settings can also be changed directly by modifying powerbev.yml. Please see the config.py for more information.

🏄 Prediction

Evaluation

Download trained weights (long/short) to YOUR_PRETRAINED_WEIGHTS_PATH and run

python test.py --config powerbev/configs/powerbev.yml \
                PRETRAINED.LOAD_WEIGHTS True \
                PRETRAINED.PATH $YOUR_PRETRAINED_WEIGHTS_PATH

Visualisation

Download trained weights (long/short) to YOUR_PRETRAINED_WEIGHTS_PATH and run

python visualise.py --config powerbev/configs/powerbev.yml \
                PRETRAINED.LOAD_WEIGHTS True \
                PRETRAINED.PATH $YOUR_PRETRAINED_WEIGHTS_PATH \
                BATCHSIZE 1

This will render predictions from the network and save them to an visualization_outputs folder. Note: To visualize Ground Truth, please add the config VISUALIZATION.VIS_GT True at the end of the command

📜 License

PowerBEV is released under the MIT license. Please see the LICENSE file for more information.

🔗 Citation

@article{li2023powerbev,
  title     = {PowerBEV: A Powerful Yet Lightweight Framework for Instance Prediction in Bird's-Eye View},
  author    = {Li, Peizheng and Ding, Shuxiao and Chen, Xieyuanli and Hanselmann, Niklas and Cordts, Marius and Gall, Juergen},
  journal   = {arXiv preprint arXiv:2306.10761},
  year      = {2023}
}
@inproceedings{ijcai2023p120,
  title     = {PowerBEV: A Powerful Yet Lightweight Framework for Instance Prediction in Bird’s-Eye View},
  author    = {Li, Peizheng and Ding, Shuxiao and Chen, Xieyuanli and Hanselmann, Niklas and Cordts, Marius and Gall, Juergen},
  booktitle = {Proceedings of the Thirty-Second International Joint Conference on
               Artificial Intelligence, {IJCAI-23}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Edith Elkind},
  pages     = {1080--1088},
  year      = {2023},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2023/120},
  url       = {https://doi.org/10.24963/ijcai.2023/120},
}