Skip to content

Latest commit

 

History

History
157 lines (140 loc) · 6.09 KB

README.md

File metadata and controls

157 lines (140 loc) · 6.09 KB
Raw

PanopticFCN + Intra-Batch Supervision

[Project page] [Paper]

Code for 'Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images', Daan de Geus and Gijs Dubbelman, WACV 2023.

This code applies Intra-Batch Supervision to Panoptic FCN, and is built upon the official Panoptic FCN code.

Installation

This project is based on Detectron2, which can be constructed as follows.

  • Install Detectron2 following these instructions.
  • Create the environment variable DETECTRON2_DATASETS, where the datasets are stored
  • Setup the Cityscapes dataset following this structure.
  • Setup the Mapillary Vistas dataset following this structure.
  • Copy this project to /path/to/detectron2/projects/PanopticFCN-IBS, replacing path/to/detectron2 with the path where Detectron2 is located.
  • To prepare the datasets for our crop sampling, run these two commands:
    • python /path/to/detectron2/projects/PanopticFCN-IBS/data/cityscapes/prepare_cityscapes_sampling.py
    • python /path/to/detectron2/projects/PanopticFCN-IBS/data/mapillaryvistas/prepare_mapillary_sampling.py

Training

To train a model with 8 GPUs, run:

cd /path/to/detectron2
python projects/PanopticFCN-IBS/train.py --config-file <config.yaml> --num-gpus 8

For example, to launch a training with Panoptic FCN + IBS on Cityscapes with ResNet-50 backbone on 4 GPUs, one should execute:

cd /path/to/detectron2
python projects/PanopticFCN-IBS/train.py --config-file projects/PanopticFCN-IBS/configs/cityscapes/PanopticFCN-R50-cityscapes-ibs-cropsampling.yaml --num-gpus 4

For example, to launch a training with Panoptic FCN + IBS on Mapillary Vistas with ResNet-50 backbone on 8 GPUs, one should execute:

cd /path/to/detectron2
python projects/PanopticFCN-IBS/train.py --config-file projects/PanopticFCN-IBS/configs/mapillary-vistas/PanopticFCN-R50-mapillaryvistas-ibs-cropsampling.yaml --num-gpus 8

Evaluation

To evaluate a pre-trained model with 4 GPUs, run:

cd /path/to/detectron2
python projects/PanopticFCN-IBS/train.py --config-file <config.yaml> --num-gpus 4 --eval-only MODEL.WEIGHTS /path/to/model_checkpoint

Results

Results and models on Cityscapes. Note: like for the original Panoptic FCN, there is a large variance between results of different trainings with Panoptic FCN on Cityscapes.

Method Crop sampling Backbone Iters PQ PQ_th PQ_st Acc_th Prec_th config model
PanopticFCN no R50 65k 59.5 52.2 64.8 81.3 86.8 config TBD
PanopticFCN + IBS yes R50 65k 60.8 54.7 65.3 87.1 92.6 config TBD

Results and models on Mapillary Vistas

Method Crop sampling Backbone Iters PQ PQ_th PQ_st Acc_th Prec_th config model
PanopticFCN no R50 150k 35.5 31.8 40.3 74.7 77.9 config TBD
PanopticFCN + IBS yes R50 150k 36.3 33.6 40.0 77.0 82.2 config TBD

Citing us

Please consider citing our work if it is useful for your research.

@inproceedings{degeus2023ibs,
  title={Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images},
  author={{de Geus}, Daan and Dubbelman, Gijs},
  booktitle={IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  year={2023}
}

Also consider citing the original Panoptic FCN paper.

@inproceedings{li2021panopticfcn,
  title={Fully Convolutional Networks for Panoptic Segmentation},
  author={Yanwei Li, Hengshuang Zhao, Xiaojuan Qi, Liwei Wang, Zeming Li, Jian Sun, and Jiaya Jia},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2021}
}