dmnet

DMNet

Dynamic Multi-scale Filters for Semantic Segmentation

Introduction

Official Repo

Code Snippet

Abstract

Multi-scale representation provides an effective way toaddress scale variation of objects and stuff in semantic seg-mentation. Previous works construct multi-scale represen-tation by utilizing different filter sizes, expanding filter sizeswith dilated filters or pooling grids, and the parameters ofthese filters are fixed after training. These methods oftensuffer from heavy computational cost or have more param-eters, and are not adaptive to the input image during in-ference. To address these problems, this paper proposes aDynamic Multi-scale Network (DMNet) to adaptively cap-ture multi-scale contents for predicting pixel-level semanticlabels. DMNet is composed of multiple Dynamic Convolu-tional Modules (DCMs) arranged in parallel, each of whichexploits context-aware filters to estimate semantic represen-tation for a specific scale. The outputs of multiple DCMsare further integrated for final segmentation. We conductextensive experiments to evaluate our DMNet on three chal-lenging semantic segmentation and scene parsing datasets,PASCAL VOC 2012, Pascal-Context, and ADE20K. DMNetachieves a new record 84.4% mIoU on PASCAL VOC 2012test set without MS COCO pre-trained and post-processing,and also obtains state-of-the-art performance on Pascal-Context and ADE20K.

Results and models

Cityscapes

Method	Backbone	Crop Size	Lr schd	Mem (GB)	Inf time (fps)	Device	mIoU	mIoU(ms+flip)	config	download
DMNet	R-50-D8	512x1024	40000	7.0	3.66	V100	77.78	79.14	config	model | log
DMNet	R-101-D8	512x1024	40000	10.6	2.54	V100	78.37	79.72	config	model | log
DMNet	R-50-D8	769x769	40000	7.9	1.57	V100	78.49	80.27	config	model | log
DMNet	R-101-D8	769x769	40000	12.0	1.01	V100	77.62	78.94	config	model | log
DMNet	R-50-D8	512x1024	80000	-	-	V100	79.07	80.22	config	model | log
DMNet	R-101-D8	512x1024	80000	-	-	V100	79.64	80.67	config	model | log
DMNet	R-50-D8	769x769	80000	-	-	V100	79.22	80.55	config	model | log
DMNet	R-101-D8	769x769	80000	-	-	V100	79.19	80.65	config	model | log

ADE20K

Method	Backbone	Crop Size	Lr schd	Mem (GB)	Inf time (fps)	Device	mIoU	mIoU(ms+flip)	config	download
DMNet	R-50-D8	512x512	80000	9.4	20.95	V100	42.37	43.62	config	model | log
DMNet	R-101-D8	512x512	80000	13.0	13.88	V100	45.34	46.13	config	model | log
DMNet	R-50-D8	512x512	160000	-	-	V100	43.15	44.17	config	model | log
DMNet	R-101-D8	512x512	160000	-	-	V100	45.42	46.76	config	model | log

Citation

@InProceedings{He_2019_ICCV,
author = {He, Junjun and Deng, Zhongying and Qiao, Yu},
title = {Dynamic Multi-Scale Filters for Semantic Segmentation},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2019}
}