Weakly-supervised-learning-for-image-analysis

• Recently, weak-supervised image analysis has become a hot topic in medical&natural image computing. Unfortunately, there are only a few open-source codes and datasets, since the privacy policy and others. For easy evaluation and fair comparison, we are trying to build a weak-supervised image analysis benchmark to boost the weak-supervised learning research in the image computing community.
• If you are interested, you can push your implementations or ideas to this repo or contact me at any time.
• My personal interest is mainly focused on medical image segmentation tasks, but this repo will also collect many papers on natural image detection and segmentation tasks.

Typical weak annotations include image-level labels, bounding boxes, points, and scribbles. This repo focus on points and scribbles.

Content

• Literature List
• Benchmark
  • Medical Images
    • VS & BraTS
    • COVID-19
    • ACDC
    • MSCMRseg
    • LVSC
    • CHAOS
  • Natural Images
    • COD10K,CAMO,CHAMELEON
    • DUTS testing dataset, ECSSD, DUT, PASCAL-S, HKU-IS, THUR
    • MS-COCO, PASCAL VOC , Bees, CrowdHuman and Objects365
  • Others
• Tutorial
  • 中文
  • English
• Conclusion
• Questions and Suggestions

Literature List

Keywords

scrib.: scribble level label  |  point.: point level label   | box.: bounding box label   | img.: image level label   |  

Statistics: 🔥 code is available & stars >= 100  |  ⭐ popular & cited in a survey  |  🌻 natural scene images  |  🌎 remote sensing images  |  🏥 medical images

Date	1st Institute	Title	Code	Publication	Label	Dataset
2022-08	University of Electronic Science and Technology of China 成电王国泰组	PA-Seg: Learning from Point Annotations for 3D Medical Image Segmen- tation using Contextual Regularization and Cross Knowledge Distillation	None	Arxiv under TMI revision	point.	🏥 VS, BraTS
2022-07	City University of Hong Kong	Weakly-Supervised Camouflaged Object Detection with Scribble Annotation	None	Arxiv	scrib.	🌻 COD10K, CAMO, CHAMELEON
2022-06	Fudan University	CycleMix: A Holistic Strategy for Medical Image Segmentation from Scribble Supervision	github	CVPR 2022	scrib.	🏥 ACDC, MSCMRseg
2022-03	Shanghai Jiao Tong University	Scribble2D5: Weakly-Supervised Volumetric Image Segmentation via Scribble Annotations	github	MICCAI 2022	scrib.	🏥 ACDC, VS, CHAOS
2022-03	University of Electronic Science and Technology of China 成电王国泰组	Scribble-Supervised Medical Image Segmentation via Dual-Branch Network and Dynamically Mixed Pseudo Labels Supervision	github	MICCAI 2022	scrib.	🏥 ACDC
2022-06	AWS AI Labs	Omni-DETR: Omni-Supervised Object Detection with Transformers	github	CVPR 2022	point. box. img.	🌻 MS-COCO, PASCAL VOC, Bees, CrowdHuman, Objects365
2021-09	Wuhan University of Science and Technology	Weakly Supervised Segmentation of COVID19 Infection with Scribble Annotation on CT Image	None	Pattern Recognition	scrib.	🏥 COVID-19
2021-06	UC Berkeley	Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learning	github	ICLR	scrib. point. box. img.	🌻 Pascal VOC 2012
2021-03	Hong Kong University of Science and Technology	Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion	github	CVPR	scrib.	🌻 Interactive Video Object Segmentation
2021-03	University of Edinburgh	Learning to Segment from Scribbles using Multi-scale Adversarial Attention Gates	github	TMI	scrib.	🏥 Heart Segmentation, Abdominal Segmentation
2021-01	Element AI	A Weakly Supervised Consistency-based Learning Method for COVID-19 Segmentation in CT Images	github	WACV	point.	🏥 COVID-19
2020-07	Australian National University	Weakly-Supervised Salient Object Detection via Scribble Annotations	github	CVPR	scrib.	🌻 DUTS testing dataset, ECSSD, DUT, PASCAL-S, HKU-IS, THUR
2020-09	Rutgers University	Weakly Supervised Deep Nuclei Segmentation Using Partial Points Annotation in Histopathology Images	None	TMI	point.	🏥
2020-06	Ulsan National Institute of Science and Technology	Scribble2Label: Scribble-Supervised Cell Segmentation via Self-Generating Pseudo-Labels with Consistency	github	MICCAI	scrib.	🏥 Cell segmentation

Benchmark

Medical images

Vestibular Schwannoma

• VS

Brain Tumor Segmentation

• BraTS

IS-COVID dataset

Label	Methods	dice	Jaccard	sensitivity	specificity	MAE
Scribble	p-UNet[55]	0.660	0.516	0.833	0.825	0.138
	WS0D[54]	0.684	0.533	0.842	0.871	0.114
	S2L[44]	0.708	0.550	0.805	0.926	0.091
	USTM-Net	0.725	0.582	0.854	0.967	0.086
Full	U-Net[49]	0.736	0.595	0.867	0.961	0.082

Lesion Segmentation (CC-COVID) dataset

Label	Methods	Consolidation			Ground-Glass Opacity			Average
		Dice	SE	SP	Dice	SE	SP	Dice	SE	SP
Scribble	p-UNet [55]	0.672	0.806	0.908	0.643	0.789	0.894	0.658	0.798	0.901
	WSOD [54]	0.695	0.833	0.917	0.674	0.801	0.902	0.685	0.817	0.910
	S2L [44]	0.724	0.857	0.934	0.698	0.840	0.928	0.711	0.849	0.931
	USTM-Net	0.736	0.862	0.958	0.709	0.829	0.947	0.723	0.846	0.953
Point	WSCL [18]	0.705	0.827	0.920	0.681	0.803	0.916	0.693	0.815	0.918
Full	U-Net [49]	0.748	0.874	0.966	0.713	0.825	0.952	0.731	0.850	0.959

Heart Segmentation

• ACDC dataset, scribble available
• LVSC dataset, scribble generation
• MSCMRseg dataset, scribble available

Abdominal Segmentation

• CHAOS dataset,scribble generation
• result style in the table: (Dice) mean±std.

SupervisionType	Model	ACDC	LVSC	CHAOS-T1	CHAOS-T2
Scribble	UNet pcE	79.0±0.06	62.3±0.09	34.4±0.06	37.5±0.06
Scribble	UNet wpcE	69.4±0.07	59.1±0.07	40.0±0.05	52.1±0.05
Scribble	UNet cRF	69.6±0.07	60.4±0.08	40.5±0.05	44.7±0.06
Scribble	TS-UNet cRF	37.3±0.08	50.5±0.07	29.3±0.05	27.6±0.05
Scribble	PostDAE	69.0±0.06	58.6±0.07	29.1±0.06	35.5±0.05
Scribble	UNet D	61.8±0.08	31.7±0.09	44.0±0.03	46.3±0.01
Scribble	ACCL	82.6±0.05	65.9±0.08	48.3±0.07	49.7±0.05
Scribble	Valvano et al.	84.3±0.04	65.5±0.08	56.8±0.05	57.8±0.04
Mask	UNet UB	82.0±0.qs	67.2±0.07	60.8±0.06	58.6±0.01
Mask	UNet D UB	83.9±0.05	67.9±0.09	63.9±0.05	60.8±0.01

Cell Segmentation

• EM&Data Science Bowl 2018&MoNuSeg
• result style in the table: Dice[mIoU]

Label	Method	EM	DSB-BF	DSB-Fluo	DSB-Histo	MoNuSeg
Scribble	GrabCut[8]	0.5288[0.6066]	0.7328[0.7207]	0.8019[0.7815]	0.6969[0.5961]	0.1534[0.0703]
Scribble	Pseudo-Label[6]	0.9126[0.9096]	0.6177[0.6826]	0.8109[0.8136]	0.7888[0.7096]	0.6113[0.5607]
Scribble	pCEOnly[16]	0.9000[0.9032]	0.7954[0.7351]	0.8293[0.8375]	0.7804[0.7173]	0.6319[0.5766]
Scribble	rLoss[16]	0.9108[0.9100]	0.7993[0.7280]	0.8334[0.8394]	0.7873[0.7177]	0.6337[0.5789]
Scribble	Scribble2Label	0.9208[0.9167]	0.8236[0.7663]	0.8426[0.8443]	0.7970[0.7246]	0.6408[0.5811]
Point	Qu[13]	-	-	-	0.5544[0.7204]	0.6099[0.7127]
Full	Full	0.9298[0.9149]	0.8774[0.7879]	0.8688[0.8390]	0.8134[0.7014]	0.7014[0.6677]

Natural Images

Camouflaged Object Detection

• COD10K
• CAMO
• CHAMELEON

Semi or Weak-Supervised Object Detection

• MS-COCO
• PASCAL VOC

• Bees
• CrowdHuman
• Objects365

Salient Object Detection

• DUTS
• ECSSD
• DUT
• PASCAL-S
• HKU-IS
• THUR

Others

Tutorial

• 中文：

1. https://zhuanlan.zhihu.com/p/81404885
2. https://baijiahao.baidu.com/s?id=1632614040925107215&wfr=spider&for=pc

• English：

1. https://ai.stanford.edu/blog/weak-supervision
2. https://www.snorkel.org/blog/weak-supervision
3. Zhou Z H. A brief introduction to weakly supervised learning. National science review, 2018, 5(1): 44-53.

Conclusion

• This repository provides daily-update literature reviews, algorithms' implementation, and some examples of using PyTorch for weak-supervised image segmentation. The project is under development. In the future, it will support 2D and 3D semi-supervised image segmentation and includes five widely-used algorithms' implementations.

• In the next two or three months, we will provide more algorithms' implementations, examples, and pre-trained models.

Questions and Suggestions

• If you have any questions or suggestions about this project, please contact me through email: lichen14@nudt.edu.cn.