This repository provides extensive examples of synthetic liver tumors generated by our novel strategies. Check to see if you could tell which is real tumor and which is synthetic tumor. More importantly, our synthetic tumors can be used for training AI models, and have proven to achieve a similar (actually, better) performance in real tumor segmentation than a model trained on real tumors.

Amazing, right?

MedCraft
│  main.py
│  monai_trainer.py // Training script using MONAI framework
│  transfer_label.py
│  tumor_analyzer.py // Analyzes tumor data
│  tumor_saver.py  // Saves tumor data
│  validation.py // Validation script
│
├─datafolds
│
├─external
│  └─surface-distance // External library for surface distance calculations
│
├─networks // Contains various neural network architectures
│
├─networks2 // Alternative implementations of network architectures
│
├─optimizers
│      lr_scheduler.py // Learning rate scheduler
│      __init__.py
│
└─TumorGenerated
        TumorGenerated.py
        utils.py // Utility functions for tumor generation
        filter.py
        __init__.py

Model

Organ	Tumor	Model	Pre-trained?	Download
liver	real	unet	no	link
liver	real	swin_unetrv2_base	no	link
liver	synt	unet	no	link
liver	synt	swin_unetrv2_base	no	link
pancreas	real	unet	no	link
pancreas	real	swin_unetrv2_base	no	link
pancreas	synt	unet	no	link
pancreas	synt	swin_unetrv2_base	no	link
kidney	real	unet	no	link
kidney	real	swin_unetrv2_base	no	link
kidney	synt	unet	no	link
kidney	synt	swin_unetrv2_base	no	link

You can download other materials from these links:

All other checkpoints: link

Data: Liver (link), Kidney (link), Pancreas (link)

0. Installation

Dataset

Please download these datasets and save to <data-path> (user-defined).

• 01 Multi-Atlas Labeling Beyond the Cranial Vault - Workshop and Challenge (BTCV)
• 02 Pancreas-CT TCIA
• 03 Combined Healthy Abdominal Organ Segmentation (CHAOS)
• 04 Liver Tumor Segmentation Challenge (LiTS)

wget https://www.dropbox.com/s/jnv74utwh99ikus/01_Multi-Atlas_Labeling.tar.gz # 01 Multi-Atlas_Labeling.tar.gz (1.53 GB)
wget https://www.dropbox.com/s/5yzdzb7el9r3o9i/02_TCIA_Pancreas-CT.tar.gz # 02 TCIA_Pancreas-CT.tar.gz (7.51 GB)
wget https://www.dropbox.com/s/lzrhirei2t2vuwg/03_CHAOS.tar.gz # 03 CHAOS.tar.gz (925.3 MB)
wget https://www.dropbox.com/s/2i19kuw7qewzo6q/04_LiTS.tar.gz # 04 LiTS.tar.gz (17.42 GB)
wget https://huggingface.co/datasets/qicq1c/Pubilcdataset/resolve/main/10_Decathlon/Task03_Liver.tar.gz?download=true # Task03_Liver.tar.gz (28.7 GB)

Data Setting

# Task03_Liver training data list
--json_dir /datafolds/fold_0.json
--json_dir /datafolds/fold_1.json
--json_dir /datafolds/fold_2.json
--json_dir /datafolds/fold_3.json
--json_dir /datafolds/fold_4.json

Dependency

The code is tested on python 3.8, Pytorch 1.11.

conda create -n medcraft python=3.8
source activate medcraft (or conda activate medcraft)
cd MedCraft
pip install external/surface-distance
pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu113
pip install -r requirements.txt

Label

Our synthetic algorithm requires label as 0: background, 1: liver, you need to transfer the label before training AI model.

python transfer_label.py --data_path <data-path>  # <data-path> is user-defined data path to save datasets

or you can just download the label

wget https://www.dropbox.com/s/8e3hlza16vor05s/label.zip

1. Train segmentation models using synthetic tumors

conda activate medcraft
cd MedCraft
train_path=datafolds/healthy_ct
val_path=datafolds/10_Decathlon/Task03_Liver
fold=0
dist=$((RANDOM % 99999 + 10000))

# UNET (no.pretrain)
python -W ignore main.py --optim_lr=4e-4 --batch_size=2 --lrschedule=warmup_cosine --optim_name=adamw --model_name=unet --val_every=200 --max_epochs=4000 --save_checkpoint --workers=2 --noamp --distributed --dist-url=tcp://127.0.0.1:$dist --cache_num=200 --val_overlap=0.5 --syn --ellipsoid --logdir="runs/synt.no_pretrain.unet$fold" --train_dir $train_path --val_dir $val_path --json_dir datafolds/fold_$fold.json

2. Train segmentation models using real tumors (for comparison)

conda activate medcraft
cd MedCraft
train_path=datafolds/healthy_ct
val_path=datafolds/10_Decathlon/Task03_Liver
fold=0
dist=$((RANDOM % 99999 + 10000))

# UNET (no.pretrain)
python -W ignore main.py --optim_lr=4e-4 --batch_size=2 --lrschedule=warmup_cosine --optim_name=adamw --model_name=unet --val_every=200 --max_epochs=4000 --save_checkpoint --workers=2 --noamp --distributed --dist-url=tcp://127.0.0.1:$dist --cache_num=200 --val_overlap=0.5 --logdir="runs/real.no_pretrain.unet$fold" --train_dir $train_path --val_dir $val_path --json_dir datafolds/gmm_fold_$fold.json

3. Evaluation

AI model trained by synthetic tumors

conda activate medcraft
cd MedCraft
val_path=datafolds/10_Decathlon/Task03_Liver
fold=0

# UNET (no.pretrain)
python -W ignore validation.py --model=unet --val_overlap=0.75 --val_dir $val_path --json_dir datafolds/fold_$fold.json --log_dir runs/synt.no_pretrain.unet$fold --save_dir outs

AI model trained by real tumors

conda activate medcraft
cd MedCraft
val_path=datafolds/10_Decathlon/Task03_Liver
fold=0

# UNET (no.pretrain)
python -W ignore validation.py --model=unet --val_overlap=0.75 --val_dir $val_path --json_dir datafolds/gmm_fold_$fold.json --log_dir runs/real.no_pretrain.unet$fold --save_dir outs

Related Papers

FreeTumor: Advance Tumor Segmentation via Large-Scale Tumor Synthesis Linshan Wu, Jiaxin Zhuang, Xuefeng Ni, Hao Chen arXiv | 3 Jun 2024 paper

From Pixel to Cancer: Cellular Automata in Computed Tomography Yuxiang Lai, Xiaoxi Chen, Angtian Wang, Alan Yuille, Zongwei Zhou MICCAI | 13 May 2024 paper

Generative Enhancement for 3D Medical Images Zhu, Lingting and Codella, Noel and Chen, Dongdong and Jin, Zhenchao and Yuan, Lu and Yu, Lequan arXiv preprint arXiv:2403.12852 | 19 Mar 2024 paper

Towards Generalizable Tumor Synthesis Qi Chen, Xiaoxi Chen, Haorui Song, Zhiwei Xiong, Alan Yuille, Chen Wei, Zongwei Zhou CVPR | 29 Feb 2024 paper

SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentation Zhaohu Xing, Tian Ye, Yijun Yang, Guang Liu, Lei Zhu arXiv preprint arXiv:2401.13560 | 25 Feb 2024 paper

Label-Free Liver Tumor Segmentation Qixin Hu, Yixiong Chen, Junfei Xiao, Shuwen Sun, Jieneng Chen, Alan Yuille, Zongwei Zhou CVPR | 27 March 2023 paper