[AAAI 2024] Descanning: From Scanned to the Original Images with a Color Correction Diffusion Model

Junghun Cha^1*, Ali Haider^1*, Seoyun Yang¹, Hoeyeong Jin¹, Subin Yang¹, A. F. M. Shahab Uddin², Jaehyoung Kim¹, Soo Ye Kim³, Sung-Ho Bae¹

¹ Kyung Hee University, MLVC Lab., Republic of Korea
² Jashore University of Science and Technology, Bangladesh
³ Adobe Research, USA

This repository is the official PyTorch implementation of "Descanning: From Scanned to the Original Images with a Color Correction Diffusion Model".

We will open our dataset DESCAN-18K and pre-trained models as public soon! Please kindly wait for updates.

[Paper]

💾 Dataset: DESCAN-18K

🔭 Requirements

python >= 3.8  
torch >= 1.10.2
torchvision >= 0.11.3  
tqdm >= 4.62.2  
numpy >= 1.22.1  
opencv-python >= 4.5.4.60  
natsort >= 8.1.0  
matplotlib >= 3.4.3  
Pillow >= 9.4.0  
scipy >= 1.7.3  
scikit-image >= 0.16.2  

pip install -r requirements.txt

💫 Training

To train DescanDiffusion, it is needed to train two modules: Color Encoder and Conditional DDPM.

Color Encoder (Global Color Correction)

1. Configure settings in color_encoder/train_color_encoder.py. (e.g. dataset path, batch size, epochs).

• If you want to log the training process, set logging=True.

2. Execute the below code to train the color encoder.

   python3 color_encoder/train_color_encoder.py
   

3. The last saved model will become color_encoder.h5. It will used to train the conditional DDPM (below part).

Conditional DDPM (Local Generative Refinement)

1. Configure settings in diffusion/train_diffusion.py. (e.g. dataset path, pre-trained color encoder path, steps).

• If you want to log the training process, set logging=True.
• If you want to adjust synthetic data generation probability, adjust distort_threshold. (It is 0.25 in the paper).

2. Execute the below code to train the conditional DDPM.

   python3 diffusion/train_diffusion.py
   

3. The last saved model will become DescanDiffusion.pth. It will used to infer scanned images of the testing set.

💫 Testing

1. Configure settings in diffusion/sampling_diffusion.py. (e.g. testing set path, pre-trained models path, steps).
2. Execute the below code to test our DescanDiffusion.

   python3 diffusion/sampling_diffusion.py
   

3. The inferred images (i.e., descanned images) will be saved into test_DescanDiffusion.

---

Abstract

A significant volume of analog information, i.e., documents and images, have been digitized in the form of scanned copies for storing, sharing, and/or analyzing in the digital world. However, the quality of such contents is severely degraded by various distortions caused by printing, storing, and scanning processes in the physical world. Although restoring high-quality content from scanned copies has become an indispensable task for many products, it has not been systematically explored, and to the best of our knowledge, no public datasets are available. In this paper, we define this problem as Descanning and introduce a new high-quality and large-scale dataset named DESCAN-18K. It contains 18K pairs of original and scanned images collected in the wild containing multiple complex degradations. In order to eliminate such complex degradations, we propose a new image restoration model called DescanDiffusion consisting of a color encoder that corrects the global color degradation and a conditional denoising diffusion probabilistic model (DDPM) that removes local degradations. To further improve the generalization ability of DescanDiffusion, we also design a synthetic data generation scheme by reproducing prominent degradations in scanned images. We demonstrate that our DescanDiffusion outperforms other baselines including commercial restoration products, objectively and subjectively, via comprehensive experiments and analyses.

DescanDiffusion Architecture

Qualitative Comparisons

News

✨ [2024-03-26] Our codes are released.
📃 [2024-02-08] Our paper is uploaded on arXiv.
🎉 [2023-12-09] Our paper is accepted by AAAI 2024.

Citation

If this repository is useful to your research, please consider citing our works! 😊

@article{cha2024descanning,
        title={Descanning: From Scanned to the Original Images with a Color Correction Diffusion Model},
        author={Cha, Junghun and Haider, Ali and Yang, Seoyun and Jin, Hoeyeong and Yang, Subin
                and Uddin, AFM and Kim, Jaehyoung and Kim, Soo Ye and Bae, Sung-Ho},
        journal={arXiv preprint arXiv:2402.05350},
        year={2024}
}