Skip to content

Latest commit

 

History

History
152 lines (120 loc) · 10.5 KB

README.md

File metadata and controls

152 lines (120 loc) · 10.5 KB
Raw

📖 Spatially-Adaptive Feature Modulation for Efficient Image Super-Resolution

google colab logo OpenXLab Hugging Face Demo Hugging Face Models video visitors GitHub Stars
[Paper]   [Supp]

Long Sun, Jiangxin Dong, Jinhui Tang, and Jinshan Pan
IMAG Lab, Nanjing University of Science and Technology

An overview of the proposed SAFMN. SAFMN first transforms the input LR image into the feature space using a convolutional layer, performs feature extraction using a series of feature mixing modules (FMMs), and then reconstructs these extracted features by an upsampler module. The FMM block is mainly implemented by a spatially-adaptive feature modulation (SAFM) layer and a convolutional channel mixer (CCM).

🚩 News

  • [2024-07-16] Add 🤗HuggingFace online demo!
  • [2024-05-08] Add the light_SAFMN++, which is the 1st place winner of the fidelity track of the Real-time 4K Super Resolution Challenge for compressed AVIF images, and is invited to give an oral presentation at the AIS2024 workshop.
  • [2024-03-27] Add the improved ESR model SAFMN++, which ranks Top4 in the Overall Performance track and Runtime track of NTIRE2024 ESR challenge
  • [2024-03-16] Add SAFMN_BCIE, which enhances the quality of JPEG images compressed with a large range of quality factors. The corresponding inference code can be found here.
  • [2023-11-22] The code for ONNX export is available here
  • [2023-09-08] Add 🐼 OpenXLab online demo!
  • [2023-08-31] Update SAFMN_Real_x4.pth
  • [2023-08-31] Add SAFMN_Real_x2.pth
  • [2023-08-21] Colab demo for SAFMN on x4 real-world image SR is available google colab logo
  • [2023-07-14] Our SAFMN is accepted to ICCV 2023
  • [2023-06-06] The report of NTIRE 2023 Challenge on Efficient Super-Resolution is available here
  • [2023-04-17] The SAFMN ranks Top6 for overall performance in the NTIRE2023 ESR challenge.
  • [2023-04-17] The SAFMN variant ranks Top3 for model complexity in the NTIRE2023 ESR Challenge.
  • [2023-03-22] The code and checkpoint for the NTIRE2023 Efficient Super-Resolution Challenge is available here.
  • [2023-03-22] The pre-trained model with high-order degradation on the LSDIR dataset is available.
  • [2023-03-13] The source codes, checkpoints and visual results are available.
  • [2023-02-26] The paper is available here.

👀 Demos

  • Real-world SR Results
Real-World Image (x4) Real-ESRGAN SwinIR SAFMN (ours)
  • Results on Blind Compressed Images

🔧 Requirements and Installation

  • Python 3.8, PyTorch >= 1.11
  • BasicSR 1.4.2
  • Platforms: Ubuntu 18.04, cuda-11

Installation

# Clone the repo
git clone https://github.com/sunny2109/SAFMN.git
# Install dependent packages
cd SAFMN
pip install -r requirements.txt
# Install BasicSR
python setup.py develop

You can also refer to this INSTALL.md for installation

Training and Testing

Training

Run the following commands for training:

# train SAFMN for x4 effieicnt SR
python basicsr/train.py -opt options/train/SAFMN/train_DF2K_x4.yml
# train SAFMN for x4 classic SR
python basicsr/train.py -opt options/train/SAFMN/train_L_DF2K_x4.yml

⚡ Quick Inference

  • Download the pretrained models.
  • Download the testing dataset.
  • Run the following commands:
# test SAFMN for x4 efficient SR
python basicsr/test.py -opt options/test/SAFMN/test_benchmark_x4.yml
# test SAFMN for x4 classic SR
python basicsr/test.py -opt options/test/SAFMN/test_L_benchmark_x4.yml
# test SAFMN for x4 real-world SR (without ground-truth)
python basicsr/test.py -opt options/test/SAFMN/test_real_img_x4.yml
# test SAFMN for x4 real-world SR (large input)
python inference/inference_real_safmn.py --input test_demo --output results/test_demo --scale 4 --large_input
  • The test results will be in './results'.

Pretrained Models and Results

  • Pretrained Models

    We have provided three ways to download our checkpoints.

Degradation Model Zoo Visual Results
BI-Efficient SR Google Drive/Baidu Netdisk with code: SAFM Google Drive/Baidu Netdisk with code: SAFM
BI-Classic SR Google Drive/Baidu Netdisk with code: SAFM Google Drive/Baidu Netdisk with code: SAFM
x4 Real-world Google Drive/Baidu Netdisk with code: SAFM
  • Efficient SR Results

  • Classic SR Results

Citation

If this work is helpful for your research, please consider citing the following BibTeX entry.

@inproceedings{sun2023safmn,
    title={Spatially-Adaptive Feature Modulation for Efficient Image Super-Resolution},
    author={Sun, Long and Dong, Jiangxin and Tang, Jinhui and Pan, Jinshan},
    booktitle={ICCV},
    year={2023}
 }

🧩 Projects that use SAFMN

If you develop/use SAFMN in your projects, welcome to let me know.

📧 Contact

If you have any questions, please feel free to reach me out at cs.longsun@gmail.com

🤗 Acknowledgement

This code is based on BasicSR toolbox. Thanks for the awesome work.