Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach

Lingchen Sun^1,2 | Rongyuan Wu^1,2 | Zhiyuan Ma¹ | Shuaizheng Liu^1,2 | Qiaosi Yi^1,2 | Lei Zhang^1,2

¹The Hong Kong Polytechnic University, ²OPPO Research Institute

⏰ Update

The code and model will be ready soon.

• 2024.12.4: The paper and this repo are released.

⭐ If PiSA-SR is helpful to your images or projects, please help star this repo. Thanks! 🤗

🌟 Overview Framework

(a) Training procedure of PiSA-SR. During the training process, two LoRA modules are respectively optimized for pixel-level and semantic-level enhancement.

(b) Inference procedure of PiSA-SR. During the inference stage, users can use the default setting to reconstruct the high-quality image in one-step diffusion or adjust λ_pix and λ_sem to control the strengths of pixel-level and semantic-level enhancement.

😍 Visual Results

Adjustable SR Results

By increasing the guidance scale λ_pix on the pixel-level LoRA module, the image degradations such as noise and compression artifacts can be gradually removed; however, a too-strong λ_pix will make the SR image over-smoothed. By increasing the guidance scale λ_sem on the semantic-level LoRA module, the SR images will have more semantic details; nonetheless, a too-high λ_sem will generate visual artifacts.

Comparisons with Other DM-Based SR Methods

Citations

If our code helps your research or work, please consider citing our paper. The following are BibTeX references:

@article{sun2024pisasr,
  title={Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach},
  author={Sun, Lingchen and Wu, Rongyuan and Ma, Zhiyuan and Liu, Shuaizheng and Yi, Qiaosi and Zhang, Lei},
  journal={arXiv preprint arXiv:2412.03017},
  year={2024}
}

License

This project is released under the Apache 2.0 license.

Acknowledgement

Contact

If you have any questions, please contact: ling-chen.sun@connect.polyu.hk

statistics