L1BSR: Exploiting Detector Overlap for Self-Supervised SISR of Sentinel-2 L1B Imagery

Ngoc Long Nguyen, Jérémy Anger, Axel Davy, Pablo Arias, Gabriele Facciolo

Centre Borelli, ENS Paris-Saclay

---

This repository is the official PyTorch implementation of L1BSR: Exploiting Detector Overlap for Self-Supervised SISR of Sentinel-2 L1B Imagery (Best Student Paper at EarthVision 2023).

L1BSR produces a 5m high-resolution (HR) output with all bands correctly registered from a single 10m low-resolution (LR) Sentinel-2 L1B image with misaligned bands. Note that L1BSR is trained on real data with self-supervision, i.e. without any ground truth HR targets.

Contents

1. Overview
2. Testing
3. Training
4. Citation
5. License and Acknowledgement

Overview

There are two key modules integral to the training of the L1BSR:

1. The REConstruction (REC) module: performs joint super-resolution and band-alignment for the L1B BGRN data.
2. The Cross-Spectral Registration (CSR) module: produces a dense flow between 2 images of different spectral bands.

Both modules are trained with self-supervision. Note that the CSR is used only during the training of L1BSR, whereas at inference, only the REC is needed.

Testing

For your convenience we provide some test BGRN images (~10Mb) in /examples.

If you want a quick inspection of our two key modules REC and CSR, checkout our IPOL demo

We also provide the testing code main.py. Like in the demo, you can choose the task (super-resolution or cross-spectral registration) for our networks (REC or CSR, respectively) to perform.

Examples:

# Super-resolution: This code below super-resolves (x2) the image in "examples/00.tif"
# and saves it in "output.tif".
python main.py examples/00.tif output.tif --device cuda --task superresolution
# Cross-spectral registration: This code below aligns the bands Blue, Red, and NIR of
# the image in "examples/00.tif" to its Green band and saves the output in "output.tif".
python main.py examples/00.tif output.tif --device cuda --task registration

Training

The training codes for both the CSR and REC modules will be soon available. Stay tuned!

Citation

@inproceedings{nguyen2023l1bsr,
    title={L1BSR: Exploiting Detector Overlap for Self-Supervised Single-Image Super-Resolution of Sentinel-2 L1B Imagery},
    author={Nguyen, Ngoc Long and Anger, J{\'e}r{\'e}my and Davy, Axel and Arias, Pablo and Facciolo, Gabriele},
    booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    pages={2012--2022},
    year={2023}
    }

License and Acknowledgement

This project is released under the GPL-3.0 license. The codes are based on RCAN. Please also follow their licenses. Thanks for their awesome works.