Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update weights and re-evaluate models #274

Merged
merged 3 commits into from
Apr 25, 2021
Merged

Update weights and re-evaluate models #274

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
658de2f
Update weights and re-evaluate models
ckkelvinchan Apr 24, 2021
9d30c03
Update links to weights
ckkelvinchan Apr 25, 2021
3c1612e
Add back SSIM to configuration
ckkelvinchan Apr 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions configs/restorers/basicvsr/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@

Evaluated on RGB channels for REDS4 and Y channel for others. The metrics are `PSNR`/`SSIM`.

The pretrained weights of SPyNet can be found [here](https://download.openmmlab.com/mmediting/restorers/basicvsr/spynet_20210409-c6c1bd09.pth) .
The pretrained weights of SPyNet can be found [here](https://download.openmmlab.com/mmediting/restorers/basicvsr/spynet_20210409-c6c1bd09.pth).


| Method | REDS4 (BIx4)<br>PSNR/SSIM (RGB) | Vimeo-90K-T (BIx4)<br>PSNR/SSIM (Y) | Vid4 (BIx4)<br>PSNR/SSIM (Y) | UDM10 (BDx4)<br>PSNR/SSIM (Y) | Vimeo-90K-T (BDx4)<br>PSNR/SSIM (Y) | Vid4 (BDx4)<br>PSNR/SSIM (Y) | Download |
|:------------------------------------------------------------------------------------------------------------------------------:|:-------------------------------:|:-----------------------------------:|:----------------------------:|:-----------------------------:|:-----------------------------------:|:----------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| [basicvsr_reds4](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/basicvsr/basicvsr_reds4.py) | **31.4170/0.8909** | 36.2870/0.9388 | 27.2223/0.8298 | 33.4510/0.9297 | 34.5053/0.9280 | 24.4390/0.7441 | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_reds4_20120409-089ec9ef.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_reds4_20210409_092646.log.json) |
| [basicvsr_vimeo90k_bi](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/basicvsr/basicvsr_vimeo90k_bi.py) | 30.3128/0.8660 | **37.2299/0.9447** | **27.2296/0.8227** | 34.5488/0.9423 | 34.8713/0.9313 | 25.0377/0.7622 | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_vimeo90k_bi_20210409-7b3e0c43.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_vimeo90k_bi_20210409_132702.log.json) |
| [basicvsr_vimeo90k_bd](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/basicvsr/basicvsr_vimeo90k_bd.py) | 29.0376/0.8481 | 34.7094/0.9332 | 26.2356/0.8000 | **39.8802/0.9683** | **37.5730/0.9495** | **27.9278/0.8537** | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_vimeo90k_bd_20210409-6b10d46a.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_vimeo90k_bd_20210409_132740.log.json) |
| [basicvsr_reds4](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/basicvsr/basicvsr_reds4.py) | **31.4170/0.8909** | 36.2848/0.9395 | 27.2694/0.8318 | 33.4478/0.9306 | 34.4700/0.9286 | 24.4541/0.7455 | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_reds4_20120409-0c415450.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_reds4_20210409_092646.log.json) |
| [basicvsr_vimeo90k_bi](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/basicvsr/basicvsr_vimeo90k_bi.py) | 30.3128/0.8660 | **37.2026/0.9451** | **27.2755/0.8248** | 34.5554/0.9434 | 34.8097/0.9316 | 25.0517/0.7636 | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_vimeo90k_bi_20210409-400e5dbc.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_vimeo90k_bi_20210409_132702.log.json) |
| [basicvsr_vimeo90k_bd](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/basicvsr/basicvsr_vimeo90k_bd.py) | 29.0376/0.8481 | 34.6427/0.9335 | 26.2708/0.8022 | **39.9953/0.9695** | **37.5501/0.9499** | **27.9791/0.8556** | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_vimeo90k_bd_20210409-af7b6371.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/basicvsr/basicvsr_vimeo90k_bd_20210409_132740.log.json) |
8 changes: 4 additions & 4 deletions configs/restorers/iconvsr/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@

Evaluated on RGB channels for REDS4 and Y channel for others. The metrics are `PSNR`/`SSIM`.

The pretrained weights of the IconVSR components can be found here: [SPyNet](https://download.openmmlab.com/mmediting/restorers/basicvsr/spynet_20210409-c6c1bd09.pth), [EDVR-M for REDS](https://download.openmmlab.com/mmediting/restorers/iconvsr/edvrm_reds_20210413-3867262f.pth), and [EDVR-M for Vimeo-90K](https://download.openmmlab.com/mmediting/restorers/iconvsr/edvrm_vimeo90k_20210413-e40e99a8.pth) .
The pretrained weights of the IconVSR components can be found here: [SPyNet](https://download.openmmlab.com/mmediting/restorers/basicvsr/spynet_20210409-c6c1bd09.pth), [EDVR-M for REDS](https://download.openmmlab.com/mmediting/restorers/iconvsr/edvrm_reds_20210413-3867262f.pth), and [EDVR-M for Vimeo-90K](https://download.openmmlab.com/mmediting/restorers/iconvsr/edvrm_vimeo90k_20210413-e40e99a8.pth).


| Method | REDS4 (BIx4)<br>PSNR/SSIM (RGB) | Vimeo-90K-T (BIx4)<br>PSNR/SSIM (Y) | Vid4 (BIx4)<br>PSNR/SSIM (Y) | UDM10 (BDx4)<br>PSNR/SSIM (Y) | Vimeo-90K-T (BDx4)<br>PSNR/SSIM (Y) | Vid4 (BDx4)<br>PSNR/SSIM (Y) | Download |
|:---------------------------------------------------------------------------------------------------------------------------:|:-------------------------------:|:-----------------------------------:|:----------------------------:|:-----------------------------:|:-----------------------------------:|:----------------------------:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| [iconvsr_reds4](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/iconvsr/iconvsr_reds.py) | **31.6926/0.8951** | 36.4963/0.9410 | **27.4350/0.8334** | 35.3336/0.9463 | 34.4626/0.9280 | 25.2045/0.7721 | [model](https://download.openmmlab.com/mmediting/restorers/iconvsr/iconvsr_reds4_20210413-74343d4f.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/iconvsr/iconvsr_reds4_20210413_222735.log.json) |
| [iconvsr_vimeo90k_bi](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/iconvsr/iconvsr_vimeo90k_bi.py) | 30.3452/0.8659 | **37.4008/0.9464** | 27.3738/0.8274 | 34.2639/0.9388 | 34.6193/0.9293 | 24.6648/0.7480 | [model](https://download.openmmlab.com/mmediting/restorers/iconvsr/iconvsr_vimeo90k_bi_20210413-ba2ffd72.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/iconvsr/iconvsr_vimeo90k_bi_20210413_222757.log.json) |
| [iconvsr_vimeo90k_bd](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/iconvsr/iconvsr_vimeo90k_bd.py) | 29.0150/0.8465 | 34.7454/0.9336 | 26.2763/0.8006 | **39.9403/0.8006** | **37.7794/0.9513** | **28.1916/0.8592** | [model](https://download.openmmlab.com/mmediting/restorers/iconvsr/iconvsr_vimeo90k_bd_20210414-292a63ac.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/iconvsr/iconvsr_vimeo90k_bd_20210414_084128.log.json) |
| [iconvsr_reds4](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/iconvsr/iconvsr_reds.py) | **31.6926/0.8951** | 36.4983/0.9416 | **27.4809/0.8354** | 35.3377/0.9471 | 34.4299/0.9287 | 25.2110/0.7732 | [model](https://download.openmmlab.com/mmediting/restorers/iconvsr/iconvsr_reds4_20210413-d2e1da11.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/iconvsr/iconvsr_reds4_20210413_222735.log.json) |
| [iconvsr_vimeo90k_bi](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/iconvsr/iconvsr_vimeo90k_bi.py) | 30.3452/0.8659 | **37.3729/0.9467** | 27.4238/0.8297 | 34.2595/0.9398 | 34.5548/0.9295 | 24.6666/0.7491 | [model](https://download.openmmlab.com/mmediting/restorers/iconvsr/iconvsr_vimeo90k_bi_20210413-3fcc5198.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/iconvsr/iconvsr_vimeo90k_bi_20210413_222757.log.json) |
| [iconvsr_vimeo90k_bd](https://github.com/open-mmlab/mmediting/blob/master/configs/restorers/iconvsr/iconvsr_vimeo90k_bd.py) | 29.0150/0.8465 | 34.6780/0.9339 | 26.3109/0.8028 | **40.0640/0.9697** | **37.7573/0.9517** | **28.2464/0.8612** | [model](https://download.openmmlab.com/mmediting/restorers/iconvsr/iconvsr_vimeo90k_bd_20210414-3f3b0201.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/iconvsr/iconvsr_vimeo90k_bd_20210414_084128.log.json) |
4 changes: 2 additions & 2 deletions configs/restorers/iconvsr/iconvsr_vimeo90k_bd.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
pixel_loss=dict(type='CharbonnierLoss', loss_weight=1.0, reduction='mean'))
# model training and testing settings
train_cfg = dict(fix_iter=5000)
test_cfg = dict(metrics=['PSNR', 'SSIM'], crop_border=0, convert_to='y')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is removing 'SSIM' intended?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, as it would slow down the training. Therefore I put only PSNR here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Users may remove 'SSIM', or together with 'PSNR' themselves if they want to speed up training. It's better to leave it here since our result page contains both SSIM and PSNR.

test_cfg = dict(metrics=['PSNR'], crop_border=0, convert_to='y')

# dataset settings
train_dataset_type = 'SRVimeo90KMultipleGTDataset'
Expand Down Expand Up @@ -117,7 +117,7 @@
type=test_dataset_type,
lq_folder='data/vimeo90k/BDx4',
gt_folder='data/vimeo90k/GT',
ann_file='data/Vimeo90K/meta_info_Vimeo90K_test_GT.txt',
ann_file='data/vimeo90k/meta_info_Vimeo90K_test_GT.txt',
pipeline=test_pipeline,
scale=4,
num_input_frames=7,
Expand Down
4 changes: 2 additions & 2 deletions configs/restorers/iconvsr/iconvsr_vimeo90k_bi.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
pixel_loss=dict(type='CharbonnierLoss', loss_weight=1.0, reduction='mean'))
# model training and testing settings
train_cfg = dict(fix_iter=5000)
test_cfg = dict(metrics=['PSNR', 'SSIM'], crop_border=0, convert_to='y')
test_cfg = dict(metrics=['PSNR'], crop_border=0, convert_to='y')

# dataset settings
train_dataset_type = 'SRVimeo90KMultipleGTDataset'
Expand Down Expand Up @@ -117,7 +117,7 @@
type=test_dataset_type,
lq_folder='data/vimeo90k/BIx4',
gt_folder='data/vimeo90k/GT',
ann_file='data/Vimeo90K/meta_info_Vimeo90K_test_GT.txt',
ann_file='data/vimeo90k/meta_info_Vimeo90K_test_GT.txt',
pipeline=test_pipeline,
scale=4,
num_input_frames=7,
Expand Down