[Feature] Support fp16 training

[ckkelvinchan]

This PR adds support to fp16 training. Tested on BasicVSR.

Note: This PR is currently in progress, will add unittests in soon future.

[codecov]

Codecov Report

Merging #320 (f1fc363) into master (613b260) will decrease coverage by 1.22%.
The diff coverage is 50.29%.

❗ Current head f1fc363 differs from pull request most recent head 8489c93. Consider uploading reports for the commit 8489c93 to get more accurate results

@@            Coverage Diff             @@
##           master     #320      +/-   ##
==========================================
- Coverage   80.65%   79.43%   -1.23%     
==========================================
  Files         162      168       +6     
  Lines        8110     8425     +315     
  Branches     1193     1236      +43     
==========================================
+ Hits         6541     6692     +151     
- Misses       1425     1584     +159     
- Partials      144      149       +5     

Flag	Coverage Δ
unittests	79.43% <50.29%> (-1.23%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmedit/core/runner/checkpoint.py	16.98% <16.98%> (ø)
mmedit/apis/train.py	20.45% <22.22%> (-0.54%)	⬇️
mmedit/core/runner/iter_based_fp16_runner.py	23.14% <23.14%> (ø)
mmedit/core/runner/apex_amp_utils.py	38.46% <38.46%> (ø)
mmedit/models/restorers/basicvsr.py	61.32% <41.66%> (-7.92%)	⬇️
mmedit/core/__init__.py	100.00% <100.00%> (ø)
mmedit/core/runner/__init__.py	100.00% <100.00%> (ø)
mmedit/datasets/pipelines/__init__.py	100.00% <100.00%> (ø)
mmedit/datasets/pipelines/augmentation.py	97.04% <100.00%> (+0.04%)	⬆️
mmedit/datasets/pipelines/loading.py	95.26% <100.00%> (+0.02%)	⬆️
... and 11 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 613b260...8489c93. Read the comment docs.

[nijkah]

This is just a small question.
Actually, I raised a similar issue open-mmlab/mmcv#1719 in the mmcv to support Apex.
But maintainers explained that mixed precision training is supported by Fp16OptimizerHook.
Is there any reason to use apex?

[zengyh1900]

This pull request is closed because it has been open for 30 days with no activity.