Add decay to CyclicLrUpdater

[HAOCHENYE]

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

I followed #1059 to modify and fixed the cyclic lr decay.

Modification

Fixed lr update error and unit test. The lr curve of config:

lr_config = dict(
    _delete_="step",
    policy='cyclic',
    # anneal_strategy='linear',
    target_ratio=(10, 1),
    cyclic_times=5,
    step_ratio_up=0.4,
    gamma=0.5,
)

lr_config = dict(
    _delete_="step",
    policy='cyclic',
    # anneal_strategy='linear',
    target_ratio=(10, 1e-4),
    cyclic_times=5,
    step_ratio_up=0.4,
    gamma=0.5,
)

lr_config = dict(
    _delete_="step",
    policy='cyclic',
    # anneal_strategy='linear',
    target_ratio=(0.5, 1),
    cyclic_times=5,
    step_ratio_up=0.4,
    gamma=0.5,
) # support target_ratio[0] < 1

BC-breaking (Optional)

No.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

Checklist

Before PR:

• I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
• Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
• Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
• New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

• If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects, like MMDet or MMCls.
• CLA has been signed and all committers have signed the CLA in this PR.

[CLAassistant]

All committers have signed the CLA.

[teamwong111]

LGTM

[zhouzaida]

It would be nice if an image can be provided in the PR message to visualize the learning rate. Maybe you can refer to https://mmclassification.readthedocs.io/en/latest/tools/visualization.html#learning-rate-schedule-visualization.

[zhouzaida]

We also need to add the two arguments anneal_strategy and gamma for CyclicMomentumUpdaterHook.

mmcv/mmcv/runner/hooks/momentum_updater.py

Line 226 in 86ed509

class CyclicMomentumUpdaterHook(MomentumUpdaterHook):

[zhouzaida]

The PR can be merged after resolving the comments.