Support receptive field search of CNN models. (TPAMI paper rf-next) #2056

gasvn · 2022-06-15T04:44:42Z

Motivation

Merging a general receptive field search method to mmcv and mmdet.
(paper: RF-Next: Efficient Receptive Field Search for Convolutional Neural Networks TPAMI 2022 pdf)

Modification

The RFSearch module is included.

BC-breaking (Optional)

Does the modification introduce changes that break the backward-compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

This pr is a part of pr in mmdet open-mmlab/mmdetection#8191

Checklist

Before PR:

I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects, like MMDet or MMCls.
CLA has been signed and all committers have signed the CLA in this PR.

CLAassistant · 2022-06-15T04:44:47Z

All committers have signed the CLA.

mmcv/cnn/rfsearch/search.py

zhouzaida · 2022-06-15T12:51:01Z

Hi @gasvn , thanks for your contributions. Please add docstring and type hints for your added files.

gasvn · 2022-06-16T14:00:38Z

Hi @gasvn , thanks for your contributions. Please add docstring and type hints for your added files.

I have added docstring and type hints.

gasvn · 2022-07-01T03:25:57Z

Is there any further update about this pull request?

mmcv/cnn/rfsearch/operator.py

jbwang1997 · 2022-07-01T06:51:40Z

mmcv/cnn/rfsearch/operator.py

+
+class ConvRFSearchOp(nn.Module):
+
+    def __init__(self, op_layer: nn.Module, global_config: Dict = {}):


Is better to set the default global_confg as None. The dictionary may cause some unexpected problems.

Changing the global_config to None cannot pass the code checking. So we keep it unchanged for now.

mmcv/cnn/rfsearch/operator.py

mmcv/cnn/rfsearch/utils.py

mmcv/cnn/rfsearch/search.py

gasvn · 2022-08-29T10:34:34Z

Is there anything I can do to make some progress? Thanks~

zhouzaida · 2022-08-29T16:42:21Z

Is there anything I can do to make some progress? Thanks~

Hi @gasvn , sorry for my late reply. I'm thinking about where to put the implementation of this hook.

mmcv/cnn/rfsearch/search.py

mmcv/cnn/rfsearch/operator.py

gasvn · 2022-10-03T07:11:09Z

Hi, I have solved the comments above. Please let me know if there is anything I can do. Thanks.

zhouzaida · 2022-10-26T16:38:03Z

Hi, I have solved the comments above. Please let me know if there is anything I can do. Thanks.

Hi @gasvn , sorry for my late reply.

mmcv/cnn/rfsearch/operator.py

mmcv/cnn/rfsearch/search.py

lzyhha · 2022-11-21T12:14:14Z

Hello, it seems that there is an error in build_cu102 (Error: retrieving gpg key timed out.). Is it a package that failed to download? @zhouzaida

lzyhha · 2022-11-21T14:03:33Z

I have tested this latest commit with RF-mmdetection, and everything is OK. And the RF-mmdetection has been updated following the changes in this latest commit.

And what is the schedule to merge this PR?

@zhouzaida

zhouzaida · 2022-11-21T16:48:26Z

I re-run the job and it should be OK.

mmcv/cnn/__init__.py

zhouzaida · 2022-11-21T16:55:01Z

I have tested this latest commit with RF-mmdetection, and everything is OK. And the RF-mmdetection has been updated following the changes in this latest commit.

And what is the schedule to merge this PR?

@zhouzaida

Hi @lzyhha , this PR looks good to me.

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

lzyhha · 2022-11-22T03:10:44Z

Hello, thanks for your working. May I know when will this PR be completed and merged? @zhouzaida

mmcv/cnn/rfsearch/operator.py

zytx121

LGTM

Co-authored-by: Yue Zhou <592267829@qq.com>

mmcv/cnn/rfsearch/search.py

mmcv/cnn/rfsearch/operator.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

zhouzaida · 2022-11-22T07:17:45Z

Thanks for @gasvn and @lzyhha contributions. This PR will be merged after CI passes.

gasvn · 2022-11-22T11:04:11Z

Thanks for @gasvn and @lzyhha contributions. This PR will be merged after CI passes.

Thanks everyone for your help!!!!!!

lzyhha · 2022-12-15T12:29:50Z

I create a new PR to support RF-Next in more applications, e.g., mmsegmentation. @zhouzaida

* support rfsearch * add labs for rfsearch * format * format * add docstring and type hints * clean code Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * rm unused func * update code * update code * update code * update details * fix details * support asymmetric kernel * support asymmetric kernel * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review * add unit tests for rfsearch * set device for Conv2dRFSearchOp * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * remove unused function search_estimate_only * move unit tests * Update tests/test_cnn/test_rfsearch/test_operator.py Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Update mmcv/cnn/rfsearch/operator.py Co-authored-by: Yue Zhou <592267829@qq.com> * change logger * Update mmcv/cnn/rfsearch/operator.py Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> Co-authored-by: lzyhha <819814373@qq.com> Co-authored-by: Zhongyu Li <44114862+lzyhha@users.noreply.github.com> Co-authored-by: Yue Zhou <592267829@qq.com>

* support rfsearch * add labs for rfsearch * format * format * add docstring and type hints * clean code Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * rm unused func * update code * update code * update code * update details * fix details * support asymmetric kernel * support asymmetric kernel * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review * add unit tests for rfsearch * set device for Conv2dRFSearchOp * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * remove unused function search_estimate_only * move unit tests * Update tests/test_cnn/test_rfsearch/test_operator.py Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Update mmcv/cnn/rfsearch/operator.py Co-authored-by: Yue Zhou <592267829@qq.com> * change logger * Update mmcv/cnn/rfsearch/operator.py Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> Co-authored-by: lzyhha <819814373@qq.com> Co-authored-by: Zhongyu Li <44114862+lzyhha@users.noreply.github.com> Co-authored-by: Yue Zhou <592267829@qq.com> [Fix] Fix skip_layer for RF-Next (open-mmlab#2489) * judge skip_layer by fullname * lint * skip_layer first * update unit test

* support rfsearch * add labs for rfsearch * format * format * add docstring and type hints * clean code Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * rm unused func * update code * update code * update code * update details * fix details * support asymmetric kernel * support asymmetric kernel * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review * add unit tests for rfsearch * set device for Conv2dRFSearchOp * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * remove unused function search_estimate_only * move unit tests * Update tests/test_cnn/test_rfsearch/test_operator.py Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Update mmcv/cnn/rfsearch/operator.py Co-authored-by: Yue Zhou <592267829@qq.com> * change logger * Update mmcv/cnn/rfsearch/operator.py Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> Co-authored-by: lzyhha <819814373@qq.com> Co-authored-by: Zhongyu Li <44114862+lzyhha@users.noreply.github.com> Co-authored-by: Yue Zhou <592267829@qq.com> [Fix] Fix skip_layer for RF-Next (#2489) * judge skip_layer by fullname * lint * skip_layer first * update unit test

* support rfsearch * add labs for rfsearch * format * format * add docstring and type hints * clean code Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * rm unused func * update code * update code * update code * update details * fix details * support asymmetric kernel * support asymmetric kernel * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review * add unit tests for rfsearch * set device for Conv2dRFSearchOp * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * remove unused function search_estimate_only * move unit tests * Update tests/test_cnn/test_rfsearch/test_operator.py Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Update mmcv/cnn/rfsearch/operator.py Co-authored-by: Yue Zhou <592267829@qq.com> * change logger * Update mmcv/cnn/rfsearch/operator.py Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> Co-authored-by: lzyhha <819814373@qq.com> Co-authored-by: Zhongyu Li <44114862+lzyhha@users.noreply.github.com> Co-authored-by: Yue Zhou <592267829@qq.com> [Fix] Fix skip_layer for RF-Next (open-mmlab#2489) * judge skip_layer by fullname * lint * skip_layer first * update unit test

mm-assistant bot assigned imabackstabber Jun 15, 2022

gasvn mentioned this pull request Jun 15, 2022

Support receptive field search of CNN models. (TPAMI paper rf-next) open-mmlab/mmdetection#8191

Merged

zhouzaida reviewed Jun 15, 2022

View reviewed changes

mmcv/cnn/rfsearch/search.py Outdated Show resolved Hide resolved

jbwang1997 reviewed Jul 1, 2022

View reviewed changes

mmcv/cnn/rfsearch/operator.py Outdated Show resolved Hide resolved

jbwang1997 reviewed Jul 1, 2022

View reviewed changes

mmcv/cnn/rfsearch/operator.py Outdated Show resolved Hide resolved

jbwang1997 reviewed Jul 1, 2022

View reviewed changes

mmcv/cnn/rfsearch/operator.py Outdated Show resolved Hide resolved

jbwang1997 reviewed Jul 1, 2022

View reviewed changes

mmcv/cnn/rfsearch/operator.py Outdated Show resolved Hide resolved

jbwang1997 reviewed Jul 1, 2022

View reviewed changes

mmcv/cnn/rfsearch/utils.py Outdated Show resolved Hide resolved

jbwang1997 reviewed Jul 1, 2022

View reviewed changes

mmcv/cnn/rfsearch/utils.py Outdated Show resolved Hide resolved

jbwang1997 reviewed Jul 1, 2022

View reviewed changes

mmcv/cnn/rfsearch/utils.py Outdated Show resolved Hide resolved

jbwang1997 reviewed Jul 1, 2022

View reviewed changes

mmcv/cnn/rfsearch/search.py Outdated Show resolved Hide resolved

jbwang1997 reviewed Jul 1, 2022

View reviewed changes

mmcv/cnn/rfsearch/search.py Outdated Show resolved Hide resolved

jbwang1997 reviewed Jul 1, 2022

View reviewed changes

mmcv/cnn/rfsearch/search.py Outdated Show resolved Hide resolved

zytx121 reviewed Sep 20, 2022

View reviewed changes

zhouzaida reviewed Oct 26, 2022

View reviewed changes

mmcv/cnn/rfsearch/operator.py Outdated Show resolved Hide resolved

zhouzaida reviewed Oct 26, 2022

View reviewed changes

mmcv/cnn/rfsearch/operator.py Outdated Show resolved Hide resolved

zhouzaida reviewed Oct 26, 2022

View reviewed changes

mmcv/cnn/rfsearch/operator.py Outdated Show resolved Hide resolved

zhouzaida reviewed Oct 26, 2022

View reviewed changes

mmcv/cnn/rfsearch/operator.py Outdated Show resolved Hide resolved

zhouzaida reviewed Oct 26, 2022

View reviewed changes

mmcv/cnn/rfsearch/operator.py Outdated Show resolved Hide resolved

zhouzaida reviewed Oct 26, 2022

View reviewed changes

mmcv/cnn/rfsearch/operator.py Show resolved Hide resolved

zhouzaida reviewed Oct 26, 2022

View reviewed changes

mmcv/cnn/rfsearch/search.py Outdated Show resolved Hide resolved

zhouzaida reviewed Nov 21, 2022

View reviewed changes

mmcv/cnn/__init__.py Outdated Show resolved Hide resolved

zhouzaida reviewed Nov 21, 2022

View reviewed changes

mmcv/cnn/__init__.py Outdated Show resolved Hide resolved

Apply suggestions from code review

860577c

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

zhouzaida approved these changes Nov 22, 2022

View reviewed changes

ZwwWayne requested a review from zytx121 November 22, 2022 05:08

zytx121 reviewed Nov 22, 2022

View reviewed changes

mmcv/cnn/rfsearch/operator.py Outdated Show resolved Hide resolved

zytx121 approved these changes Nov 22, 2022

View reviewed changes

Update mmcv/cnn/rfsearch/operator.py

b6bbcb2

Co-authored-by: Yue Zhou <592267829@qq.com>

zhouzaida reviewed Nov 22, 2022

View reviewed changes

mmcv/cnn/rfsearch/search.py Outdated Show resolved Hide resolved

zhouzaida reviewed Nov 22, 2022

View reviewed changes

mmcv/cnn/rfsearch/operator.py Outdated Show resolved Hide resolved

lzyhha and others added 2 commits November 22, 2022 14:56

change logger

26955e7

Update mmcv/cnn/rfsearch/operator.py

b339444

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

zhouzaida approved these changes Nov 22, 2022

View reviewed changes

zhouzaida merged commit 6e3827d into open-mmlab:master Nov 22, 2022

lzyhha mentioned this pull request Dec 15, 2022

Fix skip_layer for RF-Next #2489

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support receptive field search of CNN models. (TPAMI paper rf-next) #2056

Support receptive field search of CNN models. (TPAMI paper rf-next) #2056

gasvn commented Jun 15, 2022 •

edited

Loading

CLAassistant commented Jun 15, 2022 •

edited

Loading

zhouzaida commented Jun 15, 2022

gasvn commented Jun 16, 2022 •

edited

Loading

gasvn commented Jul 1, 2022

jbwang1997 Jul 1, 2022 •

edited

Loading

gasvn Jul 7, 2022

gasvn commented Aug 29, 2022

zhouzaida commented Aug 29, 2022

gasvn commented Oct 3, 2022

zhouzaida commented Oct 26, 2022

lzyhha commented Nov 21, 2022 •

edited

Loading

lzyhha commented Nov 21, 2022 •

edited

Loading

zhouzaida commented Nov 21, 2022

zhouzaida commented Nov 21, 2022

lzyhha commented Nov 22, 2022

zytx121 left a comment

zhouzaida commented Nov 22, 2022

gasvn commented Nov 22, 2022

lzyhha commented Dec 15, 2022 •

edited

Loading


		class ConvRFSearchOp(nn.Module):

		def __init__(self, op_layer: nn.Module, global_config: Dict = {}):

Support receptive field search of CNN models. (TPAMI paper rf-next) #2056

Support receptive field search of CNN models. (TPAMI paper rf-next) #2056

Conversation

gasvn commented Jun 15, 2022 • edited Loading

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

CLAassistant commented Jun 15, 2022 • edited Loading

zhouzaida commented Jun 15, 2022

gasvn commented Jun 16, 2022 • edited Loading

gasvn commented Jul 1, 2022

jbwang1997 Jul 1, 2022 • edited Loading

Choose a reason for hiding this comment

gasvn Jul 7, 2022

Choose a reason for hiding this comment

gasvn commented Aug 29, 2022

zhouzaida commented Aug 29, 2022

gasvn commented Oct 3, 2022

zhouzaida commented Oct 26, 2022

lzyhha commented Nov 21, 2022 • edited Loading

lzyhha commented Nov 21, 2022 • edited Loading

zhouzaida commented Nov 21, 2022

zhouzaida commented Nov 21, 2022

lzyhha commented Nov 22, 2022

zytx121 left a comment

Choose a reason for hiding this comment

zhouzaida commented Nov 22, 2022

gasvn commented Nov 22, 2022

lzyhha commented Dec 15, 2022 • edited Loading

gasvn commented Jun 15, 2022 •

edited

Loading

CLAassistant commented Jun 15, 2022 •

edited

Loading

gasvn commented Jun 16, 2022 •

edited

Loading

jbwang1997 Jul 1, 2022 •

edited

Loading

lzyhha commented Nov 21, 2022 •

edited

Loading

lzyhha commented Nov 21, 2022 •

edited

Loading

lzyhha commented Dec 15, 2022 •

edited

Loading