[Refact] context rewrite #38

yhna940 · 2022-05-31T04:06:43Z

This is a PR forcing run to work with the context being changed by the context manager.
We plan to work on refactoring the context manager into a decorator type.

github-actions · 2022-05-31T06:48:38Z

Coverage report

The coverage rate is 83.21%

The branch rate is 60.3%

87% of new lines are covered.

Diff Coverage details (click to unfold)

mmtune/mm/tasks/mmdet.py

100.0% of new lines are covered

mmtune/mm/tasks/base.py

72.22222222222223% of new lines are covered

mmtune/utils/config.py

54.54545454545455% of new lines are covered

mmtune/mm/context/rewriters/builder.py

100.0% of new lines are covered

mmtune/mm/context/rewriters/register.py

100.0% of new lines are covered

mmtune/mm/hooks/reporter.py

100.0% of new lines are covered

mmtune/mm/context/rewriters/instantiate.py

100.0% of new lines are covered

mmtune/apis/tune.py

40.0% of new lines are covered

mmtune/mm/tasks/blackbox.py

0.0% of new lines are covered

mmtune/ray/spaces/builder.py

66.66666666666667% of new lines are covered

mmtune/mm/context/rewriters/path.py

100.0% of new lines are covered

mmtune/mm/tasks/builder.py

100.0% of new lines are covered

mmtune/utils/logger.py

66.66666666666667% of new lines are covered

mmtune/mm/tasks/mmcls.py

100.0% of new lines are covered

mmtune/utils/container.py

100.0% of new lines are covered

mmtune/ray/searchers/nevergrad.py

100.0% of new lines are covered

mmtune/mm/context/manager.py

100.0% of new lines are covered

mmtune/mm/context/rewriters/decouple.py

100.0% of new lines are covered

mmtune/mm/hooks/checkpoint.py

100.0% of new lines are covered

mmtune/mm/context/rewriters/base.py

83.33333333333334% of new lines are covered

mmtune/ray/stoppers/early_drop.py

100.0% of new lines are covered

mmtune/ray/spaces/base.py

100.0% of new lines are covered

mmtune/utils/init.py

100.0% of new lines are covered

mmtune/mm/tasks/sphere.py

100.0% of new lines are covered

mmtune/version.py

100.0% of new lines are covered

mmtune/mm/context/rewriters/dump.py

100.0% of new lines are covered

mmtune/mm/context/rewriters/patch.py

100.0% of new lines are covered

mmtune/mm/tasks/mmseg.py

50.0% of new lines are covered

mmtune/mm/context/rewriters/merge.py

100.0% of new lines are covered

mmtune/mm/context/rewriters/init.py

100.0% of new lines are covered

mmtune/mm/tasks/mmtrainbase.py

50.0% of new lines are covered

mmtune/mm/context/rewriters/base.py

mmtune/mm/context/rewriters/suffix.py

yhna940 · 2022-06-02T07:35:38Z

The test coverage increase will be carried out in another pull request.

mmtune/utils/config.py

mmtune/mm/context/rewriters/dump.py

mmtune/mm/context/rewriters/merge.py

Co-authored-by: Hakjin Lee <nijkah@gmail.com>

nijkah · 2022-06-02T08:46:11Z

mmtune/mm/context/rewriters/patch.py

@@ -15,10 +29,27 @@ def unwrap_regexp(value, regexp=WRAPPING_REGEXP):


 @REWRITERS.register_module()
-class BatchConfigPathcer:
+class BatchConfigPathcer(BaseRewriter):


I think its name and docstring can be improved. I cannot get it what it does until now.

Adding an example may help.

nijkah · 2022-06-02T08:48:10Z

mmtune/mm/context/rewriters/patch.py

@@ -33,10 +64,28 @@ def __call__(self, context: dict):


 @REWRITERS.register_module()
-class SequeunceConfigPathcer:
+class SequeunceConfigPathcer(BaseRewriter):


Same as BatchConfigPatcher,
it seems important to its main function is to interpret - in the configuration file.

nijkah · 2022-06-02T09:01:23Z

mmtune/mm/context/rewriters/path.py

+
+
+@REWRITERS.register_module()
+class PathJoinTrialId(BaseRewriter):


How about AppendTrialIDtoPath?

nijkah · 2022-06-02T09:07:14Z

mmtune/mm/hooks/reporter.py

 from mmcv.runner.dist_utils import get_dist_info
 from mmcv.runner.hooks.logger import LoggerHook
 from torch import distributed as dist


 @HOOKS.register_module()
 class RayTuneLoggerHook(LoggerHook):
+    """Logger hook for Ray Tune."""


Suggested change

"""Logger hook for Ray Tune."""

"""MMCV Logger hook for Ray Tune."""

nijkah · 2022-06-02T09:10:31Z

mmtune/mm/tasks/base.py



-@TASKS.register_module()
 class BaseTask(metaclass=ABCMeta):
    """Wrap the apis of target task."""


The docstring seems inappropriate.
"""Base class to specify the target task."""

nijkah · 2022-06-02T09:11:00Z

mmtune/mm/tasks/base.py


    def set_args(self, args: Sequence[str]) -> None:
-        self.args = self.parse_args(args)
+        """Parse and Set the args.


Suggested change

"""Parse and Set the args.

"""Parse and set the args.

nijkah · 2022-06-02T09:11:26Z

mmtune/mm/tasks/base.py

-        context_manager = ContextManager(**status)
-        return context_manager(self.run)(*args, **kwargs)
+    def context_aware_run(self, searched_cfg: Dict, **kwargs) -> Any:
+        """Gathers and refines the information received by users and Raytune to


Suggested change

"""Gathers and refines the information received by users and Raytune to

"""Gather and refine the information received by users and Ray.tune to

nijkah · 2022-06-02T09:12:21Z

mmtune/mm/tasks/base.py


    @abstractmethod
-    def run(self, *args, **kwargs) -> None:
+    def run(self, *, args: argparse.Namespace, **kwargs) -> None:
+        """The objective task.


Description seems inappropriate.

Just Run the target task.

nijkah · 2022-06-02T09:33:23Z

mmtune/mm/tasks/mmtrainbase.py

+                          searched_cfg,
+                          backend='nccl',
+                          **kwargs) -> None:
+        """Gathers and refines the information received by users and Raytune to


Suggested change

"""Gathers and refines the information received by users and Raytune to

"""Gather and refine the information received by users and Ray.tune to

nijkah

Looks Great to Me.

yhna940 added 8 commits May 30, 2022 18:04

Tmp

51f9538

Add test code

e047af6

Fix typo

6dac479

Fix typo

af857b0

Fix minor bug

13f27a5

Fix decouple

88f476d

Fix additional args

9773601

Fix cfg

047a5ce

yhna940 changed the title ~~Feature/refact context~~ [Refact] context rewrite May 31, 2022

yhna940 added 6 commits May 31, 2022 13:11

Fix red var

048aef2

Fix test code

32fd4f2

Fix typo

ef284bd

Fix rewriter

3071822

Fix test code

0e26edd

Fix test code

ce745e4

yhna940 requested review from nijkah and KKIEEK May 31, 2022 06:49

nijkah reviewed May 31, 2022

View reviewed changes

mmtune/mm/context/rewriters/base.py Outdated Show resolved Hide resolved

nijkah reviewed May 31, 2022

View reviewed changes

mmtune/mm/context/rewriters/suffix.py Outdated Show resolved Hide resolved

yhna940 added 4 commits May 31, 2022 16:49

Add docstring

125db96

Fix docstring

fc0713d

Add docstring

fed7700

Fix docstring

08a79f6

yhna940 marked this pull request as draft June 1, 2022 04:31

Add docstring

2282e36

yhna940 marked this pull request as ready for review June 2, 2022 07:32

yhna940 requested a review from nijkah June 2, 2022 07:32

KKIEEK approved these changes Jun 2, 2022

View reviewed changes

nijkah reviewed Jun 2, 2022

View reviewed changes

mmtune/utils/config.py Outdated Show resolved Hide resolved

nijkah reviewed Jun 2, 2022

View reviewed changes

mmtune/mm/context/rewriters/dump.py Outdated Show resolved Hide resolved

nijkah reviewed Jun 2, 2022

View reviewed changes

mmtune/mm/context/rewriters/merge.py Outdated Show resolved Hide resolved

yhna940 and others added 4 commits June 2, 2022 17:41

Log dump failed

94fe9cd

Update mmtune/apis/tune.py

9507f5d

Co-authored-by: Hakjin Lee <nijkah@gmail.com>

Update mmtune/mm/context/manager.py

728ac38

Co-authored-by: Hakjin Lee <nijkah@gmail.com>

Update mmtune/mm/context/manager.py

ac692b3

Co-authored-by: Hakjin Lee <nijkah@gmail.com>

yhna940 force-pushed the feature/refact-context branch from 0a76912 to ac692b3 Compare June 2, 2022 08:42

yhna940 and others added 2 commits June 2, 2022 17:43

Update mmtune/mm/context/rewriters/dump.py

a673a8e

Co-authored-by: Hakjin Lee <nijkah@gmail.com>

Update mmtune/mm/context/rewriters/merge.py

b896b70

Co-authored-by: Hakjin Lee <nijkah@gmail.com>

nijkah reviewed Jun 2, 2022

View reviewed changes

Add ex

571b28f

nijkah reviewed Jun 2, 2022

View reviewed changes

yhna940 added 4 commits June 3, 2022 09:56

Fix Docstring

1673a75

Fix Docstring

8399eab

Fix Docstring

88c1829

Fix docstring

fb522c3

yhna940 requested review from nijkah and KKIEEK June 3, 2022 01:26

nijkah approved these changes Jun 3, 2022

View reviewed changes

yhna940 merged commit 2462401 into master Jun 3, 2022

nijkah deleted the feature/refact-context branch September 8, 2022 06:41



		@REWRITERS.register_module()
		class PathJoinTrialId(BaseRewriter):

	"""Logger hook for Ray Tune."""
	"""MMCV Logger hook for Ray Tune."""

	"""Gathers and refines the information received by users and Raytune to
	"""Gather and refine the information received by users and Ray.tune to

[Refact] context rewrite #38

[Refact] context rewrite #38

Conversation

yhna940 commented May 31, 2022 • edited Loading

github-actions bot commented May 31, 2022 • edited Loading

Coverage report

mmtune/mm/tasks/mmdet.py

mmtune/mm/tasks/base.py

mmtune/utils/config.py

mmtune/mm/context/rewriters/builder.py

mmtune/mm/context/rewriters/register.py

mmtune/mm/hooks/reporter.py

mmtune/mm/context/rewriters/instantiate.py

mmtune/apis/tune.py

mmtune/mm/tasks/blackbox.py

mmtune/ray/spaces/builder.py

mmtune/mm/context/rewriters/path.py

mmtune/mm/tasks/builder.py

mmtune/utils/logger.py

mmtune/mm/tasks/mmcls.py

mmtune/utils/container.py

mmtune/ray/searchers/nevergrad.py

mmtune/mm/context/manager.py

mmtune/mm/context/rewriters/decouple.py

mmtune/mm/hooks/checkpoint.py

mmtune/mm/context/rewriters/base.py

mmtune/ray/stoppers/early_drop.py

mmtune/ray/spaces/base.py

mmtune/utils/init.py

mmtune/mm/tasks/sphere.py

mmtune/version.py

mmtune/mm/context/rewriters/dump.py

mmtune/mm/context/rewriters/patch.py

mmtune/mm/tasks/mmseg.py

mmtune/mm/context/rewriters/merge.py

mmtune/mm/context/rewriters/init.py

mmtune/mm/tasks/mmtrainbase.py

yhna940 commented Jun 2, 2022

nijkah Jun 2, 2022

Choose a reason for hiding this comment

nijkah Jun 2, 2022

Choose a reason for hiding this comment

nijkah Jun 2, 2022 • edited Loading

Choose a reason for hiding this comment

nijkah Jun 2, 2022 • edited Loading

Choose a reason for hiding this comment

nijkah Jun 2, 2022 • edited Loading

Choose a reason for hiding this comment

nijkah Jun 2, 2022 • edited Loading

Choose a reason for hiding this comment

nijkah Jun 2, 2022

Choose a reason for hiding this comment

nijkah Jun 2, 2022 • edited Loading

Choose a reason for hiding this comment

nijkah Jun 2, 2022

Choose a reason for hiding this comment

nijkah Jun 2, 2022

Choose a reason for hiding this comment

nijkah left a comment

Choose a reason for hiding this comment

yhna940 commented May 31, 2022 •

edited

Loading

github-actions bot commented May 31, 2022 •

edited

Loading

nijkah Jun 2, 2022 •

edited

Loading

nijkah Jun 2, 2022 •

edited

Loading

nijkah Jun 2, 2022 •

edited

Loading

nijkah Jun 2, 2022 •

edited

Loading

nijkah Jun 2, 2022 •

edited

Loading