Tracker rewrite and lazy process checker #1079
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
|
Post approval I'll ping the folks at W&B to ensure they agree and like the proposed API |
|
As discussed offline: I put back in |
src/accelerate/accelerator.py
Outdated
| if len(getattr(self, "trackers", [])) > 0: | ||
| for tracker in self.trackers: | ||
| if tracker.name == name: | ||
| return tracker.tracker | ||
| raise ValueError(f"{name} is not an available tracker stored inside the `Accelerator`.") | ||
| # Handle tracker only made on main process | ||
| return GeneralTracker(_blank=True) |
There was a problem hiding this comment.
Needed so that doing get_tracker doesn't return None if accelerator.init_trackers was used and the user wants to call get_tracker and not have to account for is_main_process when using the returned tracker. GeneralTracker now uses a pass so that the methods are silently called and noop
|
@pacman100 sure! Overall for a "general use" there isn't much impact. But for users that want to use the trackers outside the accelerator object they can now do: accelerator = Accelerator(log_with="wandb")
tracker = accelerator.get_tracker("wandb")
- with accelerator.on_main_process():
- tracker.log(...)
+ tracker.log(...) |
Co-authored-by: Sourab Mangrulkar <13534540+pacman100@users.noreply.github.com>
This PR rewrites the tracker classes to be a regular class with init checks rather than an abstract class, letting them be flexible enough but also verifying the information that must be there is included.
This PR also makes the process execution for trackers happen at the tracker level, rather than at the
Acceleratorlevel, so that trackers can be utilized on their own. To do so, a newon_main_processdecorator was added in thetracking.pyspecifically which checks if the tracker class has a value foron_main_processtoTrueorFalse. Depending on this the logging function (and/or init) will then execute. This decorator is executed lazily, so that thePartialStatewill have been created not at import time.