Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add progress bars for experiment launch timing #38

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

mitchnegus
Copy link
Member

@mitchnegus mitchnegus commented Jan 6, 2025

This builds off a bunch of previous work, but adds a handful of new features too. The big ideas are:

Experiment Setup Progress Bars

This adds progress bars for graph creation and experiment setup; these use combine progress bars and rich Live displays so that output are presented cleanly (e.g., graph creation progress, VM cache progress, and MC printouts do not clobber each other).

Notably, this MR extends the AbstractPlugin to accept an (optional) console keyword argument. This should have no impact on existing MC plugins that do not display output, but it allows MCs that do display output during setup to leverage this object and avoid competition with any live output coming directly from FIREWHEEL.

Negative Time Progress Bar

The MR also adds a progress bar for experiment negative time to the output of firewheel vm mix. While defining progress through negative time is only particularly well-defined on a per VM basis, it seemed less than ideal to show n progress bars for each of the n VMs in the experiment. Instead, this uses the heuristic that overall progress through negative time is defined as the number of (negative time) schedule events that have been completed per the total number of negative time schedule events across all VMs.

For practicality, "event" completion is defined for each VM as any negative time event that was scheduled before the VMs current time, which means that this heuristic is strictly conservative. Some subset of events in the queue that were all scheduled at the same time may actually have completed, but since the VM time has not advanced, they are not counted as "complete" by this metric.

Noting that schedules can be adapted as the experiment evolves, the total number of negative time events is not guaranteed to remain constant. Rich allows progress bars to update their total values, so this does not cause any trouble in implementation; however; it might surprise users who notice that their totals aren't fixed.

Finally, when the experiment is setting up or in positive time, the negative time progress bar is hidden to avoid visual clutter.

@mitchnegus mitchnegus marked this pull request as draft January 14, 2025 17:56
Copy link
Collaborator

@sdelliot sdelliot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, I like both these ideas. I tested this out and everything seemed to work as expected. However, this is a large PR with two distinct (although related) features. Can you please split them up into their distinct components? I'll go back through and review the code after that has happened.

src/firewheel/control/experiment_graph.py Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants