Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EarlyStopper updates #4679

Merged
merged 1 commit into from
Sep 5, 2021
Merged

EarlyStopper updates #4679

merged 1 commit into from
Sep 5, 2021

Conversation

glenn-jocher
Copy link
Member

@glenn-jocher glenn-jocher commented Sep 5, 2021

Updates to EarlyStopping PR #4576 to address concerns in #4653:

  • Default increased from 30 epochs to 100 epochs to minimize impact to existing workflows.
  • Added stopper.possible_stop attribute to indicate possibility of stop next epoch, utilized to create plots and save checkpoints in case epoch is actually final epoch: final_epoch = (epoch + 1 == epochs) or stopper.possible_stop
  • Fixed Multi-GPU bug (EarlyStopping completely disabled in Multi-GPU now, existing TODO inlined in README).

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Improving early stopping logic in YOLOv5 training.

📊 Key Changes

  • Modified the final epoch check to consider a possible stop signal from the EarlyStopping class.
  • Refined the early stopping check to apply only to single-GPU training (RANK == -1 condition).
  • Increased the default early stopping patience from 30 to 100 epochs.
  • Added a possible_stop attribute to the EarlyStopping class for better anticipation of training completion.

🎯 Purpose & Impact

  • 🎯 Purpose: To provide a more robust and clear early stopping mechanism, allowing training to stop earlier when no improvements are observed, and to help users anticipate the end of training.
  • 🎉 Impact: Users benefit from faster training times when no further improvements are made, potentially saving resources. The clarity in training stopping conditions can also help in better managing long training sessions.

@glenn-jocher glenn-jocher self-assigned this Sep 5, 2021
@glenn-jocher glenn-jocher linked an issue Sep 5, 2021 that may be closed by this pull request
@glenn-jocher glenn-jocher merged commit 5487451 into master Sep 5, 2021
@glenn-jocher glenn-jocher deleted the update/early_stopper branch September 5, 2021 17:09
positive666 referenced this pull request in positive666/yolo_research Sep 6, 2021
CesarBazanAV pushed a commit to CesarBazanAV/yolov5 that referenced this pull request Sep 29, 2021
BjarneKuehl pushed a commit to fhkiel-mlaip/yolov5 that referenced this pull request Aug 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

why early stopping occurs, how to cancel early stopping
1 participant