Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

on_train_end callback is not called if training is stopped early #657

Closed
neggert opened this issue Jan 2, 2020 · 1 comment
Closed

on_train_end callback is not called if training is stopped early #657

neggert opened this issue Jan 2, 2020 · 1 comment
Labels
bug Something isn't working

Comments

@neggert
Copy link
Contributor

neggert commented Jan 2, 2020

            # early stopping
            met_min_epochs = epoch >= self.min_epochs - 1
            if self.enable_early_stop and (met_min_epochs or self.fast_dev_run):
                should_stop = self.early_stop_callback.on_epoch_end(epoch=epoch,
                                                                    logs=self.callback_metrics)
                # stop training
                stop = should_stop and met_min_epochs
                if stop:
                    self.main_progress_bar.close()
                    return

        self.main_progress_bar.close()

        model.on_train_end()
@neggert neggert added the bug Something isn't working label Jan 2, 2020
@williamFalcon
Copy link
Contributor

@neggert good catch. PR?

HarshSharma12 added a commit to HarshSharma12/pytorch-lightning that referenced this issue Jan 21, 2020
HarshSharma12 added a commit to HarshSharma12/pytorch-lightning that referenced this issue Jan 21, 2020
HarshSharma12 added a commit to HarshSharma12/pytorch-lightning that referenced this issue Jan 21, 2020
ensures on train end is called even if early stopping is applied
HarshSharma12 added a commit to HarshSharma12/pytorch-lightning that referenced this issue Jan 21, 2020
HarshSharma12 added a commit to HarshSharma12/pytorch-lightning that referenced this issue Jan 21, 2020
HarshSharma12 added a commit to HarshSharma12/pytorch-lightning that referenced this issue Jan 21, 2020
ensures on train end is called even if early stopping is applied
HarshSharma12 added a commit to HarshSharma12/pytorch-lightning that referenced this issue Jan 21, 2020
HarshSharma12 added a commit to HarshSharma12/pytorch-lightning that referenced this issue Jan 22, 2020
HarshSharma12 added a commit to HarshSharma12/pytorch-lightning that referenced this issue Jan 22, 2020
ensures on train end is called even if early stopping is applied
HarshSharma12 added a commit to HarshSharma12/pytorch-lightning that referenced this issue Jan 22, 2020
@Borda Borda mentioned this issue Jan 23, 2020
4 tasks
HarshSharma12 added a commit to HarshSharma12/pytorch-lightning that referenced this issue Jan 23, 2020
ensures on train end is called even if early stopping is applied
HarshSharma12 added a commit to HarshSharma12/pytorch-lightning that referenced this issue Jan 23, 2020
@Borda Borda closed this as completed Feb 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
3 participants