Add initial patience & delta mode to early_stopping #14
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(Mostly) addresses #11.
The main changes are:
min_delta
being ignored by theearly_stopping
callback.delta_mode
option to early stopping that allowsmin_delta
to either be interpreted as a"absolute"
or"relative"
difference.initial_patience
option, which is the same aspatience
but only considers the firstn
steps. This is useful when the initial stages of training are very noisy, etc. It is similar tostart_from_epoch
from Keras.checkpoint
callback, for consistency).This doesn't address the problem that the best weights are only returned if the patience has run out. After looking more closely at the code, I think that fixing it requires a more considerable refactoring of the code, so I'll make another issue to discuss. In short, I would like to upgrade the callback functionality such that each callback can have multiple callback functions. E.g., the
early_stopping
callback could have a generic__loop_callback__
that is called on the schedule specified by the user, and aon_epoch_end
that is called at the end of training. This seems to be how the Keras callbacks work, and as far as I understand, it isn't possible with the current setup. But more on that later, let me know what you think of these changes!