Add training.before_update callback

[shadeMe]

Description

This callback can be used to implement training paradigms like gradual (un)freezing of components (e.g: the Transformer) after a certain number of training steps to mitigate catastrophic forgetting during fine-tuning.

Types of change

New feature

Checklist

• I confirm that I have the right to submit this contribution under the project's MIT license.
• I ran the tests, and all new and existing tests passed.
• My changes don't require a change to the documentation, or if they do, I've added all required information.

[danieldk]

Did you find out if there is an easy way to test this?

[shadeMe]

Did you find out if there is an easy way to test this?

Nothing short of bundling a small, toy corpus file and invoking the train and init_nlp functions like I do in the gradual transformer unfreezing PR.

[adrianeboyd]

Can you add some minimal tests for this? It doesn't look like there are any training-specific callback tests yet, but there are similar tests for the [nlp] callbacks.

[adrianeboyd]

I think you should be able to test this with train_while_improving without a separate .spacy file? (I'm pretty sure the tests are going to fail because it's not automatically included in the package, and we'd want to be careful in general about adding a lot of extra data to the test suite.)

[shadeMe]

Ah, didn't realize the extra data wouldn't be included anyway. I'll try out your suggestion.

[adrianeboyd]

It's not that we can't add extra data with some additional settings, and it might make sense to have one toy .spacy file that's shared between different tests.

As I've mentioned a few times, the extra CLI tests in the CI should probably be moved into a proper test suite that's separate from the package tests and we can have the CI run both, and that would be an easier place for longer/larger tests.