Is it possible to generate a config.cfg for a spaCy v2 model?

[CedricMingneau]

Since I've updated from spaCy v2 to v3, I can't load my (self-trained) v2 models anymore because they don't have a config.cfg file. As the CLI is able to convert 'old' traindata from .json to .spacy, I was wondering if there was some tool to automatically infer a config file for a v2 model. After searching for some time, I couldn't find a way to do this.

Is there a way to create a config-file from a v2 model? Is there perhaps another way to load my v2 models using spaCy v3? Or am I forced to use an older version of spaCy for these models?

Thanks in advance!

[ines]

Hi! The problem here goes a bit deeper: the model implementations and various other aspects of the internals have changed between v2 and v3, in order to support more accurate pipelines, shared embeddings (including transformers) and so on. So even if you generated a config file, the weights trained with v2 wouldn't be compatible with the v3 components, so your pipeline wouldn't be able to make meaningful predictions.

So the best solution for your situation would be to retrain your models using spaCy v3. You can use spacy init config to auto-generate a config file for the components you're using, spacy convert to convert your data and spacy train to train your pipeline.

[CedricMingneau]

I already feared it was going to come to that. It's unfortunate, but understandable.
Thanks for the quick response Ines!

[ines]

Yeah, at some point we just need to make breaking changes in order to move forward (and we make sure to only ever do that between minor or major release versions).

That said, forwards-compatibility should be a lot easier in the future with spaCy v3 and the config system with function registries: all architectures and functions are versioned, so if we make a breaking change to an architecture, we can just add a new version of it, and move the old one to spacy-legacy (installed by default with spaCy to provide older functions). This means that if you pipeline was trained with v1 of a given model architecture, you can still load it with that and the weights will be compatible – even if the latest implementation in spaCy is v5 and works differently 🙂

[sudharsan2020]

@ines We have a scenario in which the training data used to build the v2 model is lost. The newer models are running on spacy v3. How can we side-load both 2. x and 3. x models in an application? Is there a workaround for this?

[danieldk]

It's not possible to have two different versions of the same Python package in the same interpreter. So the recommended way to do this is to have spaCy 2 and 3 in different virtual environments/processes.