-
-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Change GPU efficient textcat to use CNN, not BOW in generated configs #11900
Change GPU efficient textcat to use CNN, not BOW in generated configs #11900
Conversation
If you generate a config with a textcat component using GPU (transformers), the defaut option (efficiency) uses a BOW architecture, which does not use tok2vec features. While that can make sense as part of a larger pipeline, in the case of just a transformer and a textcat, that means the transformer is doing a lot of work for no purpose. This changes it so that the CNN architecture is used instead. It could also be changed to be the same as the accuracy config, which uses the ensemble architecture.
This doesn't add the
|
Thanks for pointing that out, I fixed it. Going to mark this as a draft while the issue with #11925 is investigated, since it looks like affects the CNN textcat with transformers. |
* Switch ubuntu-latest to ubuntu-20.04 in main tests * Only use 20.04 for 3.6
Brought this out of draft since #11968 is resolved, though this probably shouldn't be merged until it's released. |
thinc v8.1.8 hasn't been released yet, so this should fail for now. |
This test looks good: explosion/spacy-transformers#357 So this change can be merged for v3.5.1. |
Putting this back in draft because it appears to break the training quickstart in the deploy preview. |
Locally it's not completely broken, but it doesn't appear to do what it's intended to do. This may take a bit longer to debug because of the buggy/frustrating jinja2js conversion. |
I think the older website preview in the PR had gotten too old or out-of-sync with more recent website changes, and regenerated with the minor change above this now looks like it's working as intended to me. Locally the issue was that |
…explosion#11900) * Change GPU efficient textcat to use CNN, not BOW If you generate a config with a textcat component using GPU (transformers), the defaut option (efficiency) uses a BOW architecture, which does not use tok2vec features. While that can make sense as part of a larger pipeline, in the case of just a transformer and a textcat, that means the transformer is doing a lot of work for no purpose. This changes it so that the CNN architecture is used instead. It could also be changed to be the same as the accuracy config, which uses the ensemble architecture. * Add the transformer when using a textcat with GPU * Switch ubuntu-latest to ubuntu-20.04 in main tests (explosion#11928) * Switch ubuntu-latest to ubuntu-20.04 in main tests * Only use 20.04 for 3.6 * Require thinc v8.1.7 * Require thinc v8.1.8 * Break up longer expression --------- Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
Description
If you generate a config with a textcat component using GPU (transformers), the defaut option (efficiency) uses a BOW architecture, which does not use tok2vec features. While that can make sense as part of a larger pipeline, in the case of just a transformer and a textcat, that means the transformer is doing a lot of work for no purpose.
This changes it so that the CNN architecture is used instead. It could also be changed to be the same as the accuracy config, which uses the ensemble architecture.
I haven't tested this in the browser yet because I'm having difficulty with the site's build env, but it works fine in the CLI, and imagine it's fine - there are no actual logic changes.
Types of change
Checklist