Add TextCatReduce.v1 #13181

danieldk · 2023-12-05T12:52:14Z

Description

This is a textcat classifier that pools the vectors generated by a tok2vec implementation and then applies a classifier to the pooled representation. Three reductions are supported for pooling: first, max, and mean. When multiple reductions are enabled, the reductions are concatenated before providing them to the classification layer.

This model is a generalization of the TextCatCNN model, which only supports mean reductions and is a bit of a misnomer, because it can also be used with transformers. This change also reimplements TextCatCNN.v2 using the new TextCatReduce.v1 layer.

One thing I wasn't certain of is what to do with the TextCatCNN.v2 docs. We could move them to legacy, however, this would mean that the existing TextCatCNN does not work anymore and there may be existing pages and tutorials linking to that.

Types of change

Feature

Checklist

I confirm that I have the right to submit this contribution under the project's MIT license.
I ran the tests, and all new and existing tests passed.
My changes don't require a change to the documentation, or if they do, I've added all required information.

This is a textcat classifier that pools the vectors generated by a tok2vec implementation and then applies a classifier to the pooled representation. Three reductions are supported for pooling: first, max, and mean. When multiple reductions are enabled, the reductions are concatenated before providing them to the classification layer. This model is a generalization of the TextCatCNN model, which only supports mean reductions and is a bit of a misnomer, because it can also be used with transformers. This change also reimplements TextCatCNN.v2 using the new TextCatReduce.v1 layer.

svlandeg

Nice!

One thing I wasn't certain of is what to do with the TextCatCNN.v2 docs. We could move them to legacy, however, this would mean that the existing TextCatCNN does not work anymore and there may be existing pages and tutorials linking to that.

The docs can be moved to legacy, and the code can go into spacy-legacy, but all existing code should ideally just continue to work as-is. Why wouldn't/couldn't it?

https://spacy.io/api/architectures#TextCatCNN could just refer to the legacy docs and/or the new TextCatReduce.

spacy/ml/models/textcat.py

spacy/tests/pipeline/test_textcat.py

website/docs/api/architectures.mdx

danieldk · 2023-12-07T14:41:22Z

Nice!

One thing I wasn't certain of is what to do with the TextCatCNN.v2 docs. We could move them to legacy, however, this would mean that the existing TextCatCNN does not work anymore and there may be existing pages and tutorials linking to that.

The docs can be moved to legacy, and the code can go into spacy-legacy, but all existing code should ideally just continue to work as-is. Why wouldn't/couldn't it?

I just noticed that there is a word missing in my initial comment:

One thing I wasn't certain of is what to do with the TextCatCNN.v2 docs. We could move them to legacy, however, this would mean that the existing TextCatCNN (documentation) anchor does not work anymore and there may be existing pages and tutorials linking to that.

Currently, pages can link to https://spacy.io/api/architectures#TextCatCNN

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

svlandeg · 2023-12-07T15:09:23Z

Currently, pages can link to https://spacy.io/api/architectures#TextCatCNN

Right - yes it dawned on me what you meant after writing the comment. We could just have a short dummy section, forwarding to the correct place.

adrianeboyd · 2023-12-08T06:38:35Z

One thing I wasn't certain of is what to do with the TextCatCNN.v2 docs. We could move them to legacy, however, this would mean that the existing TextCatCNN does not work anymore and there may be existing pages and tutorials linking to that.

I think you can add a redirect to netlify.toml to handle that kind of change.

danieldk · 2023-12-08T10:06:22Z

One thing I wasn't certain of is what to do with the TextCatCNN.v2 docs. We could move them to legacy, however, this would mean that the existing TextCatCNN does not work anymore and there may be existing pages and tutorials linking to that.

I think you can add a redirect to netlify.toml to handle that kind of change.

Cool, works with anchors too 🙏 .

danieldk · 2023-12-08T10:11:32Z

One thing I wasn't certain of is what to do with the TextCatCNN.v2 docs. We could move them to legacy, however, this would mean that the existing TextCatCNN does not work anymore and there may be existing pages and tutorials linking to that.

I think you can add a redirect to netlify.toml to handle that kind of change.

Cool, works with anchors too 🙏 .

Or at least I got that impression from the existing rule files. But it doesn't seem to actually work, e.g. the existing rule for

https://spacy.io/api#cython

does not seem to cause a redirection to

https://spacy.io/api/cython

?

…chor

netlify.toml

spacy/cli/templates/quickstart_training.jinja

spacy/errors.py

spacy/ml/models/textcat.py

spacy/tests/test_models.py

* Add TextCatReduce.v1 This is a textcat classifier that pools the vectors generated by a tok2vec implementation and then applies a classifier to the pooled representation. Three reductions are supported for pooling: first, max, and mean. When multiple reductions are enabled, the reductions are concatenated before providing them to the classification layer. This model is a generalization of the TextCatCNN model, which only supports mean reductions and is a bit of a misnomer, because it can also be used with transformers. This change also reimplements TextCatCNN.v2 using the new TextCatReduce.v1 layer. * Doc fixes Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com> * Fully specify `TextCatCNN` <-> `TextCatReduce` equivalence * Move TextCatCNN docs to legacy, in prep for moving to spacy-legacy * Add back a test for TextCatCNN.v2 * Replace TextCatCNN in pipe configurations and templates * Add an infobox to the `TextCatReduce` section with an `TextCatCNN` anchor * Add last reduction (`use_reduce_last`) * Remove non-working TextCatCNN Netlify redirect * Revert layer changes for the quickstart * Revert one more quickstart change * Remove unused import * Fix docstring * Fix setting name in error message --------- Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com> Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>

danieldk added enhancement Feature requests and improvements feat / textcat Feature: Text Classifier labels Dec 5, 2023

danieldk force-pushed the features/textcat-reduce branch from 79eeba7 to 6147103 Compare December 6, 2023 09:38

svlandeg reviewed Dec 7, 2023

View reviewed changes

Doc fixes

ab97add

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

Fully specify TextCatCNN <-> TextCatReduce equivalence

71aa6f4

danieldk added 3 commits December 8, 2023 10:38

Move TextCatCNN docs to legacy, in prep for moving to spacy-legacy

9fb573e

Add back a test for TextCatCNN.v2

5fa33ce

Replace TextCatCNN in pipe configurations and templates

09800bd

danieldk added 2 commits December 8, 2023 12:03

Add an infobox to the TextCatReduce section with an TextCatCNN an…

e310ddf

…chor

Add last reduction (use_reduce_last)

7669a45

adrianeboyd reviewed Dec 11, 2023

View reviewed changes

netlify.toml Outdated Show resolved Hide resolved

Remove non-working TextCatCNN Netlify redirect

a53c482

svlandeg reviewed Dec 11, 2023

View reviewed changes

spacy/cli/templates/quickstart_training.jinja Outdated Show resolved Hide resolved

danieldk added 2 commits December 11, 2023 16:56

Revert layer changes for the quickstart

87816d1

Revert one more quickstart change

65a1dd4

adrianeboyd reviewed Dec 20, 2023

View reviewed changes

spacy/errors.py Outdated Show resolved Hide resolved

adrianeboyd reviewed Dec 20, 2023

View reviewed changes

spacy/ml/models/textcat.py Outdated Show resolved Hide resolved

adrianeboyd reviewed Dec 20, 2023

View reviewed changes

spacy/tests/test_models.py Outdated Show resolved Hide resolved

adrianeboyd added 3 commits December 20, 2023 17:21

Remove unused import

684bd5e

Fix docstring

3d86539

Fix setting name in error message

288f623

adrianeboyd merged commit 7ebba86 into explosion:master Dec 21, 2023
13 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add TextCatReduce.v1 #13181

Add TextCatReduce.v1 #13181

danieldk commented Dec 5, 2023

svlandeg left a comment •

edited

Loading

danieldk commented Dec 7, 2023

svlandeg commented Dec 7, 2023

adrianeboyd commented Dec 8, 2023

danieldk commented Dec 8, 2023

danieldk commented Dec 8, 2023

Add TextCatReduce.v1 #13181

Add TextCatReduce.v1 #13181

Conversation

danieldk commented Dec 5, 2023

Description

Types of change

Checklist

svlandeg left a comment • edited Loading

Choose a reason for hiding this comment

danieldk commented Dec 7, 2023

svlandeg commented Dec 7, 2023

adrianeboyd commented Dec 8, 2023

danieldk commented Dec 8, 2023

danieldk commented Dec 8, 2023

svlandeg left a comment •

edited

Loading