Documentation on jointly learning feature representations with a higher task

[LukeB42]

I can't find any documentation on training, say, FastText jointly with a classification task.

The callbacks parameter is mentioned in the API documentation but not explained.

[menshikh-iv]

Hi @LukeB42, we working on documentation now

The callbacks parameter is mentioned in the API documentation but not explained.

will be fixed in #1944 soon

I can't find any documentation on training, say, FastText jointly with a classification task.

gensim is about unsupervised learning, you can use inferred vectors in any way (including for classification). Also, we have in plan to make "problem oriented" tutorial chain (including popular task like classification, clustering, etc).

[LukeB42]

@menshikh-iv Thanks for explaining that and for hipping me to #1944.

For the problem-oriented tutorial chain do you plan on explaining that accuracy improvements might be obtained by regularising the unsupervised representation via the supervised task?

It'd be a good way to showcase the callbacks API, IMHO.

[menshikh-iv]

It'd be a good way to showcase the callbacks API, IMHO.

We already have an example from #1944, if needed, we'll show it in tutorials again

For the problem-oriented tutorial chain do you plan on explaining that accuracy improvements might be obtained by regularising the unsupervised representation via the supervised task?

Hm, not sure what you exactly mean here

[LukeB42]

Looking forward to the documentation update but from what I can see in that PR there doesn't seem to be much of anything on the callbacks API.

I'm trying to regularise the word embeddings via a classification task.

Is there an example of backpropagating through a classification task while fitting the FastText model?

[menshikh-iv]

@LukeB42 oh sorry, my mistake, this fixed in different PR https://github.com/RaRe-Technologies/gensim/pull/2026/files#diff-f3f747c490afb30c4f40c2804e459b63R1

Is there an example of backpropagating through a classification task while fitting the FastText model?

No, this is the really specific case (of course useful, but I don't think that we'll add it).

[LukeB42]

Ah no worries. Again many thanks @menshikh-iv!