Support pretrained word2vec model when train doc2vec #2703
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,177 +1,61 @@ | ||
| doc2vec in gensim – support pretrained word2vec | ||
| ================================== | ||
|
|
||
| This is a forked gensim version, which edits the default doc2vec model to support pretrained word2vec during training doc2vec. It forked from gensim 3.8. | ||
|
There was a problem hiding this comment. Write your documentation so that it's useful from the point of view of the reader. "This is a forked gensim version" is not relevant to the user. Furthermore, it becomes misleading the moment we actually merge this PR. |
||
|
|
||
| The default doc2vec model in gensim does't support pretrained word2vec model. But according to Jey Han Lau and Timothy Baldwin's paper, [An Empirical Evaluation of doc2vec with Practical Insights into Document Embedding Generation(2016)](https://arxiv.org/abs/1607.05368), using pretrained word2vec model usually gets better results in NLP tasks. The author also released a [forked gensim verstion](https://github.com/jhlau/gensim) to perform pretrained embeddings, but it is from a very old gensim version, which can't be used in gensim 3.8(the latest gensim version when I release this fork). | ||
|
There was a problem hiding this comment. This is also irrelevant. This kind of information is good inside the PR, as motivation and background (it may already be there). |
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
| Features and notice | ||
| ============= | ||
| * 1.Support pretrained word2vec when train doc2vec. | ||
| * 2.Support Python 3. | ||
| * 3.Support gensim 3.8. | ||
| * 4.The pretrainned word2vec model should be C text format. | ||
| * 5.The dimension of the pretrained word2vec and the to be trained doc2vec should be the same. | ||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
| Use the model | ||
| ============= | ||
|
|
||
| 1.Install the forked gensim | ||
| --------------------------- | ||
|
|
||
| * Clone gensim to your machine | ||
| > git clone https://github.com/maohbao/gensim.git | ||
|
|
||
| * install gensim | ||
| > python setup.py install | ||
|
|
||
|
|
||
| 2.Train your doc2vec model | ||
| --------------------------- | ||
|
|
||
| > pretrained_emb = "word2vec_pretrained.txt" # This is a pretrained word2vec model of C text format | ||
| > | ||
| > model = gensim.models.doc2vec.Doc2Vec( | ||
| corpus_train, # This is the documents corpus to be trained which should meet gensim's format | ||
|
|
||
| vector_size=300, | ||
| min_count=1, epochs=20, | ||
| dm=0, | ||
| pretrained_emb=pretrained_emb) | ||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
| Publications | ||
| ============= | ||
|
|
||
| * 1.Jey Han Lau and Timothy Baldwin. [An Empirical Evaluation of doc2vec with Practical Insights into Document Embedding Generation. In Proceedings of the 1st Workshop on Representation Learning for NLP, 2016.](https://arxiv.org/abs/1607.05368) | ||
|
|
||
| * 2.[The initial forked gensim version](https://github.com/jhlau/gensim) | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think instead of replacing the top-level README.md file, you should put this documentation somewhere else. Ideally, it should be in a tutorial or a howto.
See https://radimrehurek.com/gensim/auto_examples/howtos/run_doc.html#sphx-glr-auto-examples-howtos-run-doc-py for more info.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is my first time to do PR on github, I already followed your advice on word2vec.py and doc2vec.py, I also know that I should not change the top README.md file, but I really don't how and where to write the document, thank you for more advice!