Pretrain large BERT model with RetVec - code and colab tutorial

[MarinaZhang]

No description provided.

[stefan-it]

Hi @MarinaZhang ,

many thanks for releasing the code for the RetVec paper! I would be interesting in pretraining a BERT model with RetVec so I just wanted to ask, if you e.g. have already a demo code and colab tutorial available 🤔

Many thanks!

[MarinaZhang]

Hello! are you interested in pre-training a BERT model on your own dataset or just finetuning a pre-trained one? We don't have a demo colab available yet but hopefully soon, I'll reopen this issue to work on it