Pretraining accuracy for retromae v1

[wwx13]

Great job!
Hello , i wonder if you can tell me the training mlm accuracy of encoder and decoder. Im training my retromae model now.

[staoxiao]

Hi, thanks for your interest in our work!
Actually, we didn't test the mlm accuracy of retromae on any data. We view the retrieval performance after fine-tuning as the quality of pre-trained model.

[soledad921]

Hi, I am also quite interested in this work. Could you tell me how much the loss function of the model decreased when you were training the model on Wikipedia? I am trying to pretrain a BERT encoder using retromae v1 on my local dataset (a document pool extracted from Wikipedia). But I have no idea if my model is fully trained?