Fine-Tuned BERT-base on Squad v1.

[Maaarcocr]

I have fine-tuned the TF model on SQuAD v1 and I've made the weights available at: https://s3.eu-west-2.amazonaws.com/nlpfiles/squad_bert_base.tgz

I get 88.5 FM using these weights on SQuAD dev. (If I recall correctly I get roughly 82 EM).

I think it may be beneficial to have these weights here, so that people could play with SQuAD and BERT without the need of fine-tuning, which requires a decent enough setup. Let me know what you think!

[thomwolf]

Thanks for the details.
This PyTorch repo is starting to be used by a larger community so we would have to be a little more precise than just rough numbers if we want to include such pre-trained weights.
If you want to add your weights to the repo, you should convert the weights in the PyTorch repo model and get evaluation results on SQuAD with the PyTorch model so everybody has a clean knowledge of what they are using. Otherwise I think it's better that people do their own training and know what are the capabilities of the fine-tuned model they are using.
Feel free to come back and re-open the issue if this something you would like to do.

[wasiahmad]

@thomwolf On SQuAD v1.1, BERT (single) scored 85.083 EM and 91.835 F1 as reported in their paper but when I fine-tuned BERT using run_squad.py I got {"exact_match": 81.0975, "f1": 88.7005}. Why there is a difference? What I am missing?