The loss of MLM training

[AcetoneChen]

I tried to pretrain the model with provided data, but the loss did not descend substantially (1.3 to 1.2 after 200 epoches)
Then I tried to use the pretrained model provided to evaluate on the provided data
(./Model/Pretrained/ESM2SI_3.1_fiveSpeciesCao_6layers_16heads_128embedsize_4096batchToks_MLMLossMin.pkl), but the pretrained model still have a loss of 1.22 on the dataset.
So the magnitude of loss should be that large?