【开源实习】 Albert 模型微调

[4everWZ]

Albert mindnlp 微调

• Albert模型微调任务链接：【开源实习】albert模型微调 · Issue #IAUONP · MindSpore/community - Gitee.com
• 实现了Albert-base-v1 基准权重 在 [Sentiment analysis of IMDb reviews - Stanford University] 数据集上的微调
• base model: albert/albert-base-v1 · Hugging Face
• dataset: stanfordnlp/imdb · Datasets at Hugging Face

Result for finetune

training for 3 epochs

torch

Epoch	eval_loss
1	0.3868
2	0.2978
3	0.3293
Evaluation results	0.2978

评估结果

Accuracy	Precision	Recall	F1_score
0.9212	0.9218	0.9284	0.9218

mindspore

Epoch	eval_loss
1	0.2677
2	0.2314
3	0.2332
Evaluation results	0.2314

评估结果

Accuracy	Precision	Recall	F1_score
0.9219	0.9238	0.9218	0.9228