Bert_XJY_ZJUT

[JingyangXiang]

将Mindspore中的Bert迁移到tinyms，详情请见bert.md

[leonwanghui]

@zjuter0126 Did u use GPU backend to train Bert network?

[JingyangXiang]

@zjuter0126 Did u use GPU backend to train Bert network?

Yes,on TeslaV100-SXM2 32GB.According to mindspore,the zhwiki is used to pretrain the bert.

[leonwanghui]

@zjuter0126 Got it, I will take a test locally later, thanks for pulling this request. And if it's possible, could u try to insert the code into the tinyms framework? Thanks!

[JingyangXiang]

@zjuter0126 Got it, I will take a test locally later, thanks for pulling this request. And if it's possible, could u try to insert the code into the tinyms framework? Thanks!

I will try my best.However,I'am not sure I can insert them in tinyms beautifully.I may need to trouble you to fine-tune it later

[leonwanghui]

@zjuter0126 Got it, I will take a test locally later, thanks for pulling this request. And if it's possible, could u try to insert the code into the tinyms framework? Thanks!

I will try my best.However,I'am not sure I can insert them in tinyms beautifully.I may need to trouble you to fine-tune it later

Don't have to worry about it, there is no trouble at all : )

[JingyangXiang]

@zjuter0126 Got it, I will take a test locally later, thanks for pulling this request. And if it's possible, could u try to insert the code into the tinyms framework? Thanks!

I will try my best.However,I'am not sure I can insert them in tinyms beautifully.I may need to trouble you to fine-tune it later

Don't have to worry about it, there is no trouble at all : )

I delete some unnecessary *.py and carefully put other into tinyms frames.It may be not beautiful.You can see them in new branch.Hope this satisfies you.The usage of code is same to the former.Just please bert_for_pre_pretraining.py/config.py/utils.py in the same files. And SentimentNet's warning comes from mindspore=1.1.1.

[leonwanghui]

LGTM

[leonwanghui]

LGTM

[leonwanghui]

Some review comments.

[leonwanghui]

LGTM

[leonwanghui]

@zjuter0126 Congras!