extend to other pre-trained LMs

[lytum]

Hi,

thanks for sharing the excellent work. May I ask if this knowledge-graph based model can be extended into other pre-trained models, like RoBERTa or GPT2? My understanding is Yes. Could you give me more explanation about that? Thanks a lot!!

[autoliuweijie]

Hi,

thanks for sharing the excellent work. May I ask if this knowledge-graph based model can be extended into other pre-trained models, like RoBERTa or GPT2? My understanding is Yes. Could you give me more explanation about that? Thanks a lot!!

Yes, the model structures of RobERTa and BERT are the same.