Support for DeBERTa V2

[ishabi]

Question
Will DeGERTa V2 be supported by FARM?

Additional context
I was experimenting DeBERTa language model to train NER model, but it always fails. Will it be supported in future?

Thanks

[Timoeller]

Hey @ishabi there is currently no support for DeBERTa in FARM.

Would you like to contribute this?
It should not be too complex since you can use the underlying transformers implementation, you just have to create a wrapper in our farm.modelling.language_model script and write some test.

Any help would be appreciated.

[Timoeller]

Hey @ishabi it seems adding DeBERTa v1 and v2 will be more difficult to implement because we currently only support fast tokenizers.

A fast tokenizer for DeBERTa v1 was merged 5 days ago (huggingface/transformers#11387). But there’s still no fast tokenizer for DeBERTa v2 (huggingface/transformers#11529). I think we might also have to wait until these are in the next release.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 21 days if no further activity occurs.