Why is Pararallel Composition blocked for XLM-Roberta

[jinyongyoo]

Parallel composition of adapters are only supported for a few model types listed here:
https://github.com/Adapter-Hub/adapter-transformers/blob/9a6bf1757b684a4c627c5a35a56e61ea706dccee/src/transformers/adapters/composition.py#L101-L103

But I believe XLM-Roberta has the same architecture as Roberta, so I think parallel composition will also work with XLM-Roberta models. Is it possible to use parallel composition with XLM-Roberta models?

[jinyongyoo]

Nevermind seems like parallel composition does not work with XLM-Roberta models.

Edit

The error I'm getting is RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling cublasCreate(handle)

As far as I can tell, it's occurring at this line and only when I'm using GPU.

https://github.com/Adapter-Hub/adapter-transformers/blob/9a6bf1757b684a4c627c5a35a56e61ea706dccee/src/transformers/models/roberta/modeling_roberta.py#L211

Edit 2
Well turns it was because of GPU OOM issue (strangely didn't raise OOM error directly, however). I now can confirm that parallel composition also works for xlm-roberta.

[jinyongyoo]

Can we add xlm-roberta to the whitelist? It's pretty common to use adapters with multilingual models, so I don't see why we should not allow xlm-roberta to the whitelist.

[calpt]

Hi @jinyongyoo, you're right, I think there's no reason why XLM-Roberta shouldn't be on that list. Adding in #305.