Adapter Dropout

[JensVN98]

Hi!

Is there a reason as to why the dropout rate cannot be defined in certain configurations? For example, the dropout rate can be defined in LoraConfig, but not in the more "standard" configurations (AutoConfig, Pfeiffer, Houlsby...). I realise that LoRA and PrefixTuning differ from a methodological standpoint from the Houslby and Pfeiffer configs, but it would still be interesting to be able to add some sort of additional regularisation.

Thanks!

[calpt]

Hi @JensVN98, I don't believe there's a specific reason for this other than that the original implementations didn't suggest adding a dropout rate. This shouldn't be difficult to add to the other configurations though.
Leaving this open as a feature request so we can add it in a coming update.