[Trainer] Add support for LoRA training

[tyler-griggs]

Add support for LoRA training

[csung7]

@tyler-griggs Great features! When do you think "[Trainer] Add support for LoRA training #110" will be completed and when will we be able to see a script with LoRA included?

[tyler-griggs]

Hi @csung7, sorry I only just saw your comment now! @lynnliu030 has been working on the LoRA feature. I'd expect it to be available within the next week. Once ready, we will definitely include an example script.