LoRA loading and LoRA application

[saddam213]

PR to introduce LoRA (Low Rank Adaptation) support

Adding most of this to the OnnxStack.Core as LoRA will be used for other projects outside StableDiffusion(Text Generation etc)

• IOnnxModelService to load/unload LoRA files
• New IOnnxAdaperService to manage LoRA application, support multiple applications
• Support Export of models with applied LoRA?
• Add support in IStableDiffusionService for IOnnxAdaperService
• Add UI support for selection/applying LoRA models

Research Links
https://medium.com/openvino-toolkit/enable-lora-weights-with-stable-diffusion-controlnet-pipeline-94fb06c86ea2

Discussion about this feature
#16

[Amin456789]

new lcm's 1.5 lora is very good if onnxstack can support it at some point!
https://huggingface.co/latent-consistency/lcm-lora-sdv1-5/tree/main

very low at size, can generate very good images at 4 steps

[saddam213]

Looks like I went down the wrong rabbit whole with the first implementation

However I think this may be the route
https://github.com/microsoft/onnxruntime/blob/main/csharp/test/Microsoft.ML.OnnxRuntime.Tests.Common/TrainingTest.cs

[Amin456789]

this repo just released lora and lcm lora=
https://github.com/leejet/stable-diffusion.cpp
it is a c++ sd project, it is using ggml though but maybe u can get help from its code as it is c++ related

[saddam213]

https://github.com/microsoft/Olive/blob/main/examples/directml/stable_diffusion/user_script.py#L35

[ForserX]

https://github.com/ForserX/StableDiffusionUI/blob/master/data/repo/diffusion_scripts/modules/onnx/lora.py
LoRA + LyCoris

[saddam213]

So what should have been a trivial thing to add has become impossible ATM.

There does not seem to be a way to access the model layers using the Onnx C# API :/

This will have to be done outside ONNX, so perhaps time to find another C# tensor library that can read onnx, and apply changes

[saddam213]

Not possible at this time, Closing