[RFC]: Remove LoRA bias

[jeejeelee]

Motivation.

#5733 introduced LoRA bias(by using enable_lora_bias arg). We have some doubts about this implementation and have also discussed it with the author on Slack, see lora bias thread.
Considering that LoRA bias has been introduced for about 10 months now, there are almost no issues related to it and this feature appears to be rarely used, so we are considering deleting the LoRA bias-related code.

Proposed Change.

Divided into 2 stages:

• Add deprecation warning for enable_lora_bias, Lora bias(enable_lora_bias) deprecate warning #24339
• After keeping the warning for a period of time, delete related code.

Feedback Period.

one week

CC List.

@njhill @followumesh

Any Other Things.

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[simon-mo]

In favor of cleaning up the code!

[ashwin-phadke]

Once #24339 is merged, I can cleanup the code and raise another PR.

[ashwin-phadke]

What does the timeline look like for code cleanup? Should I raise a PR and keep it merge ready when we plan to do it or hold off until we actually have the next release?
I sus[ect we will also need to update docs accordingly for both those points, do we usually put some signs or statements that say deprecating xyz? or just remove it once we delete the code?

[jeejeelee]

After the next version is released, we can consider removing the related code

[ashwin-phadke]

appreciate it, I will hold off until the release.

[jeejeelee]

@ashwin-phadke We can remove the lora_bias code now

[ashwin-phadke]

@ashwin-phadke We can remove the lora_bias code now

Sure.

[jeejeelee]

@ashwin-phadke Sorry to bother you, but could you please continue to complete this issue?

[ashwin-phadke]

Hey @jeejeelee yes I am actively working on it, I should have something up pretty soon in couple days.

[ashwin-phadke]

@jeejeelee #25807 in draft.

1. Going through the layers folder, I had a layers file and fully_sharded_layers file which no longer exist, need to look into this matter.