We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The recent commit a0a92e0 flipped A and B in the comment for the LoRA Linear module:
A
B
Linear
LoRA/loralib/layers.py
Lines 119 to 125 in a0a92e0
The LoRA Embedding module similarly has the initialization flipped (not sure if this is intentional):
Embedding
Lines 55 to 60 in a0a92e0
nn.init.normal_(self.lora_A)
nn.init.zeros_(self.lora_B)
I can open a PR to fix these if you want (though, I cannot seem to save the file without removing trailing whitespaces for some reason).
The text was updated successfully, but these errors were encountered:
No branches or pull requests
The recent commit a0a92e0 flipped
A
andB
in the comment for the LoRALinear
module:LoRA/loralib/layers.py
Lines 119 to 125 in a0a92e0
The LoRA
Embedding
module similarly has the initialization flipped (not sure if this is intentional):LoRA/loralib/layers.py
Lines 55 to 60 in a0a92e0
Following the paper, I would expect
nn.init.normal_(self.lora_A)
andnn.init.zeros_(self.lora_B)
.I can open a PR to fix these if you want (though, I cannot seem to save the file without removing trailing whitespaces for some reason).
The text was updated successfully, but these errors were encountered: