-
Notifications
You must be signed in to change notification settings - Fork 27.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[PEFT] Add warning for missing key in LoRA adapter #34068
[PEFT] Add warning for missing key in LoRA adapter #34068
Conversation
When loading a LoRA adapter, so far, there was only a warning when there were unexpected keys in the checkpoint. Now, there is also a warning when there are missing keys. This change is consistent with huggingface/peft#2118 in PEFT and the planned PR huggingface/diffusers#9622 in diffusers. Apart from this change, the error message for unexpected keys was slightly altered for consistency (it should be more readable now). Also, besides adding a test for the missing keys warning, a test for unexpected keys warning was also added, as it was missing so far.
@ArthurZucker Could you please review or suggest a reviewer? |
Ping @ArthurZucker |
@SunMarc can you take a look as well if you have some bandwidth? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, I am wondering why we won't init the model with adapter, load with transformers, then just do the matching we usually have in transformers, but that's more for me!
🤗
Thanks for the review.
I'm not sure what you mean there, but if there is something I can look into, LMK. Is there anything missing to merge the PR? If not, as I don't have rights, could you merge please 🙇 |
Don't worry it's for me ! 🤗 |
When loading a LoRA adapter, so far, there was only a warning when there were unexpected keys in the checkpoint. Now, there is also a warning when there are missing keys. This change is consistent with huggingface/peft#2118 in PEFT and the planned PR huggingface/diffusers#9622 in diffusers. Apart from this change, the error message for unexpected keys was slightly altered for consistency (it should be more readable now). Also, besides adding a test for the missing keys warning, a test for unexpected keys warning was also added, as it was missing so far.
When loading a LoRA adapter, so far, there was only a warning when there were unexpected keys in the checkpoint. Now, there is also a warning when there are missing keys. This change is consistent with huggingface/peft#2118 in PEFT and the planned PR huggingface/diffusers#9622 in diffusers. Apart from this change, the error message for unexpected keys was slightly altered for consistency (it should be more readable now). Also, besides adding a test for the missing keys warning, a test for unexpected keys warning was also added, as it was missing so far.
What does this PR do?
When loading a LoRA adapter, so far, there was only a warning when there were unexpected keys in the checkpoint. Now, there is also a warning when there are missing keys.
This change is consistent with
huggingface/peft#2118 in PEFT and the planned PR huggingface/diffusers#9622 in diffusers.
Apart from this change, the error message for unexpected keys was slightly altered for consistency (it should be more readable now). Also, besides adding a test for the missing keys warning, a test for unexpected keys warning was also added, as it was missing so far.
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.