Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PEFT] Add warning for missing key in LoRA adapter #34068

Merged

Conversation

BenjaminBossan
Copy link
Member

What does this PR do?

When loading a LoRA adapter, so far, there was only a warning when there were unexpected keys in the checkpoint. Now, there is also a warning when there are missing keys.

This change is consistent with
huggingface/peft#2118 in PEFT and the planned PR huggingface/diffusers#9622 in diffusers.

Apart from this change, the error message for unexpected keys was slightly altered for consistency (it should be more readable now). Also, besides adding a test for the missing keys warning, a test for unexpected keys warning was also added, as it was missing so far.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

When loading a LoRA adapter, so far, there was only a warning when there
were unexpected keys in the checkpoint. Now, there is also a warning
when there are missing keys.

This change is consistent with
huggingface/peft#2118 in PEFT and the planned PR
huggingface/diffusers#9622 in diffusers.

Apart from this change, the error message for unexpected keys was
slightly altered for consistency (it should be more readable now). Also,
besides adding a test for the missing keys warning, a test for
unexpected keys warning was also added, as it was missing so far.
@BenjaminBossan
Copy link
Member Author

@ArthurZucker Could you please review or suggest a reviewer?

@BenjaminBossan
Copy link
Member Author

Ping @ArthurZucker

@LysandreJik
Copy link
Member

@SunMarc can you take a look as well if you have some bandwidth?

Copy link
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I am wondering why we won't init the model with adapter, load with transformers, then just do the matching we usually have in transformers, but that's more for me!
🤗

@BenjaminBossan
Copy link
Member Author

Thanks for the review.

Thanks, I am wondering why we won't init the model with adapter, load with transformers, then just do the matching we usually have in transformers, but that's more for me!

I'm not sure what you mean there, but if there is something I can look into, LMK.

Is there anything missing to merge the PR? If not, as I don't have rights, could you merge please 🙇

@ArthurZucker ArthurZucker merged commit d9989e0 into huggingface:main Oct 24, 2024
24 checks passed
@ArthurZucker
Copy link
Collaborator

Don't worry it's for me ! 🤗

BernardZach pushed a commit to BernardZach/transformers that referenced this pull request Dec 5, 2024
When loading a LoRA adapter, so far, there was only a warning when there
were unexpected keys in the checkpoint. Now, there is also a warning
when there are missing keys.

This change is consistent with
huggingface/peft#2118 in PEFT and the planned PR
huggingface/diffusers#9622 in diffusers.

Apart from this change, the error message for unexpected keys was
slightly altered for consistency (it should be more readable now). Also,
besides adding a test for the missing keys warning, a test for
unexpected keys warning was also added, as it was missing so far.
BernardZach pushed a commit to innovationcore/transformers that referenced this pull request Dec 6, 2024
When loading a LoRA adapter, so far, there was only a warning when there
were unexpected keys in the checkpoint. Now, there is also a warning
when there are missing keys.

This change is consistent with
huggingface/peft#2118 in PEFT and the planned PR
huggingface/diffusers#9622 in diffusers.

Apart from this change, the error message for unexpected keys was
slightly altered for consistency (it should be more readable now). Also,
besides adding a test for the missing keys warning, a test for
unexpected keys warning was also added, as it was missing so far.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants