Base Model Revision

[mnoukhov]

Feature request

Currently, the adapter_config only stores the base_model_name_or_path but not the revision. This means that if a PeftModel is trained on top of a model with a revision then loading PeftModel.from_pretrained will load the main revision of the base model, not the correct revision.

We should add a base_model_revision to the adapter_config so that we can correctly load the base model

Motivation

This will cause a bug if using PeftModel.from_pretrained when the base model has a revision.

Your contribution

I can try to submit a PR if I have time in 2-3 weeks

[BenjaminBossan]

Thanks for this suggestion, it sounds good. A PR would be great, but no need to rush it.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

[BenjaminBossan]

@mnoukhov are you still interested in providing a PR?

[mnoukhov]

I looked into it and found that there's already actually a revision parameter

peft/src/peft/config.py

Line 235 in 56773b9

revision: Optional[str] = field(default=None, metadata={"help": "The specific model version to use."})

It's just not being used when calling from_pretrained here

peft/src/peft/auto.py

Line 104 in 56773b9

base_model = target_class.from_pretrained(base_model_path, **kwargs)

I've opened the PR #1658 but in order to do unit tests, I need a model with a base model with revision. If you don't mind this not being on hf-internal-testing, I can upload it to my personal account, but maybe you'd prefer the former. Let me know @BenjaminBossan

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

[BenjaminBossan]

not stale

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

[mnoukhov]

resolved by #1658