[Auto] Support AutoPeftModel for custom HF models

[younesbelkada]

What does this PR do?

Extends the recent PR: #694 by adding the support for all HF models: whisper, etc.

The logic is as follows:

• Since if a user trains a model with task_type=None, it must be a custom model, we add to the peft config dictionary a new field termed as auto_mapping, responsible of storing all required information for retrieving back this class: https://huggingface.co/ybelkada/whisper-lora-test-auto-mapping/blob/main/adapter_config.json#L2
• For users that want to register whisper/ sd etc models in the future to be compatible with AutoPeftModel they need to load the model from the Hub and push it again

Current API:

- from peft import PeftModel, PeftConfig, AutoPeftModel
+ from peft import AutoPeftModel
- from transformers import WhisperForConditionalGeneration

- model_id = "smangrul/openai-whisper-large-v2-LORA-colab"

peft_model_id = "smangrul/openai-whisper-large-v2-LORA-colab"
- peft_config = PeftConfig.from_pretrained(peft_model_id)
- model = WhisperForConditionalGeneration.from_pretrained(
-     peft_config.base_model_name_or_path, load_in_8bit=True, device_map="auto"
- )
- model = PeftModel.from_pretrained(model, peft_model_id)
+ model = AutoPeftModel.from_pretrained(peft_model_id)

Added tests for Whisper

cc @BenjaminBossan @pacman100

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[BenjaminBossan]

LGTM, but it's outside my area of expertise, so let's wait for Sourab's review before merging.

[pacman100]

Thank you @younesbelkada, this is a pretty cool way of retaining info on base modules/classes, enabling simplified UX for lot of tasks that aren't explicitly covered when using LoRAs. LGTM! 🚀