add ia3 and adalora support

[sywangyi]

What does this PR do?

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sywangyi]

huggingface/peft#1540 fixed adalora rank==0 issue in inference load model.

[sywangyi]

update peft to 0.10.0 which contains huggingface/peft#1540

[sywangyi]

adalora + deepspeed zero3 finetune fix in peft huggingface/peft#1625

[sywangyi]

IA3 finetune with "--use_flash_attention" need PR fix in huggingface/peft#1634. since q, k, v should has some data type in FusedSDPA.apply

[regisss]

For PEFT v0.10, there was an issue with higher memory consumption from PEFT v0.7: #590
Let me try to see if that works.

[regisss]

We're still having an issue with Falcon-180b and more recent versions of PEFT. Let me see how we can do.

[sywangyi]

try #895 and it could fix falcon 180b issue

[regisss]

@sywangyi LGTM! Can you share a command for both ia3 and adalora to make sure I can run them please?

[sywangyi]

Hi, @regisss . you could use "--peft_type ia3" for IA3 and "--peft_type adalora" for adalora in run_lora_clm.py command for alpaca dataset

like

python3 run_lora_clm.py \
  --model_name_or_path meta-llama/Llama-2-7b-hf \
  --dataset_name tatsu-lab/alpaca \
  --output_dir ./lora_out \
  --num_train_epochs 1 \
  --max_step 500 \
  --max_seq_len 2048 \
  --per_device_train_batch_size 1 \
  --per_device_eval_batch_size 10 \
  --evaluation_strategy epoch \
  --eval_delay 2 \
  --save_strategy no \
  --learning_rate 0.0018 \
  --warmup_ratio 0.03 \
  --lr_scheduler_type "cosine" \
  --logging_steps 1 \
  --dataset_concatenation \
  --do_train \
  --do_eval \
  --use_habana \
  --use_lazy_mode \
  --throughput_warmup_steps 3 \
  --lora_rank 4 \
  --lora_target_modules "q_proj" "v_proj" "k_proj" "o_proj" \
  --validation_split_percentage 4 \
  --peft_type ia3 \
  --pipelining_fwd_bwd \
  --bf16 \
  --attn_softmax_bf16 True 

python3 run_lora_clm.py \
  --model_name_or_path meta-llama/Llama-2-7b-hf \
  --dataset_name tatsu-lab/alpaca \
  --output_dir ./lora_out \
  --num_train_epochs 1 \
  --max_step 500 \
  --max_seq_len 2048 \
  --per_device_train_batch_size 1 \
  --per_device_eval_batch_size 10 \
  --evaluation_strategy epoch \
  --eval_delay 2 \
  --save_strategy no \
  --learning_rate 0.0018 \
  --warmup_ratio 0.03 \
  --lr_scheduler_type "cosine" \
  --logging_steps 1 \
  --dataset_concatenation \
  --do_train \
  --do_eval \
  --use_habana \
  --use_lazy_mode \
  --throughput_warmup_steps 3 \
  --lora_rank 4 \
  --lora_target_modules "q_proj" "v_proj" "k_proj" "o_proj" \
  --validation_split_percentage 4 \
  --peft_type adalora\
  --pipelining_fwd_bwd \
  --bf16 \
  --attn_softmax_bf16 True