Fix unwrapping peft models

[kkteru]

Fixes issues discussed here.

cc @younesbelkada

[younesbelkada]

Makes sense, thanks!

[younesbelkada]

Hi @kkteru
Can you merge this branch with main branch to make sure tests pass?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[kkteru]

Interesting, somehow the test_sft_trainer.py tests failed on latest main branch (without any changes from this branch) in my local. Investigating further.

[younesbelkada]

@kkteru what are the failed tests? might be unrelated to your changes I think - you can just merge your branch with upstream main and see here if the tests pass

[kkteru]

These ones fail both on the main branch and this branch (merged with main) on my system. I just rebased on top of latest main and pushed. I dont see the tests running here?

=================================================================== short test summary info ====================================================================
FAILED tests/test_sft_trainer.py::SFTTrainerTester::test_sft_trainer - AssertionError: False is not true
FAILED tests/test_sft_trainer.py::SFTTrainerTester::test_sft_trainer_infinite_with_model - AssertionError: False is not true
FAILED tests/test_sft_trainer.py::SFTTrainerTester::test_sft_trainer_infinite_with_model_epochs - AssertionError: False is not true
FAILED tests/test_sft_trainer.py::SFTTrainerTester::test_sft_trainer_with_model - AssertionError: False is not true
FAILED tests/test_sft_trainer.py::SFTTrainerTester::test_sft_trainer_with_model_num_train_epochs - AssertionError: False is not true

[younesbelkada]

The tests were failing because you had an old version of transformers, installing the 4.35.0 will fix your isses! As you can see, CI is green now :D

[younesbelkada]

Awesome clean up !