[Trainer] Fix .to call on 4bit models

[younesbelkada]

What does this PR do?

Currently the Trainer fails when calling initializing it on some scenarios using 4bit models
In fact, the device placement is correctly skipped for 8bit models but needs to be skipped as well for 4bit models as the to operation is not supported for 4bit models as well.
This PR adds a patch for that case

cc @amyeroberts @lewtun

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[amyeroberts]

LGTM - thanks for fixing!

General question - as there's two flags, "is_loaded_in_8bit" and "is_quantized", are we guaranteed that all 8bit quantized (and 4 bit) have these flags correctly set?

[younesbelkada]

@amyeroberts thanks!
Absolutely yes, for reference, here is how we set that attribute:

transformers/src/transformers/modeling_utils.py

Line 2922 in ea91c2a

model.is_quantized = load_in_8bit or load_in_4bit

[lewtun]

I can confirm that this fix resolves the error I was hitting with 4-bit models:

Traceback (most recent call last):
  File "/fsx/lewis/git/h4/scripts/evaluation/run_rm_eval.py", line 275, in <module>
    main()
  File "/fsx/lewis/git/h4/scripts/evaluation/run_rm_eval.py", line 164, in main
    trainer = RewardTrainer(
  File "/fsx/lewis/git/h4/src/h4/training/trainer.py", line 26, in __init__
    super().__init__(*args, **kwargs)
  File "/fsx/lewis/miniconda/envs/h4/lib/python3.10/site-packages/transformers/trainer.py", line 506, in __init__
    self._move_model_to_device(model, args.device)
  File "/fsx/lewis/miniconda/envs/h4/lib/python3.10/site-packages/transformers/trainer.py", line 747, in _move_model_to_device
    model = model.to(device)
  File "/fsx/lewis/miniconda/envs/h4/lib/python3.10/site-packages/transformers/modeling_utils.py", line 1889, in to
    raise ValueError(
ValueError: `.to` is not supported for `4-bit` or `8-bit` models. Please use the model as it is, since the model has already been set to the correct devices and casted to the correct `dtype`.

Thanks for the fast fix @younesbelkada !!