Environment for running the code

[lxuechen]

Could the authors share the requirements and general environment for running this code? I am also hitting another few issues, and currently trying to infer the right versions of libraries.

[artidoro]

Hey! Thanks a lot for your interest in QLoRA. The necessary changes to use QLoRA will be merged in the transformers library tomorrow morning and we will update this repo with installation instructions.

[lxuechen]

Which transformers branch / PR is this? I can install from source.

[qwopqwop200]

Try this branch.
https://github.com/2021-DGSW-Ensemble/transformers

[artidoro]

FYI this is the PR: huggingface/transformers#23479

[lxuechen]

FYI this is the PR: huggingface/transformers#23479

Great thanks!

[Qubitium]

@artidoro paged_adamw_32bit throws this error

  File "/root/transformers-4bit/src/transformers/utils/generic.py", line 348, in _missing_
    raise ValueError(
ValueError: paged_adamw_32bit is not a valid OptimizerNames, please select one of ['adamw_hf', 'adamw_torch', 'adamw_torch_fused', 'adamw_torch_xla', 'adamw_apex_fused', 'adafactor', 'adamw_bnb_8bit', 'adamw_anyprecision', 'sgd', 'adagrad']

Already on huggingface/transformers#23479

Env:
Cuda 12.1
Latest compiled bitsandbytes with 4bit merged.
Latest transformer PR 23479

[qwopqwop200]

@artidoro paged_adamw_32bit throws this error

  File "/root/transformers-4bit/src/transformers/utils/generic.py", line 348, in _missing_
    raise ValueError(
ValueError: paged_adamw_32bit is not a valid OptimizerNames, please select one of ['adamw_hf', 'adamw_torch', 'adamw_torch_fused', 'adamw_torch_xla', 'adamw_apex_fused', 'adafactor', 'adamw_bnb_8bit', 'adamw_anyprecision', 'sgd', 'adagrad']

Already on huggingface/transformers#23479

Try this branch.
https://github.com/2021-DGSW-Ensemble/transformers
This fork solves that problem. However, I'm running into another problem.
#3

[Qubitium]

@qwopqwop200 Confirmed https://github.com/2021-DGSW-Ensemble/transformers fixed paged_adamw_32bit not found.

@artidoro The transformer code that qlora mentions requires both the main PR 23479 and the paged lion PR to be merged to work, for now. May need to update the readme.

[artidoro]

Closing the issue as we now have info on how to install relevant packages. Let us know if you still have problems.