issues to fix reported from discord

[winglian]

Bambi#1600
I can report my observations from my attempts at int8 LoRA training via the trainer built into Oobabooga’s textgen webUI if it helps:

1. Myself and others are able to train LoRAs with int8 precision for the original unquantized HF llama-7b and llama-13b models
2. The LoRA from this train produced expected results at inference when applied to the unquantized llama models
3. VRAM usage during the train was observed to be evenly seemed split between cards
4. GPU utilization however was observed to alternate between the cards (one card was pulling 150 watts, the other pulling 300 watts then they’d swap) indicating a serialized but threaded workload vs true parallelization
5. Encountered but upon saving first checkpoint causing both cards to OOM. Following numerous forum threads we reverted out bitsandbytes model from 0.38.1 to 0.37.2 which resolved the issue.

[winglian]

I think these have been resolved by now