[bitsandbytes]: support read bnb pre-quantized model#5753

Merged

mgoin merged 2 commits intovllm-project:mainfrom bd-iaas-us:nf5

Jul 23, 2024

+143-39

Commits on Jul 23, 2024

[bitsandbytes]: support read bnb pre-quantized model such as lllyasviel/omost-llama-3-8b-4bits
thesues
committed
Update docs/source/quantization/bnb.rst

thesues
and
mgoin
committed