Skip to content

Commit

Permalink
Workaround HF Quantizer apply_quantization_config misuse (#180)
Browse files Browse the repository at this point in the history
* workaround hf quantizer apply none

* Add usage comment
  • Loading branch information
kylesayrs authored Oct 7, 2024
1 parent c2455b7 commit d6d823c
Showing 1 changed file with 6 additions and 2 deletions.
8 changes: 6 additions & 2 deletions src/compressed_tensors/quantization/lifecycle/apply.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,8 @@ def load_pretrained_quantization(model: Module, model_name_or_path: str):


def apply_quantization_config(
model: Module, config: QuantizationConfig, run_compressed: bool = False
) -> Dict:
model: Module, config: Union[QuantizationConfig, None], run_compressed: bool = False
) -> OrderedDict:
"""
Initializes the model for quantization in-place based on the given config
Expand All @@ -117,6 +117,10 @@ def apply_quantization_config(
:param run_compressed: Whether the model will be run in compressed mode or
decompressed fully on load
"""
# Workaround for when HF Quantizer passes None, see PR #180
if config is None:
return OrderedDict()

# remove reference to the original `config`
# argument. This function can mutate it, and we'd
# like to keep the original `config` as it is.
Expand Down

0 comments on commit d6d823c

Please sign in to comment.