Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] GPTQ Docstring, better argument grouping #841

Closed
wants to merge 8 commits into from
14 changes: 8 additions & 6 deletions src/llmcompressor/modifiers/quantization/gptq/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,8 +68,8 @@ class GPTQModifier(Modifier):

:param sequential_update: Whether or not to update weights sequentially by layer,
True saves on GPU memory, default is True
:param targets: list of layer names to compress during GPTQ, or '__ALL__'
kylesayrs marked this conversation as resolved.
Show resolved Hide resolved
to compress every layer in the model
:param sequential_targets: list of layer names to compress during GPTQ, or
'__ALL__' to compress every layer in the model
:param block_size: Used to determine number of columns to compress in one pass
:param quantize: Set to True to quantize using an existing quantization modifier,
or pass in the configuration for a quantization modifier if one does not
Expand Down Expand Up @@ -98,16 +98,18 @@ class GPTQModifier(Modifier):
"""

sequential_update: bool = True
targets: Union[str, List[str], None] = None
sequential_targets: Union[str, List[str], None] = None
block_size: int = 128
quantize: Union[bool, Dict] = True
dampening_frac: Optional[float] = 0.01
quantize: Union[bool, Dict] = True

# arguments used for quant modifier
config_groups: Optional[Dict[str, QuantizationScheme]] = None
scheme: Optional[Union[str, Dict[str, Any]]] = None
targets: Union[str, List[str], None] = None
ignore: List[str] = Field(default_factory=list)
disable_quantization_observer_epoch: Optional[float] = None
num_calibration_steps: Optional[int] = None
scheme: Optional[Union[str, Dict[str, Any]]] = None
disable_quantization_observer_epoch: Optional[float] = None

model: Optional[Any] = None
layer_compressors_: Optional[List[Any]] = None
Expand Down
Loading