Cleanup ModelCompressor, fix reload bugs

[kylesayrs]

Purpose

Track changes across PRs

• Allow ModelCompressor to load quantization configs #173
• Fix missing quant_method value #174
• Fix ModelCompressor parsing in HF Quantizer case #176

[dsikka]

@kylesayrs The first was set-up by design. This gives us two separate pathways, which we would like to keep for the time being

[dsikka]

I would suggest opening separate PRs for each bug fix that you'd like to have changed. Otherwise, the PR can't be accepted in its current state.

[kylesayrs]

@dsikka w.r.t. your first comment, doesn't this mean that users can no longer load back models using SparseAutoModelForCausalLM? The reload tests I added seem like reasonable use cases, but they fail on main without these changes.

[dsikka]

@dsikka w.r.t. your first comment, doesn't this mean that users can no longer load back models using SparseAutoModelForCausalLM? The reload tests I added seem like reasonable use cases, but they fail on main without these changes.

Please read through the description of the PR you referenced: #164
Legacy models will still be able to load through the the SparseAutoModelForCausalLM pathway when running inference. The scope of what we're changing right now is focused on inference.

[kylesayrs]

@dsikka When I was referring to loading back models, I wasn't referring to legacy models, I was referring to models compressed today (with quantization_config).

There are a couple reasons which motivate why ModelCompressor should support loading from either a compression_config (legacy) or a quantization_config, as implemented here.

1. In the reload case where HF quantizer is not available (ie the transformer version is not up-to-date), SparseAutoModelForCausalLM will need to use ModelCompressor to parse a config with the quantization_config key. I would point out the this failure in the LC tests, but LC is failing for other reasons right now, and I believe another bug would be masking this effect anyways (namely that in wrap_hf_model_class, it's assumed that if a quantization_config is present, then HF quantizer is active, which is not necessarily the case)

2. It's nice to maintain the property that if ModelCompressor saves quantization_configs, then ModelCompressor can parse/ be loaded from quantization_configs

[kylesayrs]

Relevant: #164 (review)

Do we need to set the transformers minimum version to 4.45 as well to load the models back in?

[kylesayrs]

Split up into 3 PRs, some of which are pending greater changes later