Support Flux quantization

[mengniwang95]

for model_name in "/mengni/flux/"
do
  CUDA_VISIBLE_DEVICES="7"  python3 -m auto_round \
    --scheme MXFP4
    --format "llm_compressor" \
    --output_dir "/mengni/tmp_flux" \
    --prompt_file "captions_source.tsv" \ # download from https://github.com/mlcommons/inference/raw/refs/heads/master/text_to_image/coco2014/captions/captions_source.tsv
    --metrics "clip,clip-iqa,imagereward" \
    --dataset coco2014
done

[wenhuach21]

Don’t we need to add these requirements to our repo’s requirements file? If not, we should provide an auto-round[diffusion] installation instead, right?. please check with suyue/xuhao

[chensuyue]

If those deps are not force required by auto_round, we can raise issue when user run into those code and ask them to install.

[wenhuach21]

please reminder users that this is experimental feature and only validated on limited models ,e.g.xxx

[wenhuach21]

--prompt_file "captions_source.tsv"
--metrics "clip,clip-iqa,imagereward"
1 is this for evaluation? It's hard for user to understand, please wrapper it as tasks
2 please follow the original behavior, if tasks is not set, no evaluation is conducted

@n1ck-guo @yiliu30 As there are so many args in main.py, we need a better way for auto-round -h

[mengniwang95]

--prompt_file "captions_source.tsv" --metrics "clip,clip-iqa,imagereward" 1 is this for evaluation? It's hard for user to understand, please wrapper it as tasks 2 please follow the original behavior, if tasks is not set, no evaluation is conducted

@n1ck-guo @yiliu30 As there are so many args in main.py, we need a better way for auto-round -h

--prompt_file and --metrics are for evaluation. It is hard to wrapper prompt_file because there is no standard dataset in huggingface and other repos usually ask user to prepare dataset by themselves https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/examples/diffusers/README.md#data-format

[wenhuach21]

@WeiweiZhang1 I forgot the reason why we need to set the data_type, could we fix this issue, I thinks bits=16 is enough for fp layers

[mengniwang95]

updated, bits and act_bits are needed

[wenhuach21]

@WeiweiZhang1 Could you help fix the act_bits issue later? It shouldn’t need to be set manually.

[mengniwang95]

@wenhuach21 CI has passed, can we merge this PR?