Dev quant tools and fix graph file management #495

ccssu · 2024-01-05T05:15:19Z

onediff_comfy_nodes/_quant_tools.py

onediff_comfy_nodes/__init__.py

onediff_comfy_nodes/_quant_tools.py

onediff_comfy_nodes/_nodes.py

onediff_comfy_nodes/_quant_tools.py

onediff_comfy_nodes/_nodes.py

ccssu · 2024-01-11T15:06:21Z

onediff_comfy_nodes/utils/quant_ksampler_tools.py

+                mse = torch.mean((org_latent_sample - cur_latent_sample) ** 2)
+
+                ssim = 1 - mse


MSE : （原始模型生成的潜在样本空间 vs 量化模型其中一层生成的潜在样本空间）

sdxl 量化结果实验可以使用 1-mse 代表比较重建图片的 ssim 值。
sdxl_base_1_0_key_ssim_mse.txt

模型的参数名称 ssim 值 1-mse 值 time_embed.0 0.9756851169369525 0.9671265296638012 time_embed.2 0.9616645683538984 0.8562246561050415 label_emb.0.0 0.97451926291366 0.9558620862662792

下图所有 ssim < 0.97 的值（ ssim = 原始模型生成图片 vs 量化模型其中一层生成的图片)
ssim 值和 1- mse 值变化趋势是一致的，具有相关性。

下图所有 ssim < 0.98 的值（ ssim = 原始模型生成图片 vs 量化模型其中一层生成的图片)

看起来mse指标分布更广，区分度更高一些，可以尝试用mse替换ssim量化一下sdxl + deepcache看看效果

左图 sdxl + deepcache 右图 sdxl + 量化+ deepcache

Quantized conv: 3

Quantized linear: 582

image: 1024x1024 , steps: 20

onediff_comfy_nodes/_nodes.py

onediff_comfy_nodes/utils/quant_ksampler_tools.py

ccssu · 2024-01-13T15:07:56Z

src/onediff/infer_compiler/utils/graph_management_utils.py

+            count = len(
+                [v for v in args_tree.iter_nodes() if isinstance(v, flow.Tensor)]
+            )
+            return f"{graph_file}_{count}.graph"


选择在 f"{graph_file} 后加个输入参数的数量。 @strint

在 Load Checkpoint - OneDiff 节点添加个指定图文件的参数，也无法解决下面工作流的错误。

Check failed: (x== y) 错误

使用Oendiff 的一个⚠️注意点，最佳实践是确保每个数据加载器（loader）只与一个独立的采样器（ksampler）相对应。

注意点2: 在 ControlNet 节点中，strength 参数不支持值为0 和大于0的之间的切换。 0 代表相当于取消 ControlNet 作用。大于0的值代表使用 ControlNet。请不要在 0 和大于0的之间切换，否则会出现错误。

onediff_comfy_nodes/_nodes.py

hjchen2 · 2024-01-15T10:44:15Z

onediff_comfy_nodes/utils/quant_ksampler_tools.py

+                if index % 10 == 0 or index == length - 1:
+                    torch.save(calibrate_info, cached_process_output_path)


这个cache行为需要用户显式指定，否则可能会导致写过多的文件和目录？

这个cache行为需要用户显式指定，否则可能会导致写过多的文件和目录？

这里 cache 对应 onediff-quant 量化脚本中 quantize_info_and_relevance_metrics.pt 文件。

https://github.com/siliconflow/onediff-quant/blob/dc8a3ba8e553ba43d269b701f2b16aea288a78ff/tools/quantize-sd-fast-with-threshold.py#L125

是加个 cached_process_output：["enable","disable"] ？

我是觉得comfy的工具不一定需要支持cache，如果是为了让用户可以调那几个量化参数，大不了把calibrate_info在内存里复制一份，每次改完参数都生成一个新的calibrate_info输出到用户指定的目录，而原始的calibrate_info是不会被修改的。

如果非要cache的话，我有几个问题：

切换模型会不会导致被误用，如何判断cache是否会被命中？

cache的目录什么时候被清空，需不需要用户手动操作，用户如何知道该清空哪个目录？

hjchen2 · 2024-01-15T10:46:00Z

onediff_comfy_nodes/utils/quant_ksampler_tools.py

+                )
+
+                calibrate_info[sub_name] = {
+                    "ssim": similarity_mse,


ssim就叫mse会比较准确

有个问题，mse 是越大两个图越不相似。而这里的 similarity_mse （ 1-mse ）、以及真正的 ssim 是数值越大，两个图越相似。

那就叫max_mse_loss？loss会比较容易让人知道这个值越小越好

这两个参数改为 conv_mse_threshold , linear_mse_threshold , 直接不用 ssim 代表可行吗

我觉得可以，mse就是越小越相似，默认值改成0.1

ccssu and others added 4 commits January 5, 2024 00:36

init

d692c81

add _quant_tools.py

14d2612

refine

72ccb99

Merge branch 'main' into dev_quant_tools

df24b91

ccssu requested review from clackhan, doombeaker and hjchen2 January 5, 2024 05:54

ccssu commented Jan 5, 2024

View reviewed changes

onediff_comfy_nodes/_quant_tools.py Outdated Show resolved Hide resolved

ccssu added 3 commits January 7, 2024 00:02

refine quant tools

1ee0ac8

refine

89a3310

refine

718c68e

doombeaker reviewed Jan 8, 2024

View reviewed changes

onediff_comfy_nodes/_quant_tools.py Outdated Show resolved Hide resolved

ccssu commented Jan 8, 2024

View reviewed changes

onediff_comfy_nodes/_nodes.py Outdated Show resolved Hide resolved

ccssu and others added 8 commits January 9, 2024 11:00

Merge branch 'main' into dev_quant_tools

35b93b9

Merge branch 'main' into dev_quant_tools

8d4c58e

Merge branch 'main' into dev_quant_tools

aaf9606

refine

9c8aef1

refine

f0449e6

refine

21ecaa5

Merge branch 'main' into dev_quant_tools

e9a21a5

refine

f1f6ea2

ccssu commented Jan 11, 2024

View reviewed changes

fix use lora

9bf67b2

ccssu commented Jan 12, 2024

View reviewed changes

onediff_comfy_nodes/_nodes.py Outdated Show resolved Hide resolved

refine

70229b9

doombeaker reviewed Jan 12, 2024

View reviewed changes

onediff_comfy_nodes/utils/quant_ksampler_tools.py Outdated Show resolved Hide resolved

ccssu and others added 3 commits January 12, 2024 13:00

replace ssim with mean squared error similarity

e84b2d2

Merge branch 'main' into dev_quant_tools

9f05586

refine generate_graph_path

bf83a6e

ccssu and others added 2 commits January 13, 2024 22:57

Merge branch 'main' into dev_quant_tools

719ed5d

refine generate_graph_path

1bd5755

ccssu commented Jan 13, 2024

View reviewed changes

doombeaker and others added 2 commits January 15, 2024 09:16

Merge branch 'main' into dev_quant_tools

94c174d

Merge branch 'main' into dev_quant_tools

8b76939

ccssu changed the title ~~Dev quant tools~~ Dev quant tools and fix graph file management Jan 15, 2024

hjchen2 reviewed Jan 15, 2024

View reviewed changes

onediff_comfy_nodes/_nodes.py Outdated Show resolved Hide resolved

hjchen2 reviewed Jan 15, 2024

View reviewed changes

ccssu added 5 commits January 15, 2024 22:08

Cancel the compile param

827a7c5

refine QuantKSampler

6fdac4f

refine

ff73810

refine QuantKSampler

d1d7fca

compile_model_patche

d99b54a

hjchen2 approved these changes Jan 16, 2024

View reviewed changes

hjchen2 merged commit d472b13 into main Jan 16, 2024
1 of 4 checks passed

hjchen2 deleted the dev_quant_tools branch January 16, 2024 06:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dev quant tools and fix graph file management #495

Dev quant tools and fix graph file management #495

ccssu commented Jan 5, 2024 •

edited

Loading

ccssu Jan 11, 2024 •

edited

Loading

ccssu Jan 11, 2024

hjchen2 Jan 11, 2024

ccssu Jan 12, 2024

ccssu Jan 13, 2024

ccssu Jan 13, 2024 •

edited

Loading

ccssu Jan 13, 2024 •

edited

Loading

hjchen2 Jan 15, 2024

ccssu Jan 15, 2024

hjchen2 Jan 15, 2024

ccssu Jan 16, 2024

hjchen2 Jan 15, 2024

doombeaker Jan 15, 2024

hjchen2 Jan 15, 2024

ccssu Jan 15, 2024

hjchen2 Jan 15, 2024 •

edited

Loading

		mse = torch.mean((org_latent_sample - cur_latent_sample) ** 2)

		ssim = 1 - mse

		if index % 10 == 0 or index == length - 1:
		torch.save(calibrate_info, cached_process_output_path)

Dev quant tools and fix graph file management #495

Dev quant tools and fix graph file management #495

Conversation

ccssu commented Jan 5, 2024 • edited Loading

ccssu Jan 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ccssu Jan 13, 2024 • edited Loading

Choose a reason for hiding this comment

Check failed: (x== y) 错误

ccssu Jan 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hjchen2 Jan 15, 2024 • edited Loading

Choose a reason for hiding this comment

ccssu commented Jan 5, 2024 •

edited

Loading

ccssu Jan 11, 2024 •

edited

Loading

ccssu Jan 13, 2024 •

edited

Loading

ccssu Jan 13, 2024 •

edited

Loading

hjchen2 Jan 15, 2024 •

edited

Loading