An error during merging #95

miasik · 2023-12-26T10:51:31Z

Tried different models with the same result.

debugs =  ['elemental merge']
use_extra_elements =  True
 - mm_max_models =  4
config hash =  f79746d04db933c7fd3d9ae6cb70b3bd562a09ed41e48bc894b4fec58d5259e1
  - mm_use [True, False, False, False]
  - model_a Photo\nextphoto_v30.safetensors [1c1f913f3b]
  - base_model v1-5-pruned-emaonly.safetensors [6ce0161689]
  - max_models 4
  - models ['Photo\\rundiffusionFX_v10.safetensors [ad1a10552b]']
  - modes ['Add-Diff']
  - calcmodes ['Rebasin']
  - usembws [[]]
  - weights ['0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5']
  - alpha [0.75]
  - adjust 1,1,1,0,0,0,0,0
  - use elemental [False]
  - elementals ['']
  - Parse elemental merge...
model_a = Photo_nextphoto_v30
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Photo\nextphoto_v30.safetensors...
isxl = False
compact_mode =  False
 - Dynamic loading rebasin module...
Rebasin mode
 - Calulation device for Rebasin is  cuda
 - LAP library is lap
Loading model Photo_rundiffusionFX_v10...
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Photo\rundiffusionFX_v10.safetensors...
mode = Add-Diff, alpha = 0.75
Stage #1/2: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1132/1132 [00:15<00:00, 74.62it/s]
Check uninitialized #2/2: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1132/1132 [00:00<00:00, 189153.90it/s]
Rebasin calc...
maximize weight matching using scipy linear_sum_assignment...
P_bg337:  24%|██████████Special layer P_bg337 found███████████████▉                                                                                                                                                                       | 125/519 [00:12<00:25, 15.58it/s]
P_bg265: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 519/519 [00:38<00:00, 13.33it/s]
P_bg176: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 519/519 [00:35<00:00, 14.43it/s]
P_bg176: new - old = 0.0
weight order changed layers = ['P_bg371', 'P_bg324', 'P_bg337', 'P_bg358']
Save unchanged weights #2/2: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<?, ?it/s]
Apply fine tune [0.99, 1.02, 0.99, 1.02, 0.99, [0.0, 0.0, 0.0, 0.0]]
Clip is fine
 - merge processing in 95.3s (prepare: 0.4s, merging: 94.9s).
*** Error running before_process: E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\scripts.py", line 710, in before_process
        script.before_process(p, *script_args)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 3746, in before_process
        sd_models.send_model_to_cpu(sd_models.model_data.sd_model)
      File "E:\SD\automatic1111\modules\sd_models.py", line 576, in send_model_to_cpu
        m.to(devices.cpu)
      File "e:\SD\automatic1111\venv\lib\site-packages\lightning_fabric\utilities\device_dtype_mixin.py", line 54, in to
        return super().to(*args, **kwargs)
      File "e:\SD\automatic1111\venv\lib\site-packages\torch\nn\modules\module.py", line 1145, in to
        return self._apply(convert)
      File "e:\SD\automatic1111\venv\lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
        module._apply(fn)
      File "e:\SD\automatic1111\venv\lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
        module._apply(fn)
      File "e:\SD\automatic1111\venv\lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
        module._apply(fn)
      [Previous line repeated 1 more time]
      File "e:\SD\automatic1111\venv\lib\site-packages\torch\nn\modules\module.py", line 820, in _apply
        param_applied = fn(param)
      File "e:\SD\automatic1111\venv\lib\site-packages\torch\nn\modules\module.py", line 1143, in convert
        return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
    NotImplementedError: Cannot copy out of meta tensor; no data!

---
*** Error completing request
*** Arguments: ('task(26bs7tv83v5lryc)', 'A real life analog photo of a provocative adult woman,\nred lips, camisole, low leg jeans shorts, boobs drop, side boobs, groin,\njewelry,\ndetailed skin, skin pores, (freckles:0.6), (moles:0.6), (pigmentation:0.5),\ncatwalking on a street at night,\n[[Gabrielle Union | Laura Vandervoort]:0.1],\nphotography achievement, depth of field, film grain', '(worst quality:1.3), (bad quality:1.2), (low quality:1.1),\n[deformed | disfigured], poorly drawn, [bad : wrong] anatomy, [extra | missing | floating | disconnected] limb, (mutated hands and fingers)', [], 32, 'DPM++ 3M SDE Karras', 1, 1, 7, 768, 512, False, 0.3, 2, '4x_NMKD-Siax_200k', 24, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x0000029F347CFBB0>, 0, False, '', 0.8, 1518790282, False, -1, 0, 0, 0, False, False, {'ad_model': 'face_yolov8n.pt', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 32, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 32, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, False, 7, 100, 'Constant', 0, 'Constant', 0, 4, True, 'MEAN', 'AD', 1, False, 1, 1, 0, 0, -1, 0, 0, 0, 0, 0, True, -1, 1, 0, '1,1', 'Horizontal', '', 2, 1, True, 'Photo\\nextphoto_v30.safetensors [1c1f913f3b]', 'v1-5-pruned-emaonly.safetensors [6ce0161689]', 4, '1,1,1,0,0,0,0,0', {'calcmodes': ('Rebasin', 'Normal', 'Normal', 'Normal'), 'save_settings': ['fp16', 'prune', 'safetensors'], 'calc_settings': ['GPU']}, True, False, False, False, 'Photo\\rundiffusionFX_v10.safetensors [ad1a10552b]', 'None', 'None', 'None', 'Add-Diff', 'Sum', 'Sum', 'Sum', 0.75, 0.5, 0.5, 0.5, False, True, True, True, [], [], [], [], [], [], [], [], '0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', '0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', '0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', '0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', False, False, False, False, '', '', '', '', False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "E:\SD\automatic1111\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "E:\SD\automatic1111\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "E:\SD\automatic1111\modules\processing.py", line 721, in process_images
        sd_models.reload_model_weights()
      File "E:\SD\automatic1111\modules\sd_models.py", line 764, in reload_model_weights
        sd_model = reuse_model_from_already_loaded(sd_model, checkpoint_info, timer)
      File "E:\SD\automatic1111\modules\sd_models.py", line 713, in reuse_model_from_already_loaded
        send_model_to_cpu(sd_model)
      File "E:\SD\automatic1111\modules\sd_models.py", line 576, in send_model_to_cpu
        m.to(devices.cpu)
      File "e:\SD\automatic1111\venv\lib\site-packages\lightning_fabric\utilities\device_dtype_mixin.py", line 54, in to
        return super().to(*args, **kwargs)
      File "e:\SD\automatic1111\venv\lib\site-packages\torch\nn\modules\module.py", line 1145, in to
        return self._apply(convert)
      File "e:\SD\automatic1111\venv\lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
        module._apply(fn)
      File "e:\SD\automatic1111\venv\lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
        module._apply(fn)
      File "e:\SD\automatic1111\venv\lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
        module._apply(fn)
      [Previous line repeated 1 more time]
      File "e:\SD\automatic1111\venv\lib\site-packages\torch\nn\modules\module.py", line 820, in _apply
        param_applied = fn(param)
      File "e:\SD\automatic1111\venv\lib\site-packages\torch\nn\modules\module.py", line 1143, in convert
        return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
    NotImplementedError: Cannot copy out of meta tensor; no data!

---

The text was updated successfully, but these errors were encountered:

wkpark · 2023-12-26T11:29:43Z

Thank you for your reporting,
I have the same issue with the recent 1.7.x sd-webui (currently, I'm using sd-webui dev branch)

Please check some settings "Settings"->"Optimization" and turn off "FP8" optimization (Enable to Disable)

miasik · 2023-12-26T11:49:36Z

Thank you for your reporting, I have the same issue with the recent 1.7.x sd-webui (currently, I'm using sd-webui dev branch)

Please check some settings "Settings"->"Optimization" and turn off "FP8" optimization (Enable to Disable)

I don't see/find such an option

wkpark · 2023-12-26T12:26:57Z

I guess this is a similar issue. #29

miasik · 2023-12-26T12:52:52Z

I guess this is a similar issue. #29
Should I add more logs?

miasik · 2023-12-27T19:17:22Z

It looks working now for Sum mode, but Add-Diff/Rebasing still does not.
It shows its progress and a1111 renders a picture in this mode thу picture never changes. If I change mode the picture follows corresponding merges but it freezes again if I switch back to Add-Diff/Rebasing.
Also, what Rebasing and Fast Rb are?

debugs =  ['elemental merge', 'merge', 'adjust']█████████████████                                            | 32/64 [00:08<00:08,  3.85it/s]
use_extra_elements =  True
 - mm_max_models =  4
config hash =  73453409346ffcb55fca326b0e055e898aef8cacd938fcbc304fd6e4a1143e13
  - mm_use [True, False, False, False]
  - model_a Cartoons\fluffity_v21.safetensors [68536065c9]
  - base_model v1-5-pruned-emaonly.safetensors [6ce0161689]
  - max_models 4
  - models ['Photo\\epicphotogasm_lastUnicorn.safetensors [62bb78983a]']
  - modes ['Add-Diff']
  - calcmodes ['Rebasin']
  - usembws [[]]
  - weights ['0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5']
  - alpha [0.25]
  - adjust 1,1,1,0,0,0,0,0
  - use elemental [False]
  - elementals ['']
  - Parse elemental merge...
model_a = Cartoons_fluffity_v21
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Cartoons\fluffity_v21.safetensors...
isxl = False
compact_mode =  False
 - check possible UNet partial update...
 - partial changed blocks =  ['BASE', 'IN00', 'IN01', 'IN02', 'IN03', 'IN04', 'IN05', 'IN06', 'IN07', 'IN08', 'IN09', 'IN10', 'IN11', 'M00', 'OUT00', 'OUT01', 'OUT02', 'OUT03', 'OUT04', 'OUT05', 'OUT06', 'OUT07', 'OUT08', 'OUT09', 'OUT10', 'OUT11']
 - UNet partial update mode
 - Dynamic loading rebasin module...
Rebasin mode
 - Calulation device for Rebasin is  cuda
 - LAP library is lap
Loading model Photo_epicphotogasm_lastUnicorn...
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Photo\epicphotogasm_lastUnicorn.safetensors...
mode = Add-Diff, alpha = 0.25
Stage #1/2: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 876/876 [00:03<00:00, 237.75it/s]
Check uninitialized #2/2: 100%|████████████████████████████████████████████████████████████████████████| 876/876 [00:00<00:00, 146382.88it/s]
Rebasin calc...
maximize weight matching using scipy linear_sum_assignment...
P_bg324: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 12.03it/s]
P_bg324: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 11.53it/s]
P_bg324: new - old = 0.0
weight order changed layers = ['P_bg358', 'P_bg371', 'P_bg324']
*** Error running before_process: E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\scripts.py", line 710, in before_process
        script.before_process(p, *script_args)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 3482, in before_process
        theta_0 = apply_permutation(permutation_spec, first_permutation, theta_0)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 833, in apply_permutation
        return {k: get_permuted_param(ps, perm, k, params) for k in params.keys() if "model_" not in k}
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 833, in <dictcomp>
        return {k: get_permuted_param(ps, perm, k, params) for k in params.keys() if "model_" not in k}
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 773, in get_permuted_param
        for axis, p in enumerate(ps.axes_to_perm[k]):
    KeyError: 'cond_stage_model.logit_scale'

---
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:08<00:00,  3.66it/s]
Total progress:  50%|████████████████████████████████████████████                                            | 32/64 [00:08<00:08,  3.71it/s]
Total progress:  50%|████████████████████████████████████████████                                            | 32/64 [00:08<00:08,  3.89it/s]

wkpark · 2023-12-28T00:29:33Z

It looks working now for Sum mode, but Add-Diff/Rebasing still does not. It shows its progress and a1111 renders a picture in this mode thу picture never changes. If I change mode the picture follows corresponding merges but it freezes again if I switch back to Add-Diff/Rebasing. Also, what Rebasing and Fast Rb are?

debugs =  ['elemental merge', 'merge', 'adjust']█████████████████                                            | 32/64 [00:08<00:08,  3.85it/s]
use_extra_elements =  True
 - mm_max_models =  4
config hash =  73453409346ffcb55fca326b0e055e898aef8cacd938fcbc304fd6e4a1143e13
  - mm_use [True, False, False, False]
  - model_a Cartoons\fluffity_v21.safetensors [68536065c9]
  - base_model v1-5-pruned-emaonly.safetensors [6ce0161689]
  - max_models 4
  - models ['Photo\\epicphotogasm_lastUnicorn.safetensors [62bb78983a]']
  - modes ['Add-Diff']
  - calcmodes ['Rebasin']
  - usembws [[]]
  - weights ['0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5']
  - alpha [0.25]
  - adjust 1,1,1,0,0,0,0,0
  - use elemental [False]
  - elementals ['']
  - Parse elemental merge...
model_a = Cartoons_fluffity_v21
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Cartoons\fluffity_v21.safetensors...
isxl = False
compact_mode =  False
 - check possible UNet partial update...
 - partial changed blocks =  ['BASE', 'IN00', 'IN01', 'IN02', 'IN03', 'IN04', 'IN05', 'IN06', 'IN07', 'IN08', 'IN09', 'IN10', 'IN11', 'M00', 'OUT00', 'OUT01', 'OUT02', 'OUT03', 'OUT04', 'OUT05', 'OUT06', 'OUT07', 'OUT08', 'OUT09', 'OUT10', 'OUT11']
 - UNet partial update mode
 - Dynamic loading rebasin module...
Rebasin mode
 - Calulation device for Rebasin is  cuda
 - LAP library is lap
Loading model Photo_epicphotogasm_lastUnicorn...
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Photo\epicphotogasm_lastUnicorn.safetensors...
mode = Add-Diff, alpha = 0.25
Stage #1/2: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 876/876 [00:03<00:00, 237.75it/s]
Check uninitialized #2/2: 100%|████████████████████████████████████████████████████████████████████████| 876/876 [00:00<00:00, 146382.88it/s]
Rebasin calc...
maximize weight matching using scipy linear_sum_assignment...
P_bg324: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 12.03it/s]
P_bg324: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 11.53it/s]
P_bg324: new - old = 0.0
weight order changed layers = ['P_bg358', 'P_bg371', 'P_bg324']
*** Error running before_process: E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\scripts.py", line 710, in before_process
        script.before_process(p, *script_args)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 3482, in before_process
        theta_0 = apply_permutation(permutation_spec, first_permutation, theta_0)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 833, in apply_permutation
        return {k: get_permuted_param(ps, perm, k, params) for k in params.keys() if "model_" not in k}
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 833, in <dictcomp>
        return {k: get_permuted_param(ps, perm, k, params) for k in params.keys() if "model_" not in k}
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 773, in get_permuted_param
        for axis, p in enumerate(ps.axes_to_perm[k]):
    KeyError: 'cond_stage_model.logit_scale'

---
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:08<00:00,  3.66it/s]
Total progress:  50%|████████████████████████████████████████████                                            | 32/64 [00:08<00:08,  3.71it/s]
Total progress:  50%|████████████████████████████████████████████                                            | 32/64 [00:08<00:08,  3.89it/s]

Rebasin only support SD1.5 and does not support SD2.1 / SDXL currently.
cond_stage_model.logit_scale key is not for SD1.5

"Fast rebasin" only checks special weights groups.
(rebasin itself checks all permutation groups and sorts keys internally results possible permutations. but almost all orders of permutation groups are not changed (identical) except special groups. so Fast Rebasin will result identical results if it does not found special groups on merging blocks)

miasik · 2023-12-28T05:55:40Z

Rebasin only support SD1.5 and does not support SD2.1 / SDXL currently. cond_stage_model.logit_scale key is not for SD1.5

It couldn't be the reason, because both models are 1.5 — they were posted as 1.5 and they look fully functional as 1.5 models. Actually, I can merge them as 1.5 models by Sum mode.
Is there a way to just skip unexpected keys and continue merging?

wkpark · 2023-12-28T06:18:44Z

Rebasin only support SD1.5 and does not support SD2.1 / SDXL currently. cond_stage_model.logit_scale key is not for SD1.5

It couldn't be the reason, because both models are 1.5 — they were posted as 1.5 and they look fully functional as 1.5 models. Actually, I can merge them as 1.5 models by Sum mode. Is there a way to just skip unexpected keys and continue merging?

OK. I see, It needs to be checked the validity or acceptable keys in the weight matching routine.

wkpark · 2023-12-28T07:02:13Z

This issue has been automatically closed. Please reopen if this issue is not resolved

miasik · 2023-12-28T15:15:43Z

As I see — nothing has changed. Still getting the same error and the rendered picture doesn't chande

*** Error running before_process: E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\scripts.py", line 710, in before_process
        script.before_process(p, *script_args)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 3487, in before_process
        theta_0 = apply_permutation(permutation_spec, first_permutation, theta_0)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 839, in apply_permutation
        return {k: get_permuted_param(ps, perm, k, params) for k in params.keys() if _valid_key(k)}
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 839, in <dictcomp>
        return {k: get_permuted_param(ps, perm, k, params) for k in params.keys() if _valid_key(k)}
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 773, in get_permuted_param
        for axis, p in enumerate(ps.axes_to_perm[k]):
    KeyError: 'cond_stage_model.logit_scale'

* fixed mistake

wkpark · 2023-12-28T15:23:35Z

thanks for your reporting. I've fixed logic bug again

miasik · 2023-12-28T18:25:28Z

Nothing new again. Rebasin doesn't change the image

debugs =  ['elemental merge', 'merge', 'adjust']████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:12<00:00,  2.44it/s]
use_extra_elements =  True
 - mm_max_models =  4
config hash =  0abd1b6d1456626614caacd09c282b776869dcac07788f0289caa9e1a80dbfb5
  - mm_use [True, False, False, False]
  - model_a Cartoons\fluffity_v21.safetensors [68536065c9]
  - base_model Cartoons\fluffity_v21.safetensors [68536065c9]
  - max_models 4
  - models ['Photo\\epicphotogasm_lastUnicorn.safetensors [62bb78983a]']
  - modes ['Add-Diff']
  - calcmodes ['Rebasin']
  - usembws [[]]
  - weights ['0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5']
  - alpha [0.01]
  - adjust
  - use elemental [False]
  - elementals ['']
  - Parse elemental merge...
model_a = Cartoons_fluffity_v21
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Cartoons\fluffity_v21.safetensors...
isxl = False , sd2 = False
compact_mode =  False
 - check possible UNet partial update...
 - partial changed blocks =  ['BASE', 'IN00', 'IN01', 'IN02', 'IN03', 'IN04', 'IN05', 'IN06', 'IN07', 'IN08', 'IN09', 'IN10', 'IN11', 'M00', 'OUT00', 'OUT01', 'OUT02', 'OUT03', 'OUT04', 'OUT05', 'OUT06', 'OUT07', 'OUT08', 'OUT09', 'OUT10', 'OUT11']
 - UNet partial update mode
 - Dynamic loading rebasin module...
Rebasin mode
 - Calulation device for Rebasin is  cuda
 - LAP library is lap
Loading model Photo_epicphotogasm_lastUnicorn...
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Photo\epicphotogasm_lastUnicorn.safetensors...
mode = Add-Diff, alpha = 0.01
Stage #1/2: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 876/876 [00:13<00:00, 62.83it/s]
Check uninitialized #2/2: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 876/876 [00:00<00:00, 125476.75it/s]
Rebasin calc...
maximize weight matching using scipy linear_sum_assignment...
P_bg324: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 10.52it/s]
P_bg324: new - old = 0.0
weight order changed layers = ['P_bg358', 'P_bg371', 'P_bg324']
Save unchanged weights #2/2: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<?, ?it/s]
Clip is fine
 - merge processing in 17.5s (prepare: 0.5s, merging: 17.1s).
 - loading scripts.patches...
 - Fail to patch lora
*** Error running before_process: E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\scripts.py", line 710, in before_process
        script.before_process(p, *script_args)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 3684, in before_process
        base_dict[key] = theta_0[k]
    KeyError: 'cond_stage_model.logit_scale'

---
Using already loaded model Cartoons_fluffity_v21 + (Photo_epicphotogasm_lastUnicorn - v1-5-pruned-emaonly.safetensors [6ce0161689]) x alpha_0(0.25).safetensors [955683620d]: done in 0.5s (send model to device: 0.5s)
Using already loaded model Cartoons_fluffity_v21 + (Photo_epicphotogasm_lastUnicorn - v1-5-pruned-emaonly.safetensors [6ce0161689]) x alpha_0(0.25).safetensors [955683620d]: done in 0.0s
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:12<00:00,  2.37it/s]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:12<00:00,  2.37it/s]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:12<00:00,  2.43it/s]

* simple check acceptable keys in the partial update routine

wkpark · 2023-12-28T21:10:08Z

OK. this is another bug. rebasin works just fine. fixed another bug found at the partial update routine.

miasik · 2023-12-28T21:32:02Z

Ok, this time i noticed some minor changes and I cant get if it's OK
I set XYZ plot with [Model Mixer] alpha B for X and 0,0.25,0.5,0.75,1 as parameter
Then I got a grid but the changes almost invisible

New errors, one for each X value:

debugs =  ['elemental merge', 'merge', 'adjust']█▍                                                                                                                 | 30/150 [00:07<00:28,  4.26it/s]
use_extra_elements =  True
 - mm_max_models =  4
XYZ: alpha b -> 0.25
*** Error running before_process: E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\scripts.py", line 710, in before_process
        script.before_process(p, *script_args)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 2756, in before_process
        mm_finetune = ",".join([str(int(float(x))) if float(x) == int(float(x)) else str(x) for x in fines])
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 2756, in <listcomp>
        mm_finetune = ",".join([str(int(float(x))) if float(x) == int(float(x)) else str(x) for x in fines])
    ValueError: could not convert string to float: ''

* check finetune validity

wkpark · 2023-12-28T22:29:19Z

Ok, this time i noticed some minor changes and I cant get if it's OK I set XYZ plot with [Model Mixer] alpha B for X and 0,0.25,0.5,0.75,1 as parameter Then I got a grid but the changes almost invisible

New errors, one for each X value:

debugs =  ['elemental merge', 'merge', 'adjust']█▍                                                                                                                 | 30/150 [00:07<00:28,  4.26it/s]
use_extra_elements =  True
 - mm_max_models =  4
XYZ: alpha b -> 0.25
*** Error running before_process: E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\scripts.py", line 710, in before_process
        script.before_process(p, *script_args)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 2756, in before_process
        mm_finetune = ",".join([str(int(float(x))) if float(x) == int(float(x)) else str(x) for x in fines])
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 2756, in <listcomp>
        mm_finetune = ",".join([str(int(float(x))) if float(x) == int(float(x)) else str(x) for x in fines])
    ValueError: could not convert string to float: ''

thanks again. you just found another bug again.

and fixed now

miasik · 2023-12-28T22:49:35Z

Fantastic! ;-)
I see my new grid and it looks like we've finally done here :-D
Thank you so much!

Next I want to ensure that Rebasin works not worse than Train Difference. Because TD is amazing.

wkpark · 2023-12-28T22:52:33Z

Thank you for your patience and quick feedback!

wkpark self-assigned this Dec 26, 2023

wkpark added the bug Something isn't working label Dec 26, 2023

wkpark closed this as completed in f9f39fb Dec 28, 2023

miasik mentioned this issue Dec 28, 2023

Continuation for #95 #97

Closed

wkpark added a commit that referenced this issue Dec 28, 2023

fix #95

13e4b15

* fixed mistake

wkpark reopened this Dec 28, 2023

wkpark added a commit that referenced this issue Dec 28, 2023

fix another bug found at #95

ef50c40

* simple check acceptable keys in the partial update routine

wkpark added a commit that referenced this issue Dec 28, 2023

fixed XYZ support (reported at issue #95 by @miasik )

040ca7c

* check finetune validity

miasik closed this as completed Dec 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

An error during merging #95

An error during merging #95

miasik commented Dec 26, 2023

wkpark commented Dec 26, 2023

miasik commented Dec 26, 2023

wkpark commented Dec 26, 2023

miasik commented Dec 26, 2023

miasik commented Dec 27, 2023 •

edited

Loading

wkpark commented Dec 28, 2023 •

edited

Loading

miasik commented Dec 28, 2023

wkpark commented Dec 28, 2023 •

edited

Loading

wkpark commented Dec 28, 2023

miasik commented Dec 28, 2023

wkpark commented Dec 28, 2023

miasik commented Dec 28, 2023

wkpark commented Dec 28, 2023

miasik commented Dec 28, 2023 •

edited

Loading

wkpark commented Dec 28, 2023

miasik commented Dec 28, 2023

wkpark commented Dec 28, 2023

An error during merging #95

An error during merging #95

Comments

miasik commented Dec 26, 2023

wkpark commented Dec 26, 2023

miasik commented Dec 26, 2023

wkpark commented Dec 26, 2023

miasik commented Dec 26, 2023

miasik commented Dec 27, 2023 • edited Loading

wkpark commented Dec 28, 2023 • edited Loading

miasik commented Dec 28, 2023

wkpark commented Dec 28, 2023 • edited Loading

wkpark commented Dec 28, 2023

miasik commented Dec 28, 2023

wkpark commented Dec 28, 2023

miasik commented Dec 28, 2023

wkpark commented Dec 28, 2023

miasik commented Dec 28, 2023 • edited Loading

wkpark commented Dec 28, 2023

miasik commented Dec 28, 2023

wkpark commented Dec 28, 2023

miasik commented Dec 27, 2023 •

edited

Loading

wkpark commented Dec 28, 2023 •

edited

Loading

wkpark commented Dec 28, 2023 •

edited

Loading

miasik commented Dec 28, 2023 •

edited

Loading