Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some models error with, " File does not contain tensor conditioner.embedders.0.transformer.text_model.embeddings.position_ids" #136

Closed
CCpt5 opened this issue Apr 11, 2024 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@CCpt5
Copy link

CCpt5 commented Apr 11, 2024

Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch().
Startup time: 16.3s (prepare environment: 0.9s, import torch: 4.3s, import gradio: 0.9s, setup paths: 1.0s, initialize shared: 0.2s, other imports: 0.5s, list SD models: 0.5s, load scripts: 2.7s, refresh VAE: 0.1s, create ui: 4.3s, gradio launch: 0.4s, app_started_callback: 0.6s).
SDXL\2024-04-11 - PXR Artstyle - Realvision 4 - 1.2e-5 - b2 - 20rep-step00002000.safetensors [e1302c57ff] found
SDXL\2024-03-16 - OT - Pxr artstyle (no captions now) - 3998-5-874.safetensors [87ada3497b] found
SDXL\2024-03-22 - Supermerge - PXR - Lightning - Training today - Topnotch - Captioned - Pixar - mboard.safetensors [d05ee96673] found
SDXL\2024-04-08 - cont - pxr artstyle Retry during nap - base - 50img 2e-5 -b3-step00000500.safetensors [9e456a1057] found
debugs = ['elemental merge']
use_extra_elements = True

  • mm_max_models = 14
    config hash = 21604a8be0dae915d7a7e4e5038fe20726fc1e142ea7065755dd29cf9b0ca8d1
  • mm_use [True, True, True, True, False, False, False, False, False, False, False, False, False, False]
  • model_a SDXL\2024-04-11 - PXR Artstyle - Realvision 4 - 1.2e-5 - b2 - 20rep-step00002000.safetensors [e1302c57ff]
  • base_model None
  • max_models 14
  • models ['SDXL\2024-04-08 - cont - pxr artstyle Retry during nap - base - 50img 2e-5 -b3-step00000500.safetensors [9e456a1057]', 'SDXL\2024-04-11 - PXR Artstyle - Realvision 4 - 1.2e-5 - b2 - 20rep-step00004000.safetensors', 'SDXL\2024-03-16 - OT - Pxr artstyle (no captions now) - 3998-5-874.safetensors [87ada3497b]', 'SDXL\2024-03-22 - Supermerge - PXR - Lightning - Training today - Topnotch - Captioned - Pixar - mboard.safetensors [d05ee96673]']
  • modes ['Sum', 'Sum', 'Sum', 'Sum']
  • calcmodes ['Normal', 'Normal', 'Normal', 'Normal']
  • usembws [[], [], [], []]
  • weights ['0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', '0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', '0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', '0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5']
  • alpha [0.5, 0.5, 0.5, 0.5]
  • adjust
  • use elemental [False, False, False, False]
  • elementals ['', '', '', '']
  • Parse elemental merge...
    model_a = SDXL_2024-04-11 - PXR Artstyle - Realvision 4 - 1.2e-5 - b2 - 20rep-step00002000
    Loading SDXL\2024-04-11 - PXR Artstyle - Realvision 4 - 1.2e-5 - b2 - 20rep-step00002000.safetensors [e1302c57ff] from loaded model...
  • base lora_patch
    Applying attention optimization: sdp-no-mem... done.
    isxl = True , sd2 = False
    compact_mode = False
  • check possible UNet partial update...
  • partial changed blocks = ['BASE', 'IN00', 'IN01', 'IN02', 'IN03', 'IN04', 'IN05', 'IN06', 'IN07', 'IN08', 'M00', 'OUT00', 'OUT01', 'OUT02', 'OUT03', 'OUT04', 'OUT05', 'OUT06', 'OUT07', 'OUT08']
  • UNet partial update mode
    Open state_dict from file e:\Stable Diffusion Checkpoints\SDXL\2024-04-08 - cont - pxr artstyle Retry during nap - base - 50img 2e-5 -b3-step00000500.safetensors...
    mode = Sum, alpha = 0.5
    Stage Issue where model in merged but not save leads to "Cannot copy out of meta tensor; no data!" - persists through restart - had to disable app to continue #1/5: 74%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 1676/2263 [00:02<00:00, 787.22it/s]
    *** Error running before_process: D:\stable-diffusion-webui\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
    File "D:\stable-diffusion-webui\modules\scripts.py", line 817, in before_process
    script.before_process(p, *script_args)
    File "D:\stable-diffusion-webui\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 3912, in before_process
    theta_1[key] = theta_1f.get_tensor(key)
    safetensors_rust.SafetensorError: File does not contain tensor conditioner.embedders.0.transformer.text_model.embeddings.position_ids
@CCpt5
Copy link
Author

CCpt5 commented Apr 11, 2024

Model used above that caused the error (then no merge happens but generation continues): https://drive.google.com/file/d/1k-qpjDuEaQkcPMXliVbz3fAPgnaTlXAB/view?usp=sharing

@CCpt5 CCpt5 changed the title Some models trained w/ OneTrainer error with, " File does not contain tensor conditioner.embedders.0.transformer.text_model.embeddings.position_ids" Some models error with, " File does not contain tensor conditioner.embedders.0.transformer.text_model.embeddings.position_ids" Apr 11, 2024
@CCpt5
Copy link
Author

CCpt5 commented Apr 11, 2024

Another model that triggers this error: https://drive.google.com/file/d/1Px5Na6dmm2VNlWbkdDqy4xU1WcIprIeQ/view?usp=sharing

Details

100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.09it/s]
Total progress: 25%|████████████████████████████████████████████████████████▊ | 35/140 [00:07<00:23, 4.43it/s]
debugs = ['elemental merge']████████████████████████████████████████████████▊ | 35/140 [00:07<00:19, 5.28it/s]
use_extra_elements = True

  • mm_max_models = 14
    config hash = 061678adaeeaaba28f5b80f5425e80828fe5229ac16c4a1a2d1d4e2539308c21
  • mm_use [False, True, True, True, False, False, False, False, False, False, False, False, False, False]
  • model_a SDXL\2024-04-11 - PXR Artstyle - Realvision 4 - 1.2e-5 - b2 - 20rep-step00002000.safetensors [e1302c57ff]
  • base_model None
  • max_models 14
  • models ['SDXL\2024-04-11 - PXR Artstyle - Realvision 4 - 1.2e-5 - b2 - 20rep-step00004000.safetensors', 'SDXL\2024-03-16 - OT - Pxr artstyle (no captions now) - 3998-5-874.safetensors [87ada3497b]', 'SDXL\2024-03-22 - Supermerge - PXR - Lightning - Training today - Topnotch - Captioned - Pixar - mboard.safetensors [d05ee96673]']
  • modes ['Sum', 'Sum', 'Sum']
  • calcmodes ['Normal', 'Normal', 'Normal']
  • usembws [[], [], []]
  • weights ['0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', '0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', '0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5']
  • alpha [0.5, 0.5, 0.5]
  • adjust
  • use elemental [False, False, False]
  • elementals ['', '', '']
  • Parse elemental merge...
    model_a = SDXL_2024-04-11 - PXR Artstyle - Realvision 4 - 1.2e-5 - b2 - 20rep-step00002000
    Loading SDXL\2024-04-11 - PXR Artstyle - Realvision 4 - 1.2e-5 - b2 - 20rep-step00002000.safetensors [e1302c57ff] from loaded model...
  • base lora_patch
    Applying attention optimization: sdp-no-mem... done.
    isxl = True , sd2 = False
    compact_mode = False
  • check possible UNet partial update...
  • partial changed blocks = ['BASE', 'IN00', 'IN01', 'IN02', 'IN03', 'IN04', 'IN05', 'IN06', 'IN07', 'IN08', 'M00', 'OUT00', 'OUT01', 'OUT02', 'OUT03', 'OUT04', 'OUT05', 'OUT06', 'OUT07', 'OUT08']
  • UNet partial update mode
    Open state_dict from file e:\Stable Diffusion Checkpoints\SDXL\2024-04-11 - PXR Artstyle - Realvision 4 - 1.2e-5 - b2 - 20rep-step00004000.safetensors...
    Calculating sha256 for e:\Stable Diffusion Checkpoints\SDXL\2024-04-11 - PXR Artstyle - Realvision 4 - 1.2e-5 - b2 - 20rep-step00004000.safetensors: 1d86bf6008f77271a1628339efacffff195dad774fd8a6f118eccfdffab0f1b0
    mode = Sum, alpha = 0.5
    Stage Issue where model in merged but not save leads to "Cannot copy out of meta tensor; no data!" - persists through restart - had to disable app to continue #1/4: 74%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ | 1676/2263 [00:02<00:00, 717.16it/s]
    *** Error running before_process: D:\stable-diffusion-webui\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
    File "D:\stable-diffusion-webui\modules\scripts.py", line 817, in before_process
    script.before_process(p, *script_args)
    File "D:\stable-diffusion-webui\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 3912, in before_process
    theta_1[key] = theta_1f.get_tensor(key)
    safetensors_rust.SafetensorError: File does not contain tensor conditioner.embedders.0.transformer.text_model.embeddings.position_ids

100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.26it/s]
Total progress: 33%|███████████████████████████████████████████████████████████████████████████▋ | 35/105 [00:07<00:15, 4.50it/s]
Total progress: 33%|███████████████████████████████████████████████████████████████████████████▋ | 35/105 [00:07<00:13, 5.30it/s]

@wkpark wkpark self-assigned this Apr 12, 2024
@wkpark wkpark added the bug Something isn't working label Apr 12, 2024
wkpark added a commit that referenced this issue Apr 13, 2024
….text_model.embeddings.position_ids" error with SDXL models

issue #136 reported by @CCpt5
@CCpt5
Copy link
Author

CCpt5 commented Apr 13, 2024

@wkpark Working great now - thanks for the fix!!

@CCpt5 CCpt5 closed this as completed Apr 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants