converting of .safetensors not working

[GDani31]

i am trying to convert the v1-5-pruned-emaonly.safetensors but the file generated is not working.

convert.exe v1-5-pruned-emaonly.safetensors -t q4_0
loading model 'v1-5-pruned-emaonly.safetensors'
model type: checkpoint
Stable Diffusion 1.x - v1-5-pruned-emaonly.safetensors
preprocessing 0 tensors
using embedded vocab
converting 0 tensors
alphas_cumprod computed

CLIP Model Tensor count: 0
UNET Model Tensor count: 0
VAE Model Tensor count: 0

saving gguf file
model saved 'v1-5-pruned-emaonly-q4_0.gguf' correctly.

and then

sd.exe -m v1-5-pruned-emaonly-q4_0.gguf -p "anorange cat, realistic"
[INFO]  stable-diffusion.cpp:3715 - loading model from 'v1-5-pruned-emaonly-q4_0.gguf'
[INFO]  stable-diffusion.cpp:3743 - Stable Diffusion 1.x | v1-5-pruned-emaonly.safetensors
[INFO]  stable-diffusion.cpp:3751 - model data type: q4_0
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.embeddings.position_embedding.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.embeddings.token_embedding.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.layer_norm1.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.layer_norm1.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.layer_norm2.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.layer_norm2.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.mlp.fc1.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.mlp.fc1.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.mlp.fc2.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.mlp.fc2.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.self_attn.k_proj.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.self_attn.k_proj.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.self_attn.out_proj.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.self_attn.out_proj.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.self_attn.q_proj.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.self_attn.q_proj.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.self_attn.v_proj.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'cond_stage_model.transformer.text_model.encoder.layers.0.self_attn.v_proj.weight' not in model file

.
.
.

model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.1.transformer_blocks.0.attn2.to_out.0.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.1.transformer_blocks.0.attn2.to_q.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.1.transformer_blocks.0.attn2.to_v.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.1.transformer_blocks.0.ff.net.0.proj.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.1.transformer_blocks.0.ff.net.0.proj.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.1.transformer_blocks.0.ff.net.2.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.1.transformer_blocks.0.ff.net.2.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.1.transformer_blocks.0.norm1.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.1.transformer_blocks.0.norm1.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.1.transformer_blocks.0.norm2.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.1.transformer_blocks.0.norm2.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.1.transformer_blocks.0.norm3.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.1.transformer_blocks.0.norm3.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.2.conv.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.8.2.conv.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.0.emb_layers.1.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.0.emb_layers.1.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.0.in_layers.0.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.0.in_layers.0.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.0.in_layers.2.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.0.in_layers.2.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.0.out_layers.0.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.0.out_layers.0.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.0.out_layers.3.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.0.out_layers.3.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.0.skip_connection.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.0.skip_connection.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.norm.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.norm.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.proj_in.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.proj_in.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.proj_out.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.proj_out.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn1.to_k.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn1.to_out.0.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn1.to_out.0.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn1.to_q.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn1.to_v.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn2.to_k.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn2.to_out.0.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn2.to_out.0.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn2.to_q.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.attn2.to_v.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.ff.net.0.proj.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.ff.net.0.proj.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.ff.net.2.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.ff.net.2.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.norm1.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.norm1.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.norm2.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.norm2.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.norm3.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.output_blocks.9.1.transformer_blocks.0.norm3.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.time_embed.0.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.time_embed.0.weight' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.time_embed.2.bias' not in model file
[ERROR] stable-diffusion.cpp:3889 - tensor 'model.diffusion_model.time_embed.2.weight' not in model file

[FSSRepo]

It seems that I have already found the problem. In the last pull request, which is almost ready to merge, the converter bug is fixed.

[Cyberhan123]

It seems that I have already found the problem. In the last pull request, which is almost ready to merge, the converter bug is fixed.

@FSSRepo Thank you for your hard work， Can the conversion method be extracted separately as an API?

[FSSRepo]

@Cyberhan123 At the moment, the current version of the project is highly limited in exposing functions externally. Significant refactoring and testing would be required. Honestly, I don't currently have the necessary time.

[Mek101]

It seems that I have already found the problem. In the last pull request, which is almost ready to merge, the converter bug is fixed.

If by "last pull request", you mean #88, it compiles but it's still broken as the original report describes

[FSSRepo]

If by "last pull request", you mean #88, it compiles but it's still broken as the original report describes

Could you give me more details, model name, where you downloaded it, try to build it from scratch (clean cmake cache), also the program's output.

[leejet]

@GDani31 @Mek101 Now you can directly load model files from safetensors or ckpt, just try out the latest master code.

[GDani31]

@GDani31 @Mek101 Now you can directly load model files from safetensors or ckpt, just try out the latest master code.

thanks, currently testing it, it loads the .safetensors file.
i wish i could somehow help making the performance better, currently on my 1050ti it takes like 30min to generate an image while the python versions only takes 45sec - 1min.

[leejet]

@GDani31 @Mek101 Now you can directly load model files from safetensors or ckpt, just try out the latest master code.

thanks, currently testing it, it loads the .safetensors file. i wish i could somehow help making the performance better, currently on my 1050ti it takes like 30min to generate an image while the python versions only takes 45sec - 1min.

Have you used the cuda version of sd.cpp?

[GDani31]

@GDani31 @Mek101 Now you can directly load model files from safetensors or ckpt, just try out the latest master code.

thanks, currently testing it, it loads the .safetensors file. i wish i could somehow help making the performance better, currently on my 1050ti it takes like 30min to generate an image while the python versions only takes 45sec - 1min.

Have you used the cuda version of sd.cpp?

not yet, thanks for mentioning it. will try this out soon.

[Mek101]

@GDani31 @Mek101 Now you can directly load model files from safetensors or ckpt, just try out the latest master code.

The weights are StableDiffision v2 downloaded from the link provided in the README.
The prompt is the default prompt of https://github.com/EdVince/Stable-Diffusion-NCNN/blob/d7fb2edad6f76de4117cae3a4290bd11a11a3047/x86/linux/src/opencv-mobile_ncnn-demo.cpp#L31
I pulled the latest master code, cleaned the build/ directory and ran the following.

~/.../3party/stable-diffusion.cpp >>> cmake -DCMAKE_BUILD_TYPE=Release -GNinja -DSD_FLASH_ATTN=ON -S . -B build  && ninja -C build
-- The C compiler identification is GNU 13.2.1
-- The CXX compiler identification is GNU 13.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Use Flash Attention for memory optimization
CMake Deprecation Warning at ggml/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- Linux detected
-- Configuring done (0.7s)
-- Generating done (0.0s)
-- Build files have been written to: /home/mek101/Progetti/3party/stable-diffusion.cpp/build
ninja: Entering directory `build'
[5/12] Building C object thirdparty/CMakeFiles/zip.dir/zip.c.o
In file included from /home/mek101/Progetti/3party/stable-diffusion.cpp/thirdparty/zip.c:39:
/home/mek101/Progetti/3party/stable-diffusion.cpp/thirdparty/miniz.h:4988:9: note: ‘#pragma message: Using fopen, ftello, fseeko, stat() etc. path for file I/O - this path may not support large files.’
 4988 | #pragma message(                                                               \
      |         ^~~~~~~
[7/12] Building C object ggml/src/CMakeFiles/ggml.dir/ggml.c.o
/home/mek101/Progetti/3party/stable-diffusion.cpp/ggml/src/ggml.c:3155:6: warning: no previous prototype for ‘ggml_broadcast’ [-Wmissing-prototypes]
 3155 | void ggml_broadcast(
      |      ^~~~~~~~~~~~~~
/home/mek101/Progetti/3party/stable-diffusion.cpp/ggml/src/ggml.c: In function ‘ggml_compute_forward_pad_f32’:
/home/mek101/Progetti/3party/stable-diffusion.cpp/ggml/src/ggml.c:11953:11: warning: type defaults to ‘int’ in declaration of ‘so2’ [-Wimplicit-int]
11953 |     const so2 = ne00 * ne01;
      |           ^~~
/home/mek101/Progetti/3party/stable-diffusion.cpp/ggml/src/ggml.c:11954:11: warning: type defaults to ‘int’ in declaration of ‘so3’ [-Wimplicit-int]
11954 |     const so3 = ne00 * ne01 * ne02;
      |           ^~~
/home/mek101/Progetti/3party/stable-diffusion.cpp/ggml/src/ggml.c:11955:11: warning: type defaults to ‘int’ in declaration of ‘do2’ [-Wimplicit-int]
11955 |     const do2 = ne0 * ne1;
      |           ^~~
/home/mek101/Progetti/3party/stable-diffusion.cpp/ggml/src/ggml.c:11956:11: warning: type defaults to ‘int’ in declaration of ‘do3’ [-Wimplicit-int]
11956 |     const do3 = ne0 * ne1 * ne2;
      |           ^~~
/home/mek101/Progetti/3party/stable-diffusion.cpp/ggml/src/ggml.c:11948:15: warning: unused variable ‘padding_factor’ [-Wunused-variable]
11948 |     const int padding_factor = dst->op_params[0];
      |               ^~~~~~~~~~~~~~
/home/mek101/Progetti/3party/stable-diffusion.cpp/ggml/src/ggml.c: In function ‘gguf_write_to_file’:
/home/mek101/Progetti/3party/stable-diffusion.cpp/ggml/src/ggml.c:19127:28: warning: comparison of integer expressions of different signedness: ‘size_t’ {aka ‘long unsigned int’} and ‘int’ [-Wsign-compare]
19127 |             if (offset_pad != cur_offset) {
      |                            ^~
In function ‘ggml_op_name’,
    inlined from ‘ggml_get_n_tasks’ at /home/mek101/Progetti/3party/stable-diffusion.cpp/ggml/src/ggml.c:15750:17:
/home/mek101/Progetti/3party/stable-diffusion.cpp/ggml/src/ggml.c:2022:24: warning: array subscript 70 is above array bounds of ‘const char *[69]’ [-Warray-bounds=]
 2022 |     return GGML_OP_NAME[op];
      |            ~~~~~~~~~~~~^~~~
/home/mek101/Progetti/3party/stable-diffusion.cpp/ggml/src/ggml.c: In function ‘ggml_get_n_tasks’:
/home/mek101/Progetti/3party/stable-diffusion.cpp/ggml/src/ggml.c:1589:21: note: while referencing ‘GGML_OP_NAME’
 1589 | static const char * GGML_OP_NAME[GGML_OP_COUNT] = {
      |                     ^~~~~~~~~~~~
[12/12] Linking CXX executable bin/sd
~/.../3party/stable-diffusion.cpp >>> ./build/bin/sd -m v2-1_768-nonema-pruned.safetensors -p "floating hair, portrait, ((loli)), ((one girl)), cute face, hidden hands, asymmetrical bangs, beautiful detailed eyes, eye shaw, hair ornament, ribbons, bowties, buttons, pleated skirt, (((masterpiece))), ((best quality)), colorful"
[INFO]  stable-diffusion.cpp:3664 - Flash Attention enabled
[INFO]  stable-diffusion.cpp:3667 - loading model from 'v2-1_768-nonema-pruned.safetensors'
[INFO]  model.cpp:544  - load v2-1_768-nonema-pruned.safetensors using safetensors format
[ERROR] model.cpp:641  - invalid safetensor file 'v2-1_768-nonema-pruned.safetensors'
[ERROR] stable-diffusion.cpp:3671 - init model loader from file failed: 'v2-1_768-nonema-pruned.safetensors'
~/.../3party/stable-diffusion.cpp >>>

[leejet]

@Mek101 Could you pull the latest code, recompile, and give it a try? When running, could you add the -v parameter to check the output? If you don't mind, could you provide the md5sum of 'v2-1_768-nonema-pruned.safetensors'? I'd like to compare it with the file I have locally.

[Mek101]

~/.../3party/stable-diffusion.cpp >>> ./build/bin/sd -m v2-1_768-nonema-pruned.safetensors -p "floating hair, portrait, ((loli)), ((one girl)), cute face, hidden hands, asymmetrical bangs, beautiful detailed eyes, eye shaw, hair ornament, ribbons, bowties, buttons, pleated skirt, (((masterpiece))), ((best quality)), colorful"
[INFO]  stable-diffusion.cpp:3664 - Flash Attention enabled
[INFO]  stable-diffusion.cpp:3667 - loading model from 'v2-1_768-nonema-pruned.safetensors'
[WARN]  model.cpp:624  - unknown format v2-1_768-nonema-pruned.safetensors
[ERROR] stable-diffusion.cpp:3671 - init model loader from file failed: 'v2-1_768-nonema-pruned.safetensors'
~/.../3party/stable-diffusion.cpp >>> md5sum v2-1_768-nonema-pruned.safetensors                                                                                                                                                              
75a561e8855ec63ab01c22b11cee5ce5  v2-1_768-nonema-pruned.safetensors

[Mek101]

I tried the v1-5-pruned-emaonly.safetensors weights and they work as they should

[leejet]

It looks like you may have downloaded the wrong or incomplete file.

e43c2fb4baa9c30988a8d9e8ee644a33 v2-1_768-nonema-pruned.safetensors

[leejet]

This is probably because I gave the wrong url to the curl command for v2-1_768-nonema-pruned.safetensors in the documentation. It's fixed, you can try it again.

curl -L -O https://huggingface.co/stabilityai/stable-diffusion-2-1/resolve/main/v2-1_768-nonema-pruned.safetensors

[Amin456789]

hey guys, i tired to convert sdxl turbo fp16 to 5.1 gguf and got the same error as above mentioned [if i remember correctly] the model got converted but it gave me those errors after generating image, i need to do this because the model became 3gb only and it is very ram friendly this way instead of converting on fly in sd cpp