How to convert bin to safetensors when I reload the original safetensors

[JunyuanDeng]

I am now try to train the stage 2 with the provided vista.safetensors.

After training, I merged the partitioned checkpoints as pytorch_model.bin using zero_to_fp32.py and while I use bin_to_st.py to convert the resulting, an error occur:

Traceback (most recent call last):
File "/home/users/junyuan.deng/Programmes/Vista/bin_to_st.py", line 31, in
vista_bin[pretrain_k] = vista_bin[pretrain_k] + lora_weights
KeyError: '_forward_module.model.diffusion_model.input_blocks.1.1.transformer_blocks.0.attn1.to_q.weight'

What's this problem and how to solve it?

[JunyuanDeng]

Or is it possible to release the bin file after the training of first stage?

[shengcheng]

I had same issue.

[shengcheng]

I had same issue.

I find the issue. The code does not save the weights of the diffusion model untrained.

[ljb121002]

Same issue here. Seems that the saved checkpoint has no keys starting with '_forward_module.model.diffusion_model.input_blocks.' and including 'to_q' 'to_k' 'to_v'

[DYZhang09]

I had same issue.

I find the issue. The code does not save the weights of the diffusion model untrained.

I had a similar issue here. Do you know how to fix it?

[zhoujiawei3]

I had the same issue. @Little-Podi Hi, could you take a look at this issue?

[zhoujiawei3]

Just need to load the unsaved weight from the official offered checkpoint

[Yang-Jianzhang]

I had same issue.
I think the bin_to_st.py file have some bugs.

Vista/bin_to_st.py

Line 31 in 6ff19b8

vista_bin[pretrain_k] = vista_bin[pretrain_k] + lora_weights

The pytorch_model.bin file doesn't have keys like '_forward_module.model.diffusion_model.input_blocks.1.1.transformer_blocks.0.attn1.to_q.weight' or ...to_k.weight or ...to_v.weight
Can you tell us how to fix it? @Little-Podi