[Bug]: Model hashes begin to repeat

[Jonseed]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What happened?

I have four ckpt models that all show the very same hash. They are, of course, different files.

Steps to reproduce the problem

1. Put several models in the SD model folder
2. Reload models
3. Several of the models show the same hash

What should have happened?

They should have unique hashes.

Commit where the problem happens

d61f0de

What platforms do you use to access UI ?

Windows

What browsers do you use to access the UI ?

Microsoft Edge

Command Line Arguments

--xformers --autolaunch --gradio-img2img-tool color-sketch

Additional information, context and logs

How do you get the server to reassign hashes?

[zzhongzz]

I experience the same issue.
I think becuase it only calculates model hash only from the first 0x10000 bytes at offset 0x100000, as shown here.
So if two models have the same content in this region, they will show the same hash, which is quite common in merged checkpoints.

[Jonseed]

Hmm, yeah, that's probably the reason we're seeing duplicate hashes. It needs to read more of the ckpt file or calculate the hash differently to generate a unique hash.

[jn-jairo]

I made a PR #2459 about that, feel free to take a look.

[0xdevalias]

Potentially related:

• Solving issue of differents ckpt with same hash #2459
• [Feature Request]: Change the way the ckpt hash is defined #4478
• Solving issue of differents ckpt with same hash (CRC solution) #4546
• Not freeing RAM when changing between checkpoints #2180
• Model Hash #2468

[trojblue]

It's really a big issue here. literally 75% of my models have duplicate hashes rn

[Jonseed]

PR #4546 would fix it.

[0xdevalias]

PR #4546 would fix it.

As would this one, which came first:

• Solving issue of differents ckpt with same hash #2459

[jn-jairo]

Issue solved by a95f135