Separate checkpoints to 3 category. Trained, dreambooth and mix

[Biaru]

Today, an incredibly large number of mixes are produced and it is difficult to find original highly trained models in this database. Mixes should definitely be assigned to a separate category/subcategory. Dreambooth models are narrowly focused, they should also have been assigned to a separate category.

[JustMaier]

I think the distinction between "Trained" and "Dreambooth" isn't very clear. So maybe this could just be a checkbox for "This is a merge" or something like that. Thoughts?

[eadnams22]

Even that would be so nice to have!

[Biaru]

I was referring to models trained by the standard method on thousands of images using huge computing power. This is the only way to preserve the flexibility and diversity of the model.

The Dreambooth method is designed to fine-tune the model to a specific concept while using as little incoming data as possible. During this adjustment, the entire model is distorted, so if the model was trained on the face of a certain person, then as a result of training, all people generated will look somewhat like person on trained images. Even things unrelated to the subject of training, such as composition, light, clothing and background, will have a predisposition to those that were on the trained images.
If you train a dreambooth model in a certain style, then all generations will lean towards this style, even regardless of the usage of token keyword. So inevitably, models fine-tuned with Dreambooth become much more narrowly focused and biased.

Proof: https://i.redd.it/8z29vre37w5a1.jpg

Maybe the problem lies in the current Dreambooth paper implementation for Stable Diffusion which is based on code for textual inversion while original Dreambooth has no source code and is based on Imagen txt2img model.

[eadnams22]

Was just testing this today with subjects (not styles) using the A111 webUI Dreambooth extension.

I found that prior preservation (with proper training) of Dreambooth models has come a long way recently, there is a lot less (or even no noticeable) class-distortion when trained correctly, even if you don’t use class based images for prior-preservation, because it’s not just using those images anymore for preservation (which wasn’t a great method anyway).

Guessing that is because of the better text encoder (tenc) training methods, and extracting weights better.

[eadnams22]

Perhaps, this is a symptom of an underlying issue, there's so many merges that are often the same handful of models, with slightly different mixes, and we can't filter or easily see if they are similar or related. Which ties in to my pie-in-the-sky suggestion: #136 , which, going forward could identify similar merges - "Siblings" or "Cousins" (depending on their sources) vs fresher merges, or actual fine-tuning/training of new content.

[Biaru]

This is very difficult to implement. Considering how many users either do not record the exact recipe of the mix, or, for some reason, do not share it.

[JustMaier]

It would definitely just need to be a "best effort" to remember on the part of the merge creator. Eventually I'd like to create a merge tool that will keep history for the merge creator, but that's probably still a ways off.

[eadnams22]

Totally would be a helluva undertaking.

I picture it having a list (like the search) that auto-fills and allows the author to add ‘parents’ and the %, and method (diff, add, other), so instead of them listing it in the description, it’s a module they just fill in when they upload/publish.

Then a user can navigate by that genealogy to easily find trigger words.

or better yet, it aggregates the trigger words (if there are any), and auto-populates from the parents.

Feels kind of like revisions, or commits