2.0 to 2.0.1 issue

[Neosettler]

Greetings,

@lstein I've managed to generate a few hundreds images with 2.0 and comparing the result with 2.0.1. It seems that some prompts in 2.0 took far less steps to achieved a descent result. Is that related to the noise generation issue? To my surprise, one prompt didn't get close to good quality at 50 steps and took about a 100 steps to get to parity with 2.0... what's going on?

[db3000]

Likely related to #1086, where the Karras noise schedule used in 2.0 was reverted in 2.0.1 as it produced very noisy images at high step counts. See also #1083

@lstein is it worth implementing something like your suggestion in #1083 (comment) to have a threshold at which this switches off?

[db3000]

I wonder if there's a point of continuity that can be found between the noise schedules where the switch could be done. Can try generate images at each step count with both until you get basically the same image, and use that step count as a constant at which to switch over.

[michaelezra]

I noticed this issue ad hoc at 20 iterations using k_lms sampler. It did not occur with other samplers for me.

[lstein]

It makes sense to switch over from one noise schedule to the other at a certain threshold, but I'm not convinced that the issue with non-convergent images at high step counts with the Karras noise schedule isn't a bug in how Karras is implemented. Could @Neosettler share the prompt of the noisy first image they posted? The metadata seems to have been stripped during uploading.

[Neosettler]

Sent prompt on discord. Playing around with --save_intermediates, I'm noticing that image 50, at 50 steps, is not the same as image 50, at 100 steps. What am I missing?

[Neosettler]

Also, the intermediates last image have a lot of noise compare to the final result. I was under the impression that the intermediate images would have been usable but it seems like they aren't ready for prime time, what's the catch?

[lstein]

This is a property of the diffusers. They remain noisy for a long time and then coalesce near the end. Also, if you are only showing every 5th o,age, you may be missing the intermediate steps. In fact, when you run 100 steps, the algorithm stretches out the schedule it uses to add noise, so that it adds less noise to step 49/50 than step 49/100. Lincoln

…

On Mon, Oct 17, 2022 at 5:34 PM Neosettler ***@***.***> wrote: Also, the intermediates last image have a lot of noise compare to the final result. I was under the impression that the intermediate images would have been usable but it seems like they aren't ready for prime time, what's the catch? — Reply to this email directly, view it on GitHub <#1130 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA3EVID4GAEEOHCVM5QALDWDXA6HANCNFSM6AAAAAARGQWNZI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Neosettler]

Thank you for your comment lstein, is there a way to save intermediates as if every image was the final result? The idea would be to allow choosing what images looks best. As it is now, only the final image look decent... so the whole intermediates output has not much use from an artistic stand point.

[psychedelicious]

@damian0815 , a while ago you shared a branch that did what @Neosettler is asking for (sample images at every step), can you please link us to that? I think it'd be a cool feature, understanding that it dramatically increases processing time.

[Neosettler]

Totally, I'd be a big fan for that one psychedelicious!!! @lstein Would that require to start from step 0 for every iteration?

[damian0815]

@psychedelicious @Neosettler i have show-latent-steps which is way behind development - it seems in the meantime a --save-intermediates has made its way into development.

mine saves every latent step, and has the advantage of being super-fast because it's approximating RGB from the low-res latents directly, rather than doing a lengthy upscale (so for 512x512 output you get a 64x64 pixel image). probably it would be good to have both?

[psychedelicious]

We support showing both of these now in the web UI, at a user-chosen interval (I think it won't be in main until v2.2 release, can't remember). I'll close this but please let me know if this isn't what you were asking for @Neosettler .