Add 3x upscale option on the Web UI

[glonlas]

Add the "3x" upscale option on the Web UI.

[blessedcoolant]

Why? Real-ESRGAN doesn't support 3x scaling. The models are either 2x or 4x and hence the options. And this PR doesn't seem to make any changes beyond adding the UI element.

If you want to make this functional, then I suggest performing the 4x scaling and then scaling down to 3x but you need to add that functionality too.

[glonlas]

Real-ESRGAN supports x3. You can run it and get x2, x3 or x4.
This PR echoes the PR we made to fix Real-ESRGAN on M1

Try:
superman dancing with a panda bear -U 3 0.6 -G 0.8 -m ddim -s 8 -S 254908250

[blessedcoolant]

Real-ESRGAN supports x3. You can run it and get x2, x3 or x4. This PR echoes the PR we made to fix Real-ESRGAN on M1

Try: superman dancing with a panda bear -U 3 0.6 -G 0.8 -m ddim -s 8 -S 254908250

What output does it produce? Because that PR just builds on my implementation and I don't think there's any model defined for 3x upscaling in the changes.

[glonlas]

You are right this PR only adds the value in the dropdown (value="3") because the backend already supports custom values. In details, the value of this dropdown is sent to the backend then propagated to real_esrgan_upscale(image, strength, upsampler_scale, seed):.

upsampler_scale parameter is the upscale int value.
It works the same via the command line when -U parameter is used.

[glonlas]

What output does it produce? Because that PR just builds on my implementation and I don't think there's any model defined for 3x upscaling in the changes.

I see I did not answer your question at all. I do not know how Real-ESRGAN do it but if you do not pass any model as param (-n) it handles well the x3 value.

For instance, a 512x512 input will generate a 1536x1536. No visible image degradation.

But if you start passing a model it will start to have generation issues as it was reported here.

[blessedcoolant]

upsampler_scale parameter is the upscale int value. It works the same via the command line when -U parameter is used.

I understand that mate. I wrote that code.

I see I did not answer your question at all. I do not know how Real-ESRGAN do it but if you do not pass any model as param (-n) it handles well the x3 value.

For instance, a 512x512 input will generate a 1536x1536. No visible image degradation.

But if you start passing a model it will start to have generation issues as it was reported here.

When you don't pass a model, the value is defaulted to 2. And because the upsampling factor is also used to calculate the output width and height, it produces a 3x sized image but the upsampling is performed on 2. The rest of the scaling is just the image being enlarged. The reason you do not see any visible degradation is because in most cases it's harder to spot that in a 50% manual upscale. You cannot perform a Real-ESRGAN upscale without supplying a model architecture.

As I said, there's no 3x upscaling from Real-ESRGAN. If you wish to add 3x upscaling, then I'd recommend you do the 4x first and then manually downscale rather than what happens now. That'll end up producing sharper results. But honestly, I don't think it's a huge necessity.

[glonlas]

Example with:

• Prompt: "a portrait of Vincent van Gogh"
• Sampler: DDIM
• Steps: 6
• CFG: 7.5
• Size: 512x512
• Seed: 441538230

No upscale

2x

3x

4x

[blessedcoolant]

As you can see, the 4x model has far greater smoothing than the 2x model does but there's no actual pixel difference between the 2x and 3x. The 3x image is just the 2x image enlarged. So it'd be incorrect to say it's being 3x scaled using ESRGAN. It's 2x and then manually enlarged to 3x resolution.

[glonlas]

As I said, there's no 3x upscaling from Real-ESRGAN. If you wish to add 3x upscaling, then I'd recommend you do the 4x first and then manually downscale rather than what happens now. That'll end up producing sharper results. But honestly, I don't think it's a huge necessity.

Got it, it is clearer now.

In this case let me close this change to avoid confusion on this UI.

[Any-Winter-4079]

@blessedcoolant I don't know how the scale value is taken from the UI, but if you're simply reading the value from the active option in HTML, and passing it to realesrgan as -s, then it doesn't matter whether there are 2 or 3 options to select from (meaning this UI change won't break anything). And I find adding 3x convenient.

While there may be no 'true' 3x, if you run ./realesrgan-ncnn-vulkan -h
Usage: realesrgan-ncnn-vulkan -i infile -o outfile [options]...

-h                   show this help
  -i input-path        input image path (jpg/png/webp) or directory
  -o output-path       output image path (jpg/png/webp) or directory
  -s scale             upscale ratio (can be 2, 3, 4. default=4)
  -t tile-size         tile size (>=32/0=auto, default=0) can be 0,0,0 for multi-gpu
  -m model-path        folder path to the pre-trained models. default=models
  -n model-name        model name (default=realesr-animevideov3, can be realesr-animevideov3 | realesrgan-x4plus | realesrgan-x4plus-anime | realesrnet-x4plus)
  -g gpu-id            gpu device to use (default=auto) can be 0,1,2 for multi-gpu
  -j load:proc:save    thread count for load/proc/save (default=1:2:2) can be 1:2,2,2:2 for multi-gpu
  -x                   enable tta mode
  -f format            output image format (jpg/png/webp, default=ext/png)
  -v                   verbose output

you can see how 3x is mentioned (so, they expect people to use it). I get that there is no model for 3x, but people may still want 1536x1536 (from 512x512), and choosing 3x from the UI is more convenient than choosing 2x or 4x and then resizing up or down. I find that alternative pretty messy and inconvenient :)

[blessedcoolant]

Agreed. No harm I guess even if its not true 3x. It'll give a result that users can be happy with. Reopened this PR.

[lstein]

Happy to merge. We should change the CLI to allow 3x as well, no?

[hipsterusername]

I might suggest adding a note that it's at a degraded quality, whether that's just in the readme/help or directly in the WebUI.

[lstein]

I'm getting the following error when I try 3X upscaling (either with CLI or WebGUI):

>> Error running RealESRGAN or GFPGAN. Your image was not upscaled.
'NoneType' object has no attribute 'enhance'

My GFPGAN directory is up to date with the github repository. 2X and 4X are working. What might be the issue?

[glonlas]

@lstein I confirm that on development I also have the issue 'NoneType' object has no attribute 'enhance' but for x2, x3, and even x4. It worked when we reviewed this PR (the images generated, I put as examples, were generated from this PR).

I am opening an issue to track this error and start to investigate the issue.