Wrong resolution used for postprocessing shaders with non standard buffer size

[LunaMoo]

Sorry for lenghty name, I have no clue how to call that:]

Whenever buffer size is standard 480x272, postprocessing shaders that use output resolution(OutputResolution=True) behave correctly, however if game uses different buffer size for example 512x272, they are using wrong resolution. Either they use the 512x272 while game is scaled to 480x272 or opposite, either way it messes up the results quite a bit even if the difference is very small.

Not sure if this is new, most postprocessing shaders we have aren't really soo sensitive. Edit: Tested some ancient versions - definitely not new.
Easy to reproduce with any of those http://www.mediafire.com/download/y155wo45ctff91k/5xBRa.7z shaders.
In Monster Hunter portable 3rd this is actually causing screen shaking inside main menu because every second frame has different buffer size(481x273 / 480x272).

Guess it's because

ppsspp/GPU/GLES/Framebuffer.cpp

Line 216 in c2f4fad

CenterRect(&x, &y, &w, &h, 480.0f, 272.0f, (float)pixelWidth_, (float)pixelHeight_, ROTATION_LOCKED_HORIZONTAL);

480x272 is hardcoded instead detected? But should games even have buffer size above that? O.o
^ or maybe not - setting it to render or pixel res didn't do anything
.
Definitely not a fix, but maybe usefull info - I tried to comment out

ppsspp/GPU/Common/FramebufferCommon.cpp

Line 291 in d3b265a

vfb->width = std::max((int)vfb->width, drawing_width);

and following line and it got correct resolution, however it also stopped refreshing framebuffer size, so if a game changes it, it was broken again.;p

[hrydgard]

Hm, interesting. I think we should take care to only texture from the top left 480x272 of the buffers. It's OK although a little weird for games to use bigger ones, but only the top left 480x272 should be displayed so that would take care of that.

[hrydgard]

The problem is that the texelDelta uniform becomes wrong and must be updated. I will fix it later today.

[LunaMoo]

xBR seems to be completely happy with this thanks. Some methods of doing scanlines can still be affected by this issue through, I guess you saw it on the forum already.:)

For testing you can try this simple shader:
http://www.mediafire.com/download/1ecywpsqgh1pmh9/test+scanline+shader.7z
With quite probably any monster hunter game where that shader only works properly in FMV's, get's really broken in menus, sometimes it flickers, sometimes not, but there's a noticeable squash of pixels here and there.
Here's a demo where it can be reproduced: http://www.mediafire.com/download/6rfcc25f2vnm9re/MHP_3rd_Demo.7z.

[hrydgard]

Ah. The issue is that you can't use texcoord0 for anything else than sampling the texture - it won't match up with pixeldelta in any way when OutputResolution is set, as in the case where the framebuffer isn't 480x272, we rescale the texcoord to compensate for the fact that we only use a part of the texture.

Not sure what the best way to solve this is... Maybe we need to send in another coordinate that really is the output screen position (for platforms where there isn't already gl_fragCoord).

Hm never mind ... there are a bunch of rounding issues of various kinds going on. I'm having a look.

[hrydgard]

Argh. I was able to get the pixel delta correct so no more moiré effects, but because of the way the image is placed differently within the larger framebuffer if larger (we render things "flipped" - the image is not at 0.0 by open gl coordinates), the V coordinate will always jump so scanlines based on it will flicker.

We need to supply a second set of coordinates that are strictly screen aligned and not in texture space, or we need to flip rendering around when rendering to framebuffers to the OpenGL way as we have been thinking of for a while to get rid of a lot of (1.0 - y) etc (#8130). Currently not solvable any other way.

[LunaMoo]

Maybe this issue should be reopened to not forget about it? Or you're actually working on it now?:)

[hrydgard]

That should take care of it!

[LunaMoo]

Yup:) works great here. Ninja edit ~ thanks.:)