Problem with multisample postprocessing #1246
Description
This ticket is the result of the discussion in Discord and its conclusions.
There appears to be an inconsistence within JME postprocessing stack, related to multisampling, that, according to my preliminary findings, encompasses such problems as:
- Some of the stock shaders are not built to support multisampling (for example, FXAA), thus when multisampling is enabled, these shaders fail.
- The failure looks like replacing the screenbuffer texture with some randrom texture from the loaded ones, thus effectively breaking the rendering pipeline.
- There's no warning or error that notifies the user that a particular shader is not compatible with multisampling and cannot be used.
- There's no documentation notice about what shaders allow that usage or not.
- Unaware contributors make shaders that are incompatible with multisampling. That could have been avoided if the contribution guide included the relevant section for shaders on how to make them multisampling-compatible.
- The problem can be silently "eliminated" by preceeding the line of post effects of the affected
FilterPostProcessorwith one of the shaders that was by chance made compatible with multisampling. That is not a real solution, however, it's currently widely adopted among the community on a level of a belief that a lucky sequence of postprocessing effects will work and the user just needs to find it by randomly changing the order of the filters in the postprocessing stack until it works.
Reproducing the problem:
The simplest reproduction of the problem can be made by creating a FilterPostProcessor within any JME project (for example, TestPBRLighting or a barest project with a box and a sky sphere and some textures), setting the number of samples on application settings to, for example, 4, setting the number of samples on the FilterPostProcessor to the same amount, and assigning two filters to it:
FXAAFilterToneMapFilter
which will lead to the viewport screenbuffer to be replaced with the texture of the environment map:
while the reverse combination of the filters will seemingly work:
The issue is also reproduced and confirmed by Oxplay on his machine.
The statement of this issue implies that preceeding the FilterPostProcessor chain of effects with even a passthrough filter that simply does nothing except for treating the possibility of multisampling correctly does fix the symptom is confirmed.
This project https://github.com/noncom/jme-multisampling-stopgap contains such a proof
of a passthrough filter that can be used at the start of the chain with any number of following filters that don't support multisampling and eliminate the glitch.
Take note that that filter also requires
#extension GL_ARB_texture_multisample : enablewhich in turn requires specifying GLSL150 version requirement in its j3md.
However, such an "approach" is not a solution, since the problem clearly lies within the JME pipeline or FilterPostProcessor pipeline.
As of now I did not yet get into how the pipelines work inside so I don't know why exactly this "approach" fixes the texture glitch. My current guess is that such a filter simply converges the multiple texture samples into one, thus providing a sampler2D as its output, that is later nonchalantly used by the following filters. If this is true, then the following filters are already silently receiving the color data collapsed to a single sample and might not always be desirable? This must be a problem?
I am not sure how this could have been missed until now, but this looks like a huge gap in the solidity of documentation and/or rendreing/postprocessing stacks and requires some attention.