Nifty GUI doesn't display correctly after context restart

It seems JME's context restart feature is flawed because with the sample code below, the Nifty GUI refuses to display at all:

import com.jme3.app.SimpleApplication;
import com.jme3.material.Material;
import com.jme3.niftygui.NiftyJmeDisplay;
import com.jme3.scene.Geometry;
import com.jme3.scene.shape.Box;

import de.lessvoid.nifty.Nifty;
import de.lessvoid.nifty.builder.LayerBuilder;
import de.lessvoid.nifty.builder.PanelBuilder;
import de.lessvoid.nifty.builder.ScreenBuilder;
import de.lessvoid.nifty.controls.button.builder.ButtonBuilder;
import de.lessvoid.nifty.screen.Screen;
import de.lessvoid.nifty.screen.ScreenController;

public class NiftyJme3RestartTest extends SimpleApplication implements ScreenController {

    public static void main(String[] args) {
        new NiftyJme3RestartTest().start();
    }

    private NiftyJmeDisplay niftyDisplay;

    @Override
    public void simpleInitApp() {

        // this box here always renders
        Box b = new Box(1, 1, 1);
        Geometry geom = new Geometry("Box", b);
        Material mat = new Material(assetManager, "Common/MatDefs/Misc/Unshaded.j3md");
        mat.setTexture("ColorMap", assetManager.loadTexture("/com/jme3/app/Monkey.png"));
        geom.setMaterial(mat);
        rootNode.attachChild(geom);

        niftyDisplay = NiftyJmeDisplay.newNiftyJmeDisplay(assetManager, inputManager, audioRenderer, guiViewPort);

        Nifty nifty = niftyDisplay.getNifty();
        nifty.loadStyleFile("nifty-default-styles.xml");
        nifty.loadControlFile("nifty-default-controls.xml");

        ScreenController ctrl = this;

        new ScreenBuilder("start") {
            {
                controller(ctrl);
                layer(new LayerBuilder() {
                    {
                        childLayoutVertical();
                        panel(new PanelBuilder() {
                            {
                                childLayoutCenter();
                                width("100%");
                                height("50%");
                                backgroundColor("#ff0000");
                            }
                        });
                        control(new ButtonBuilder("RestartButton", "Restart Context") {
                            {
                                alignCenter();
                                valignCenter();
                                height("32px");
                                width("480px");
                                interactOnClick("restartContext()");
                            }
                        });
                    }
                });
            }
        }.build(nifty);

        guiViewPort.addProcessor(niftyDisplay);
        nifty.gotoScreen("start");

        flyCam.setDragToRotate(true);
    }

    @Override
    public void bind(Nifty nifty, Screen screen) {
    }

    @Override
    public void onStartScreen() {
    }

    @Override
    public void onEndScreen() {
    }

    public void restartContext() {
        // even without changing settings, stuff breaks!
        restart();
        // ...and re-adding the processor doesn't help at all
        guiViewPort.addProcessor(niftyDisplay);
    }

}

The code was tested with jMonkeyEngine 3.2.2-stable. When run, the output log is pretty standard - there aren't any exceptions thrown or anything. Also, it seems the flyCam 'drag to rotate' functionality breaks after restart as well, unable to... well, drag at all.

On a slightly more complicated GUI setup that I've made (with nested panels and a few more buttons written in XML) the screen became green after restarting the context.

Perhaps the problem can be related to this post?

I haven't explored the engine much, so I'm hoping somebody who knows what they're doing can investigate this issue.

[tonihele]

This seems to happen only with LWJGL 3, right? At least for me the LWJGL 2 seems to work just fine.

[tonihele]

With LWJGL 3 the keyboard and mouse listeners are also tied to the window. When the context is restarted, the window is re-created. This seems to break the keyboard and mouse listeners as they still refer the old window handle. That explains the problems you get with the flycam. This one is easy to fix.

For our game the restart results in the whole scene gone, both Nifty and the regular scene context. The Nifty is there still alive and kicking, but I just can't see it. Any buttons etc. seem to work.

[stephengold]

Pretty please may I see a pull request?

[tonihele]

I pretty please need help :) I was able to fix the keyboard & mouse issues so that it only slightly feels like a hack (re-init them to the new window). But the graphics.... I don't understand, I tried all kinds of stuff. I carefully compared the LWJGL 2 code from jME. And tried to add some renderer update code etc. what was missing. I tried to have the previous window still open and have a shared context between them... I guess the buffers are just wrong or something in the new window.

I just don't understand it. I don't have enough competence for this I'm afraid. I haven't yet give up on this but I recognize that I'm just taking stabs at it blindly. I can't grasp the actual problem.

One crappy workaround would be that we just change the full screen status and the resolution on LWJGL 3 restart. Since that is all you can do without actual context restart.

[stephengold]

Okay, I read "easy to fix" and assumed you had a complete fix. I'm a bit behind the learning curve on LWJGL 3 myself.

@Ali-RS you can assist @tonihele ?

[Ali-RS]

Sorry, unfortunately this is out of my knowledge :(
Also I have no experience with Nifty gui.

[tonihele]

Just to be clear, I don't think this is just related to just Nifty GUI. We just both happen to use it with the op. We have in app graphic settings made with Nifty GUI. But anyway, the example provided by op should be enough. I didn't actually tried it, but I assume the whole screen goes blank as in our game. That means also normal scene graph stuff just vanishes, right?

[tonihele]

Hmm, in the example the scene graph holds, just the Nifty and render statistics disappears.

[pspeed42]

In JME nifty uses its own viewport and stuff, I guess.

If Nifty is not involved then someone should be able to make a test case that illustrates the issue that doesn't use nifty. And if that non-nifty test case works fine then is probably useful information, too.

[tonihele]

What happens to our game, is probably just related to the complexity of our Nifty, it somehow hides(?) the regular viewport scene after the restart. But we can forget that for now.
Here is in pictures what happens with the test code:

Before:

After:

[tonihele]

The statsappstate is in GUI view port. If I remove the Nifty, the statsappstate still vanishes. So I can conclude that it is not only Nifty that is feeling bad. And having Nifty doesn't cause this.

[stephengold]

Regarding the interference with StatsAppState, see issue #801.

[tonihele]

I can try the solution provided in https://hub.jmonkeyengine.org/t/problem-with-gui-z-order-after-graphics-restart/40985/4. To me it sounds something that we perhaps need to do anyway for consistency. Although again, I don't understand it :) But lets see what it does.

[tonihele]

This https://hub.jmonkeyengine.org/t/problem-with-gui-z-order-after-graphics-restart/40985/4 seems to solve the #801. But this issue remains.

Well, I was able to get the Nifty GUI to show after the context restart by doing something like this in restartContext:

niftyDisplay.cleanup();
restart();
enqueue(this::setupNiftyDisplayInstance);

...where setupNiftyDisplayInstance contains all the code in the original example that sets up the GUI.

However, input does not seem to work: clicking the button again does nothing, nor does hovering over it change its style.