DeSmuME core freezes when creating savestate

[blueYOSHI9000]

Whenever I create a savestate with the DeSmuME core the visuals freeze, the game still plays in the background, I just can't see anything anymore. Resetting the game does not fix it, only restarting the emulator fixes it. When trying to load a different game in a different core the emulator simply crashes.
I tested multiple DS games, same happened in all of them, it did not happen on different cores though (tried ParaLLEl64 and Beetle PSX HW). It also didn't happen in previous versions.
video - log.txt

[Jamiras]

I could not reproduce locally. Thank you for both the video and the log file.

The only notable thing I see in the log file is this:

[INFO ] [SAV] Saving state to C:\Users\blueYOSHI\ROMs & ISOs\Emulators\RALibRetro\States\_Mario & Luigi - Bowser's Inside Story (E)-desmume_libretro.state1
[ERROR] [OGL] Error in Gl::getTexImage:
[ERROR] [OGL]   INVALID_OPERATION
[INFO ] [UTL] Pixel format is RGB565, converting to 24-bits RGB
[INFO ] [UTL] Wrote 12586576 bytes to "C:\Users\blueYOSHI\ROMs & ISOs\Emulators\RALibRetro\States\_Mario & Luigi - Bowser's Inside Story (E)-desmume_libretro.state1"

Same lines when I do it:

[INFO ] [SAV] Saving state to E:\Games\RetroAchievements\RALibRetro64\States\Mario & Luigi - Bowser's Inside Story (E)-desmume_libretro.state1
[INFO ] [UTL] Pixel format is RGB565, converting to 24-bits RGB
[INFO ] [UTL] Wrote 12594088 bytes to "E:\Games\RetroAchievements\RALibRetro64\States\Mario & Luigi - Bowser's Inside Story (E)-desmume_libretro.state1"

The new save file format embeds the image in the save file, but it shouldn't be required. Were you maybe having issues generating the image files previously and it just didn't matter? Do you have any ".png" files for save states from the earlier versions of RAlibretro?

If I modify the code to simulate a failure capturing the current screen, I don't see any problem there either. It's also strange that it seems to be specifically the DeSmuMe core.

I don't see anything else in 1.3.5 related to save states or the OpenGL APIs.

[blueYOSHI9000]

I just checked and the previous version generated .state, .state.png and .state.rap files for each savestate, I can also still load them without issues.
Another thing I noticed is that DeSmuME Screenshots also freeze the emulator, at first the screenshots saved properly but now it seems they don't. The only thing I remember doing is updating the DeSmuME core to 2021-04-26, I'm honestly not sure anymore how much of this only happened after updating.
Here's two screenshots for reference with the original timestamps (both using DeSmuME, first one before update, second one today - happens in multiple games): 2021-04-19T20-00-03Z -- 2021-04-28T10-12-33Z -- Another log for the screenshot thing (all I did was startup Emu, opened recent DS game, loaded state 1, screenshot and close emulator afterwards)

[Jamiras]

It looks like the same error when you take a screenshot.

[ERROR] [OGL] Error in Gl::getTexImage:
[ERROR] [OGL]   INVALID_OPERATION
[INFO ] [UTL] Pixel format is RGB565, converting to 24-bits RGB
[INFO ] [UTL] Wrote image 256 x 429 to C:\Users\blueYOSHI\ROMs & ISOs\Emulators\RALibRetro\Screenshots\2021-04-28T10-12-33Z.png

And since you attached that screenshot and it appears to be "blank", it's obviously not working. Unfortunately, I don't know what the error is indicating or how to address it. I still find it strange that it only affects the DeSmuMe core.

[thearchivalone]

So, theanxietybuster here. I ran into this issue recently with Kirby and found that disabling the Opengl Rasterizer solved the problem. It’s not exclusive to DeSuMuMe. MelonDS had the same issue (despite not being officially supported).

[Jamiras]

I can reproduce with the rasterizer enabled. Interestingly enough, it only happens when saving the state. After restarting, I can reload the state and continue playing just fine.

The documentation just says:

Enable OpenGL renderer.
The Frontend's video driver must be set to gl.

which provides very little insight as to why the problem occurs.

Some simple stepping over pieces of the code shows that it's this line that's causing the issue, but I don't have the OpenGL expertise to determine why.

RALibretro/src/components/Video.cpp

Line 333 in cdfb97c

Gl::getTexImage(GL_TEXTURE_2D, 0, GL_RGB, GL_UNSIGNED_SHORT_5_6_5, pixels);

It does seem to correlate to the error captured in the log file above:

[ERROR] [OGL] Error in Gl::getTexImage:
[ERROR] [OGL]   INVALID_OPERATION

[leiradel]

It's just a guess, but that code is changing the OpenGL state which may be confusing the core. The currently bound texture should be obtained before binding _texture, and then bound again before the function returns.

A quick search provided this code, which I haven't tested:

GLint current;
glGetIntegerv(GL_TEXTURE_BINDING_2D, &current);

[Jamiras]

That did not work. current is 0 after calling the function, which indicates there wasn't a 2D texture binding previously.

I think you're right though about our state change affecting whatever state is being managed internally. It could explain why the emulator starts strobing, but it doesn't explain why calling getTexImage is throwing an INVALID_OPERATION error. Maybe it's not possible to capture a 2D image when 3D rendering is occurring?

[leiradel]

That did not work. current is 0 after calling the function, which indicates there wasn't a 2D texture binding previously.

I see. I'd leave the code there anyway, maybe there are cores out there that will leave a texture bound and relying on that to function properly, even though I'm getting the feeling no core will rely on that since the frontend will, when retro_run returns, change the graphics state to draw the framebuffer on the screen.

I think you're right though about our state change affecting whatever state is being managed internally. It could explain why the emulator starts strobing, but it doesn't explain why calling getTexImage is throwing an INVALID_OPERATION error. Maybe it's not possible to capture a 2D image when 3D rendering is occurring?

I see that postHwRenderReset is called on every frame, maybe it should query the current state, set the state required by RALIbretro to draw the frame, and restore the previous state. draw also changes some OpenGL state. I think RetroArch can provide some clues whether the state must be preserved or not in between calls to retro_run. I don't think this anything to do with the issue though.

As for 2D vs 3D, OpenGL, and I belive this is true for every other graphics API, doesn't differentiate between 2D and 3D rendering. It transform vertices and map them to a 2D plane, onto which the textures will be blit etc.

Here the person says that this error can happen if the format is an integer one and the texture has a float point format, but I think RALibretro is the one which always create the texture and only creates textures with integer formats, so it's really weird. Again, maybe RetroArch can be a good source for a possible fix.