Tracking bug for issues with gbm-kms platform on NVIDIA

[Saviq]

NVIDIA is (slowly!) becoming more usable on gbm-kms, which is interesting to us for superior XWayland support.

There are still a few bugs (as of the NIDIA 545 drivers - currently the most recent drivers available in 24.04) that we either need to wait for fixes for or work around. Those are:

• gbm_surface_create fails with flags != 0 (need driver quirk, as other drivers (correctly) require GBM_BO_USE_SCANOUT for scanout buffers)
• gbm_surface_has_free_buffers spuriously returns false in some circumstances. This is not needed for normal operation; our use is only to pick up incorrect usage of the GBM API in a more debuggable way; could be quirked off #3296
• NVIDIA: eglQueryDmaBufModifiersEXT spuriously returns EGL_BAD_PARAMETER for a few formats #3278
• Our platform probing does not know that /dev/dri/card0 and /dev/dri/renderD128 refer to the same physical device. This results in both gbm-kms (bound to /dev/dri/renderD128) and eglstream-kms (bound to /dev/dri/card0) being loaded as rendering platforms

Original bug:

[2024-02-23 08:59:11.472038] <information> xwayland: Starting XWayland                                                                                                        
(WW) Option "-listen" for file descriptors is deprecated                                                                                                                      
Please use "-listenfd" instead.                                                                                                                                               
(WW) Option "-listen" for file descriptors is deprecated                                                                                                                      
Please use "-listenfd" instead.                                                                                                                                               
eglInitialize() failed                                                                                                                                                        
EGL setup failed, disabling glamor                                                                                                                                            
Failed to initialize glamor, falling back to sw

Saviq: Have a try with MIR_XWAYLAND_OPTION=-eglstream

morphis: the above solves the eglInitialize part, Failed to initialize glamor, falling back to sw is still in there

Ref.:

mir/src/platforms/eglstream-kms/server/platform.cpp

Lines 176 to 183 in d0a68ae

	mge::RenderingPlatform::RenderingPlatform(EGLDisplay dpy)
	: dpy{dpy},
	ctx{std::make_unique<BasicEGLContext>(dpy)}
	{
	// XWayland eglstream has always been kinda flaky, now it's somehow worse.
	// Disable it until we've had a chance to look at what's wrong.
	// setenv(mir_xwayland_option, "-eglstream", 1);
	}

• GLMark2Xwayland tests fail on Nvidia #3182

[Saviq]

RAOF: One of the reasons this was never a priority is that passing -eglstream there will (might) get you accelerated rendering for the X server, it won't get you acceleration for X11 clients.
Saviq: Wonder if GBM is the right direction here, then?
RAOF: Yes; that has 3D accel plumbed through to clients.
RAOF: And might even work!

[RAOF]

Problems with NVIDIA on gbm-kms so far:

• gbm_create_surface fails if you pass flags != 0 (at least USE_SCANOUT is necessary for other drivers 🤦‍♀️, and technically we should pass USE_RENDERING | USE_SCANOUT )
• gbm_surface_has_free_buffers is broken (probably?), spuriously returning false; commenting out the check gets GBMSurface-backed output on NVIDIA.
• eglQueryDmaBufModifiersEXT returns EGL_BAD_PARAMETER (spuriously? Or are we doing something wrong?)

[morphis]

I was actually able to figure out the problem why Xwayland fails to successfully run eglInitialize on Mir running with --platform-rendering-libs mir:egl-generic, my installation was missing libnvidia-egl-gbm1. That makes Xwayland successfully initialize glamor: (II) GLX: Initialized glamor GL provider for screen 0.

However X11 clients still fail to use OpenGL ES, e.g. glmark-es2 reports

Error: eglGetDisplay() failed with error: 0x3000
Error: eglGetDisplay() failed with error: 0x3000
Error: main: Could not initialize canvas

when forcing the NVIDIA EGL driver via __EGL_VENDOR_LIBRARY_FILENAMES=/usr/share/glvnd/egl_vendor.d/10_nvidia.json.

[Conan-Kudo]

FYI, Xwayland no longer has EGLStreams support: https://gitlab.freedesktop.org/xorg/xserver/-/merge_requests/1386