Steam overlay does not work with Gallium Nine

[popsUlfr]

Basically Gallium Nine handles the d3d9 calls directly so they won't go through OpenGL or Vulkan. Steam Linux only ships their gameoverlayrenderer with OpenGL and Vulkan support on Linux.

Possible implementations :

• Somehow intercept the dx9 present call and forward the rendered image to an OpenGL context to be swapped. Not sure how to implement this but this sounds like a performance hog which might negate any performance advantages gained using gallium nine.
• use Valve's dx9 overlay renderer used on windows and route calls to the steam api to the linux client (as it is done in the steamclient dlls in proton). This would not incur performance degradation since the overlay would also go though gallium nine.

Interesting resources :

• https://aixxe.net/2017/09/steam-overlay-rendering (https://github.com/aixxe/gameoverlayrenderer-imgui-dx9)

[rea987]

As far as I understand, Windows version of Steam Overlay uses gameoverlayrenderer.dll and/or gameoverlayrenderer64.dll which are most probably depend on Steam.dll, steamclient.dll and steamclient64.dll which are present in .../steamapps/compatdata/AppID/pfx/drive_c/Program Files (x86)/Steam directory. So, theoretically, it would be possible to have Steam Overlay with Gallium Nine if gameoverlayrenderer.dll and gameoverlayrenderer64.dll got extracted from Steam Windows client and somehow used instead of gameoverlayrenderer.so. As easy as it sound, I am certain that it would require extensive hacking and library linking...

[rea987]

Here are Steam Overlay related files that I found in ~/.wine/drive_c/Program Files (x86)/Steam:

• GameOverlayRenderer.dll
• GameOverlayRenderer64.dll
• GameOverlayUI.exe
• SteamOverlayVulkanLayer.dll
• SteamOverlayVulkanLayer.json
• SteamOverlayVulkanLayer64.dll
• SteamOverlayVulkanLayer64.json

So, it looks like Steam Overlay for Windows does or will support Vulkan which may make things interesting.

GameOverlayRenderer.log:
https://gist.github.com/rea987/34a1b2c8ecdb357d28b370c183ea3311

SteamOverlayVulkanLayer.json:

{
    "file_format_version" : "1.0.0",
    "layer" : {
        "name": "VK_LAYER_VALVE_steam_overlay",
        "type": "GLOBAL",
        "library_path": ".\\SteamOverlayVulkanLayer.dll",
        "api_version": "1.1.73",
        "implementation_version": "1",
        "description": "Steam Overlay Layer",
        "disable_environment": { "DISABLE_VK_LAYER_VALVE_steam_overlay_1": "1" },
		"enable_environment": { "ENABLE_VK_LAYER_VALVE_steam_overlay_1": "1" }
    }
}

SteamOverlayVulkanLayer64.json:

{
    "file_format_version" : "1.0.0",
    "layer" : {
        "name": "VK_LAYER_VALVE_steam_overlay",
        "type": "GLOBAL",
        "library_path": ".\\SteamOverlayVulkanLayer64.dll",
        "api_version": "1.1.73",
        "implementation_version": "1",
        "description": "Steam Overlay Layer",
        "disable_environment": { "DISABLE_VK_LAYER_VALVE_steam_overlay_1": "1" },
		"enable_environment": { "ENABLE_VK_LAYER_VALVE_steam_overlay_1": "1" }
    }
}

[rea987]

By the way, as of today, Steam dropped Windows XP and Vista support. Therefore, it is logical to expect them to move Steam Overlay to Vulkan and Direct3D 11.

https://support.steampowered.com/kb_article.php?ref=1558-AFCM-4577

[popsUlfr]

Well steam obviously needs a steamoverlay that supports directx 11, 12, opengl and vulkan on windows to be able to show it in game. :D

Doesn't Windows 10 actually has something internal that works like dxup by translating dx9, 10, 10.1 to 11 ? Valve wouldn't need any old d3d9 codepaths anymore in their overlay. Things look grim since the overlay isn't open source :(

[rea987]

No idea, to my best knowledge, D3D9 games still need DirectX 9 to be installed separately either by user or Steam. So, does Steam Overlay need to be same version of API to properly with a game? If that's the case, then Proton with Gallium Nine obviously cannot benefit from Steam Overlay Vulkan Layer. However, in that case, Valve cannot drop D3D9 support for Steam Overlay either.