[d3d9] Implement a software cursor

[WinterSnowfall]

Probably not an entirely sound approach, but it's a start. I believe we'll actually need to draw the cursor onto the final image ourselves, before the HUD. Some processing on the cursor bitmap surface that a game provides also needs to be done, because it needs to be trimmed to the bounding dimensions of pixels that contain color data and then blended.

I'm fine if @K0bin wants to pick up and reuse some of what I've done here, since the d3d9_cursor.cpp/h additions should be sound.

As is, the implementation is fully functional and sets/shows/hides a software cursor properly. However, the cursor surface is very unceremoniously sent onto the final backbuffer using StretchRect(). I had sort of hoped that would simply work, but alas.

As always, Dungeon Siege 2 is the game to test this with. (Fixes #3020).

[WinterSnowfall]

So, thanks to some nudges in the right direction from @K0bin and @DadSchoorse, I managed to get the blending sorted by rendering the cursor texture to a quad and drawing that on the final render target.

This works just fine for our purposes, and seems rather fast and latency free, at least more than I expected it to be for a bunch of calculations done on each frame. Perhaps there's even room for a bit of optimization, to actually skip some of the coordinate math if the cursor position remains unchanged, but meh.

There's one more thing I actually need tackle though: apparently games expect us to trim/mask edge transparent pixels from the cursor surface they set (which is handled automagically by win32/hardware cursors). Right now, because of this unhandled behavior, the cursor quad dimensions and the cursor tip coordinates are not as/where they should be.

I'll be looking to address this during cursor surface memcpy with some bit-level magic, hopefully.

[K0bin]

I gotta say I'm not a fan that you implemented it using D3D9 APIs, especially the fixed function pipeline.

I would've used the underlying DxvkContext directly and wrote some simple GLSL shaders.

[WinterSnowfall]

I would've used the underlying DxvkContext directly and wrote some simple GLSL shaders.

That's a bit beyond what I can do. In any case, @doitsujin is considering facilitating something like setCursorBitmap and setCursorPosition for us from the backend eventually, so I wouldn't overthink it at this point.

But if you want, feel free to take it forward are rewrite the offending bits. I'm only working on properly processing the application provided bitmap now, which would be useful anyway.

[WinterSnowfall]

Alright, sorted the pre-processing/clipping of the cursor bitmap too, which is something the frontend needs to handle before storing the cursor bitmap anyway. Would be nice if @Blisto91 has time to test it with Castle Strike, however I can confirm it works just fine with Dungeon Siege 2 and there's no reason for it not to work with anything else really, there's no nasty hacks this time.

Feel free to rewrite the presentation/draw part if you think that's more desirable, but if we can deffer that bit to the backend I'd be even happier (and this should do for now).

[WinterSnowfall]

Thank you for testing.

In CS as we can see, it works. Problem is that with it enabled (do not happen with disabled), try for example changing resolutions after that enabled - game getting stuck with it. On NINE via d3d8to9 for example both works and not affecting something else like that.

Try with the latest push, should be sorted now. It was an oversight of mine causing that, namely we should release the cursor bitmap texture on device reset, since it counts as a losable resource.

Actually, digging a bit deeper into what happens with cursors on device reset I came across this:

IDirect3DDevice9::SetCursorProperties

The cursor does not survive when the device is lost. This method must be called after the device is reset.

Since the documentation refers to hardware cursors (mostly), I've taken the liberty of resetting those as well on device reset.

Similar like with DS2 missing resolutions in a game options menu, user will probably do some command line width and height to go higher.

You may be hit by the "too many reported modes" situation, that we kind of know exists. Some games will limit the amount of offered resolutions, so you may not have all options visible. Try using #4128 to set only a few higher resolution modes you're interested in, see if they show up like that. If they do, this is indeed the problem, an excess of reported modes caused by your combination of monitor and video card.

[WinterSnowfall]

Works now (with enabled cursor) changing resolution do not get stuck

Great, thanks for confirming (and for finding the bug in the first place) 🎉 .

I guess, maybe app cant list all resolutions there because just its GUI is like that limited... whatever i dont mind about that :D

I mean, older games having limited resolution support is nothing new really. In many cases they'll limit the aspect ratio to 4:3, which was common at the time. It's a part of our history that will have to be modded/unofficially patched around for most games if anyone wants to retroactively add widescreen support.

[Blisto91]

Both Dungeon Siege 2 and Castle Strike seem fine here too. Tested both with Proton and the former also with regular Wine + virtual desktop.

[mrdeathjr28]

In my case cursor work with wine 9.18 new wow64

curiously in capture linux cursor seems over ds2 cursor but in game show ds2 cursor normally

[WinterSnowfall]

curiously in capture linux cursor seems over ds2 cursor but in game show ds2 cursor normally

That is expected. The game cursor with the new software cursor path is essentially a texture overlayed on top of what the game renders. And when you take a screenshot, your compositor will usually also capture your OS cursor pointer (which is hidden/invisible when you play the game).

If you use xfce4-screenshot to take the screenshot, then you can untick the "capture the mouse pointer" option and it should capture it correctly. Mind you, you only need to do this for games using software cursors, since the usual path is to use hardware cursors (thus the OS cursor directly), so this won't be a problem in most cases.

[WinterSnowfall]

I've reworked the implementation on top of the backend cursor support recently added by @doitsujin. This means a couple of things:

• the backend provided software cursor will render on top of dxvk's HUD, much like a plain non-d3d cursor
• the implementation is much leaner and less prone to glitches caused by d3d9 state