WIP: Native Port

[misyltoad]

This PR allows DXVK to be built natively on Linux with an option to specify the WSI you want to use.

This allows titles to port to Linux and potentially other platforms further down the line much easier by letting them keep their existing rendering code for D3D11.

The exact same code path can also be used on Windows if the title wishes to ship with DXVK there too, and is compatible with the regular Windows headers (ie. you can use SDL2 WSI intergration on Windows builds as well.)

[misyltoad]

[Turns out Github drafts are not the same as Gitlab ones 🐸]

[doitsujin]

I'm going to leave this open for now until a final decision is made, but as discussed on Discord, with DXVK entering maintenance mode, I'd like to avoid any significant changes or additions to the code base that are not strictly necessary.

[eszlari]

with DXVK entering maintenance mode

Is this, because you consider DXVK to be feature complete?

[doitsujin]

Is this, because you consider DXVK to be feature complete?

It's because DXVK has become a fragile, unreliable and frustrating maintenance nightmare. Most of the 1.4.x releases introduced major regressions which I cannot reproduce, and therefore cannot debug and fix.

This includes GPU hangs in Overwatch on specific maps with Nvidia GPUs (some users claim it's fixed in 1.4.6 while others still have them), rendering issues in Dishonored 2 which I can't reproduce (see ValveSoftware/Proton#823 (comment)), vertex explosions in some games which I also can't reproduce, an ongoing Star Citizen issue which I still need to debug (see #1262), and tons of weird issues that don't make any sense whatsoever (like #1266 which only seems to affect RADV).

Most of these problems are still unresolved and I have no idea how to even track them down, let alone fix them, and the ones that got "fixed" got fixed by reverting otherwise useful changes because I simply do not understand any of the issues at all.

Doing any sort of active development with this broken mess of a code base would only make this worse, and I wish I had drawn the line sooner. The only thing I still plan to do is wire up some useful Vulkan extensions and eventually merge D9VK, the rest will be bug fixing only.

[jediafr]

It does sound like a burnout. I would advise 2 weeks iatus to stop working in crunch mode. A rethinking of the framework might benefit the project long term. Valve can understand this.

EDIT : my advice to doitsujin comes from the heart for the difference he made by writing dxvk AND his commitment to it. After two decades of IT, i've experienced some burnout situations.
It's avoidable and nothing justifies being pushed to crunch , least of all gaming. Dxvk is a labour of love, and it shows, to doitsujin : take it easy and relax.

[Pyrathar]

It does sound like burnout, but sir i would like to say that im pretty sure everybody in gaming linux community is thankful for your work, this made such a huge impact in my life and im sure it did on a lot of other people aswell, take a vacation mate! you more than earned it!

[veryprofessionaldodo]

I agree with what was stated above, you did/are doing some amazing work, and I'm incredibly thankful for your achievements! @doitsujin

[devland]

Hey, Philip. We really appreciate your work. It's been inspiring to watch this project evolve over time and to see real world applications at a small (me playing Windows games in Lutris under Linux) and large scale (Valve's Proton).

I'm a developer myself and what you're experiencing sounds a lot like you need a vacation. Working on the same piece of code for long periods of time will get you tunnel vision and the only way out is to just take a break and relax. :)

[zorftw]

Hey Phil, same here, I can probably speak for the community for this matter, take a break. Do it for yourself, and instead just focus on your personal live man, you need to catch up!

Have a nice day bro!

[PMunkes]

Just wanted to say thank you for all your hard work. Take a few weeks off, at least till the new year.

[wdbm]

DXVK is awesome. I simply will not use malware like Windows, so DXVK means I can enjoy the likes of Alien: Isolation or Gwent. Thank you. :)

[brenthuisman]

Hang in there @doitsujin, you've been doing the unimaginable with unimaginable success. Take care of yourself and take your time!

[julienbenjamin]

Is this, because you consider DXVK to be feature complete?

It's because DXVK has become a fragile, unreliable and frustrating maintenance nightmare. Most of the 1.4.x releases introduced major regressions which I cannot reproduce, and therefore cannot debug and fix.

This includes GPU hangs in Overwatch on specific maps with Nvidia GPUs (some users claim it's fixed in 1.4.6 while others still have them), rendering issues in Dishonored 2 which I can't reproduce (see ValveSoftware/Proton#823 (comment)), vertex explosions in some games which I also can't reproduce, an ongoing Star Citizen issue which I still need to debug (see #1262), and tons of weird issues that don't make any sense whatsoever (like #1266 which only seems to affect RADV).

Most of these problems are still unresolved and I have no idea how to even track them down, let alone fix them, and the ones that got "fixed" got fixed by reverting otherwise useful changes because I simply do not understand any of the issues at all.

Doing any sort of active development with this broken mess of a code base would only make this worse, and I wish I had drawn the line sooner. The only thing I still plan to do is wire up some useful Vulkan extensions and eventually merge D9VK, the rest will be bug fixing only.

Is there any way to help you out, without knowing how DXVK has been designed?

[Fordi]

I don't know about the current structure, but in my experience, compatibility layers tend to start as a straightforward API surface, which descends into a mess of heuristics and hacks as edge cases are found and addressed, until a breaking point where Things Must Change. It sounds like you're nearing that crest (which sucks).

If you had all the time in the world, and didn't have to worry about risk, is there a way you would restructure DXVK to be easier to maintain? If you can define it readily, it might be that you can petition Valve to get you more resources for a re-arch to simplify maintenance without having to promise performance improvements for a while. Get them to commit to a bug moratorium and give you people to help you clean house.

But like others said, you should maybe take some vacation for the holiday season, do something nice for yourself. Graphics drivers and compatibility layers and other things users shouldn't notice are thankless, grueling work, rife with heisenbugs and opaque stacks - but you still manage to fix some crazy shit.

Let yourself have a real reward, and I think I speak for the whole community when I say you seriously deserve it.

Then, please, come back once again and be amazing.

[telmotrooper]

@doitsujin I just can't thank you enough for how much of a difference you've made in the status quo of computer gaming. Wish you best of luck!

[misyltoad]

Come play TF2 with me and @liam-middlebrook 🐸

[doitsujin]

If you had all the time in the world, and didn't have to worry about risk, is there a way you would restructure DXVK to be easier to maintain?

My main issue is that I simply don't understand most of the issues that are popping up everywhere lately, not that the code base itself is poorly structured. There are some things that could use some cleanups, but the last attempts at doing that have caused exactly the kind of regressions I was complaining about, and it really seems like touching anything causes some arbitrary breakage for no particular reason.

I don't know if we're running into driver issues (could be given that these things tend to be limited to a single vendor) or if there's some insane case of undefined behaviour going on in my code (of course dxvk being a windows library makes it a lot harder to debug such things since none of the standard tools like valgrind work properly), but it doesn't really matter either way, breakage is bad and should be avoided.

[Fordi]

If opacity is the problem, would the next big step be instrumentation?

[doitsujin]

No since that would either cause a major performance hit even if it's not needed, or make everything incredibly messy due to #ifdef magic (or both).

[Fordi]

Could try some sugar that encapsulates basic patterns to avoid ifdef cruft, like:

#ifdef TRACING
#define INSTRUMENT(stmt) beginInstrument();\
 stmt \
endInstrument();
#else
#define INSTRUMENT(stmt) stmt
#endif

Then use the macro around code that you want to keep track of, e.g.,

int thingDoer() { INSTRUMENT(
  doTheThing();
) }

Dunno how generic it could be, or what insights you could gain, and I'm probably wasting your time, though. Don't know what the patterns would even look like. I just know I've used that kinda thing to make conditional compilation look a lot neater in my own code (which, admittedly, has been dumb little arduino stuff).

[doitsujin]

I mean it's not like it couldn't be done, usually I do that manually when tracking down a crash or bug for the functions where it matters, but there are multiple issues with that approach.

• We can't debug GPU hangs this way, at all. This needs GPU-side work, and obviously extra code on the CPU side to actually do that.
• Even if purely CPU-side debugging is useful (e.g. when something segfaults), demanding games easily do >10 million API calls per second. Logging all of those, plus all the backend work, would be insane.
• There's just so much going on on multiple threads that just spamming messages everywhere won't necessarily make it easier to figure out what the actual problem was. Wine's debug channels kind of have the same problem; if things don't work due to an unimplemented feature it's easy to figure out; if the problem sits deeper than that, it often isn't.

[barotto]

The fact that many PC gamers like to overclock their systems is not helping either. I think one requirement for bug reporting should be "please, if your system is overclocked in any way (CPU, GPU, and/or RAM) retest at stock before reporting bugs."

[brianinthered]

I also thank you Phillip for all of your dedication and hard work!
I test windows games in Wine on a daily basis and post step-by-step guides in PlayOnLinux weekly. Without DXVK most of my tests will fail because of DirectX 10/11
Thanks to you we have so many games and guides that allow us to play indie "windows only" games from itch.io and AAA games from GOG!

Definitely take a break and clear your head and give it a second pair of eyes later on. Refresh yourself and enjoy some time off. It won't affect us at all... we can wait! I've waited years to see Windows DirectX 10/11 games to run in Linux so I have no problem waiting a few weeks or even months!

By the way, what bugs have you noticed? Almost every Unity3D and Unreal Engine game I've tested runs great with every version of DXVK I test. I rarely notice any issues at all.

[dhollinger]

@doitsujin Is there any situation where you'd feel comfortable handing the project maintainership off to another person or a community to maintain?

[o-bri]

Hi Philip,

thanks for your work so far!

I think it would be important, that you don not do all this work alone. It is amazing, that you came this far with this complex piece of work.

Perhaps you can take a break an Valve should organize, that you have 1-3 devs to help you and discuss problems an solutions with them.
I was working as a developer and I can remember thinking about bugs/problems for hours. And then simply explaining them to another dev made my brain understand it. Sometimes it really didn't even need an answer from the other one - just someone listening.

I hope this is not the end of the amazing duo DXVK/Proton.

Kind regards
Oliver

[gardotd426]

DXVK is awesome. I simply will not use malware like Windows, so DXVK means I can enjoy the likes of Alien: Isolation or Gwent. Thank you. :)

Alien: Isolation has a native version though?

Anyway, @doitsujin I just want to pile on with what everyone else said. Your project here has made a gigantic impact on my life, and I hope that isn't lost on you. A lot of us REALLY love games, but the thought of having to even dual boot and use Windows to play them makes us legitimately upset/sick. Sure, there are some huge games I'd love to play but can't because I won't make that sacrifice of using Windows at all, but because of you, and Valve, and @GloriousEggroll, and the people at @lutris, I can play MORE than enough amazing games. Without you, I honestly would have had to swallow my disgust and use Windows, but because of you, I get to keep thoroughly enjoying my computer while also retaining control of it and use the operating system I truly love.

You're honestly one of the Mount Rushmore faces of Linux Gaming along with GE and @Tk-Glitch, and even if you never develop DXVK again, you've done more than enough. Take a break.

[mphuZ]

doitsujin, I hope you will put your efforts into the VKD3D project.

[brunonymous]

Hello Philip,

I would like to thank you for all the work you have done on DXVK. Thanks to your work my son can play a lot of Windows games under Ubuntu Linux 18.04 (Overwatch, Monster Hunter: World, ...).

Thank you again and don't give up.

Best regards
Bruno

[Diliz]

Hello!

Thanks for all the work, I wanted to post a big hug message to the community here, but most of the ppls would not have read it xD

So this is just a big thanks, much love and stay cool ppls, sometime take a break is the best way to handle things.

PS: If you don't know what's the problem with some buggy stuff, I recommend trying to list the broken stuff, and make a study with your mates, let some mates try to identify what's broken, more brains, more possibilities :)

[Arxcis]

Hey @Joshua-Ashton. I want to start contributing to dxvk and this PR here spurred my interest in particular. What a great initiative. Native DXVK ftw! :D

Is there anything I can do to help move this PR forward? Maybe I can do some testing? Compile on my system? Code review? Fix conflicts? Anything else?

[misyltoad]

Hi @Arxcis, I actually moved development to the dxvk-native repo on my profile. That's rebased against something fairly close to master now. I'm going to hook up thread priority stuff shortly and add an option to compile statically.

[Arxcis]

Cool. That kind of makes sense for such a big change like this. Do you see it as a separate project, or do you eventually want to merge it back into dxvk ? I can imagine it must be hard keeping the two projects (compilation paths) compatible with each other.