Doorstop on Linux logging to .config/mimeapps.list

[huantianad]

Using Unix version of BepInEx 5 on Rhythm Doctor.

Whenever I launch Rhythm Doctor, doorstop seems to be logging to my .config/mimeapps.list file and it breaks all my default application handling! It adds the line Found UnityPlayer, hooking into it instead to the start of the file so my system can't parse the file.
It also makes a folder in the game directory called Found UnityPlayer, hooking into it instead in the game folder, with this file, minus the .txt extension.
profilerc.txt

It seems to be some error with how doorstop is trying to capture/redirect stdout, but for now I've compiled doorstop without logging to fix it.

[ManlyMarco]

BepInEx 5 was using Doorstop 3 until very recently. This repo is for Doorstop 4.

Try the latest release of BepInEx 5, it comes with the latest version of Doorstop that might not have those issues.

[stele95]

This problem is now more present with the latest update from Steam, forcing Linux native games to run inside their runtimes.
Take a look at this issue, specifically this comment and comments after it.
Even tho Valheim uses an older version of doorstop, the issue is the same since even the latest version of doorstop is using printf to print logs, which will not work anymore inside the steam's linux container runtime (read this comment).
I saw this PR on the old Unix version of doorstop and I think something like that should be implemented in the new version, moving logging to a separate file instead of the stdout

[huantianad]

Good to hear that I'm not going insane, wish I had actually followed up on this earlier tho...

[aldelaro5]

@huantianad @ManlyMarco so I was going to open an issue, but this is reporting the exact same problem I been investigating over on the melonloader's bootstrap side because it turns out it's also affected! I now have a VERY good idea of what's happening, but I can't myself pr a fix (melonloader's stability stuff has been taking most of my time, but the fix isn't too hard in concept...just tedious to do in C). So I might as well share what I found here (I was about to pr a fix to melon among other things).

Essentially, it's a double entry problem from other processes that the game launches. If it sounds weird, it should be fairly common because take discord RPC for example: it will do some xdg-mime stuff to do things like add associations to launch games within the client. Here's where I landed up in the case I was debugging: https://github.com/discord/discord-rpc/blob/963aa9f3e5ce81a4682c6ca3d136cddda614db33/src/discord_register_linux.cpp#L86

There's many other processes a game might spawn that might just do very innocuous things, but unfortunately, since we mess with LD_PRELOAD and LD_LIBRARY_PRELOAD and because those will be carried over by the child processes, you start to get into some strange things. For example: because of that xdg-mime above, it was doing some nonsense like processing commands and stuff on its own and that resulted in the default app list getting messed up because the commands it was processing got messed up.

Even worse: doorstop actually prevent Unity to redirect stdout and stderr to their player so EVERYTHING the thing outputs gets sent directly to the terminal which can cause some funnies like I had the output of the help of lsb-release because someone wanted its results, but the terminal and the LD_PRELOAD somehow mangled this and now the command is different.

In reality: we really shouldn't care about these processes. All doorstop should be interested in is the first entry and stop hooking stuff after that. Unfortunately, there's 2 problems:

1. LD_PRELOAD and LD_LIBRARY_PATH will still carry over the child processes meaning by default, Doorstop's constructor ALWAYS kicks in when any processes loads and there's not really a way to stop it while these env vars are set
2. The first time the constructor is called isn't guaranteed to be the correct one. For example: if I want to debug a game with gdb, I have to set the env var and then call gdb, now, doorstop will kick in...for gdb. Even worse, when you use the "run" command, now doorstop kicks in on the internal gdb's exec command AND THEN FINALLY the launch happens with the entry we're interested in

With these 2 problems in mind, I found (and tested this works on melon's bootstrap) a solution that addresses both problems. Here's how the new flow would work.

First, you need on entry to determine with good certainty that you are dealing with the executable you're interested in. Since doorstop is specific to Unity, I think there's an easy way: find out if there's a directory that exists named X_Data where X is the game's name. This should be fine to assume because I seen several instances of unity player in ghidra where they hardcode that _Data suffix and it's something I seen in ancient versions too. It's a heuristic, but we just want something "good enough" here. If such a directory exists, you can proceed and if not, exit immediately. Unfortunately, this HAS to be checked everytime the constructor kicks in, but unless the game spawns a ton of processes, it shouldn't be too bad, mostly tedious to do in C.

Once you confirmed you're in the process you're interested in, IMMEDIATELY REMOVE YOURSELF FROM LD_PRELOAD AND LD_LIBRARY_PATH! This is important: it is the only way I found to ever be able to prevent double entry. You can't just store a global bool: it won't carry over to the new process. It MIGHT be possible you could get it to work with an env var, but in general, the mere fact doorstop can kick in WELL after boot is problematic and prone to issues so better to cut the link after you are 100% going to hook. Note: you can't just unset the env vars because it's possible someone else (such as steam) was using it so you need to make sure to ONLY remove what was placed in order to boot doorstop.

Hope that helps figure out a fix :)