[🐧] GNU+Linux compatibility progress #1430

YoshiRulz · 2019-01-07T11:33:41Z

jump to known bugs

Overview:

Nothing on this page applies to the (discontinued) macOS port.
- Also, EmuHawk is only for x86-64 machines for the time being. If you have an x86 or AArch64 machine, you should have no trouble installing the rundeps, but EmuHawk will crash on start; see DllNotFoundException: libbizhash when opening any rom on Linux AArch64 #3687. Contributions welcome.
Build instructions are in the readme. tl;dr: run Dist/BuildRelease.sh.
"Installation" instructions are also in the readme. tl;dr: run output/EmuHawkMono.sh.
- No WINE required! If you get it working in WINE, good for you, but I'm going to ignore you and hide any counter-productive comments.
35-ish of 40-ish systems can be emulated, with the notable exception of N64 (Mupen64Plus isn't available, and the experimental Ares64 core is too slow).
For casual players, most things work as they would on Windows.
For TASers, most things work as they would on Windows.
For glitch hunters and romhackers, everything sucks and I'm sorry. But if a core is available on Linux, and the core implements the necessary services for a tool, then it should be at least functional.
Before 2.10:
- Command-line flags to EmuHawkMono.sh are passed through as expected. The single exception is if you pass --mono-no-redirect as the first flag, which disables redirecting stdout/stderr to EmuHawkMono_last*.txt (the default behaviour is to write to disk instead of the terminal).
- When it inevitably crashes, some errors go to the terminal/EmuHawkMono_last*.txt, some appear in dialog boxes, some both. Most of these error dialogs will either have a "Copy" button, or will allow Ctrl+C.

As of 2.10, the following systems can be emulated:

These systems haven't been tested or aren't finished:

Amiga with PUAE (unreleased core)
Amstrad CPC with CPCHawk (unreleased core)
Channel F with ChannelFHawk (I don't have roms --yoshi)
N64DD with Ares64 (I don't have roms --yoshi)
Sega Pico with Genplus-gx (I don't have roms --yoshi)

These features/subsystems work:

The core lifecycle i.e. loading roms (for the systems noted above)
Savestates
Virtual Pad
Tool Box
Lua Console (scripts can load, YMMV)
Config > Controllers... and Config > Hotkeys...
- Keyboard, mouse, and gamepad inputs, and from 2.10, haptic feedback
Config > Display...
Config > Sound...
Config > Paths...
Config > Firmwares...
Config > Autofire...
Config > Customize...
Config > Profiles...
Core selector (Config > Cores)
View > Window Size
Misc. OSD layers under View
Lua (from 2.9, same as Windows, before 2.9 see [🐧] Lua in Mono megathread #2951)
External tools (ApiHawk)
Movie playback, "traditional" movie recording
TAStudio (basic functionality is good)
Tools > RAM Watch
Hex Editor (works as of 625c657)
RAM Search
Tools > Cheats (does not crash)
Basic Bot (opens as of 1dbcdcd)
RetroAchievements

The text was updated successfully, but these errors were encountered:

YoshiRulz · 2019-01-07T12:17:08Z

Issues with EmuHawk

Startup issues:

An InvalidOperationException with "Error compiling shader", "syntax error, unexpected KW_SAMPLER_STATE" (stacktrace will include the IGL_SlimDX9.CreateVertexShader method, which has since been removed): Seems to be a bug in WINE's GPU drivers. WINE is not supported, so just stop using it and you won't hit this.
Gtk not found (missing LD_LIBRARY_PATH to libgtk-x11-2.0.so.0?), using built-in colorscheme—see ignored GTK theme below.
Unable to locate theme engine in module_path: "adwaita" seems to be a harmless error, but can be resolved by installing gnome-themes-extra.

Misc. issues:

Mono seemingly ignores the GTK theme... on my machine >:( (Others have reported mono/mono#20078 dark theme being inconsistently applied). As a result, everything is an ugly beige, and some icons in e.g. open file dialogs are missing. Hope you remember which is the parent directory button!
- THEMING UPDATE 2024-09: Some work has been put into making the default not so ugly, but hopefully the "correct" light theme can be installed and coerced to load as was done for NixOS in 9a2c163. Users who were affected by mixed dark/light theme have had success with the GTK_DATA_PREFIX='' ./EmuHawkMono.sh workaround, so perhaps that could be applied by default.
- There was a tiny amount of work put into making the mixed dark/light theme usable (b967e60), but EmuHawk was designed for a light theme. Please report if you're affected by the mixed dark/light theme, but don't report individual instances of illegible text for example—those are wontfix.
~~I think file extensions are case-sensitive on Mono. The rom loader dialog doesn't recognise my .32X file as a rom.~~ fixed in f8a688d
~~(unthrottled) FPS triples when window unfocused. Is input slowing it down? who knows~~ --yoshi
- ~~actually I can't repro this~~ --yoshi at 25a5449
- dwango repro'd this on stream. still no idea why it happens. --yoshi
- ~~Still can't repro, no change when swapping out OpenTKInputAdapter for a DummyInputAdapter~~ --yoshi at 24f69eb
- Repro'd on NixOS, both on 2.9.1 and at ebe3cad, though it's a very small difference in Release config. --yoshi
~~Keyboard shortcuts in RAM Watch don't work.~~ ToolStripMenuItem.ShortcutKeys isn't working at all.
- Just looked into this again with TAStudo and ProcessCmdKey isn't even being called. --yoshi at 681ef72
~~Menubar mnemonics are disabled.~~ Past me did it to prevent a crash, needs retesting. --yoshi at 03d10ce
- enabled on MainForm as of 28d62e6
- not working on other Forms --yoshi at 28d62e6
- probably same cause as ShortcutKeys --yoshi at 681ef72
Some file open dialogs cause a ArgumentOutOfRangeException in System.DateTimeOffset.FromUnixTimeSeconds(long). --yoshi at 03d10ce
- tracked as [WinForms] Unable to browse using OpenFileDialog without crashing where subdir has "out-of-range" timestamp mono/mono#21213
- note Paths config uses FolderBrowserDialog which seems to crash regardless of the timestamps' validity
In IToolFormAutoConfigs, Settings > Stay on Top correctly writes Window.KeepAbove = !item.Checked, but does not read the actual value i.e. toggling keep-above via the WM does not change the checkbox's state. I suggest this menu item be disabled on Linux (and macOS?). --yoshi at 03d10ce
- Fixed in 329a7de (by hiding the menu item).
Defaults for Config > Paths... could be changed to standards like $XDG_CONFIG_HOME. Making this change to system-global dirs would break from the "portable app" precedent. --yoshi
- Sorta done in NixHawk
Users with only an Intel iGPU (i.e. no discrete GPU) may get i965: Failed to submit batchbuffer: Invalid argument. Try LIBGL_ALWAYS_SOFTWARE=1 ./EmuHawkMono.sh or otherwise setting that environment variable. (workaround found by @InfamousKnight)
Dialogs which pause emulation should cause the current frame to be redrawn to MainForm, but instead drawing stops and you get the solitaire effect. --yoshi at 268ab96
Possibly related, under Plasma Wayland on NixOS: "open file" dialogs sometimes result in empty black windows, which then become "infectious" and result in even the main window only showing black, which persists until user logout / compositor shutdown. --bugQ
Dropdowns not closing: WinForms - ToolStrip dropdowns do not close properly mono/mono#12644 --yoshi at 42fc895
- workaround added in 653844e (sorry I didn't do that sooner --yoshi)
~~Controls in RecordMovie (dialog asking for filename and author for new movie) are misplaced, making keyboard navigation a requirement for recording movies (without TAStudio)~~ --yoshi at 9e60927
- Fixed in d32f734.
If you get an error like "GDI+ status: InvalidParameter" at System.Drawing.Image.get_FrameDimensionsList, try updating libgdiplus.
Hard crash when clicking+dragging scrollbar in OpenFileDialog. Nothing in stdout or stderr. ¯\_(ツ)_/¯
- Similarly, random hard crashes when clicking+dragging scrollbar in core settings dialogs, then moving cursor over the button of a dropdown menu without releasing the mouse button.
In Hex Editor, opening context menu causes scroll wheel and arrow keys to have no effect until the window is unfocused and refocused.
Log Window threading is fragile, see LogWindow something causes crash on Mono #2741.
Starting Mono while the solution is being rebuild will, rather than hitting the version check in EmuHawk.Program, cause Mono to segfault. (Not before init'ing GTK, though. I swear it's taunting me.)
May be NixOS-specific, but it seems idling with Lua scripts running+drawing but emulation paused causes the Lua Console (possible all WinForms UI?) to freeze, and unpausing emulation fixes it immediately...
- It also seems to unpause for no reason? Does it think I closed a menu and it was only temp. paused?
RetroAchievements dialogs aren't great, but they are technically usable
From 2.10, SDL2 handles sleep prevention, but while it's implemented for Linux (X11), it's not working in practice. See BizHawk does not inhibit display or system sleep while running #3644 (comment).
PictureBox applies anti-aliasing/interpolation to pixel graphics: NES PPU Viewer's "zoom box" shouldn't be anti-aliased #3764
Buttons in some windows and dialogs are missing their labels. Possibly related to Japanese locale. Initially reported by @miyakut on Discord, running Mint. 404f28f didn't work and was reverted.

Cores

Issues with Mupen

System.InvalidOperationException: got null pointer from dlopen, error: libmupen64plus.so.2: cannot open shared object file: No such file or directory (or, if standalone Mupen64Plus is installed, System.InvalidOperationException: got null pointer from dlopen, error: /usr/lib/mupen64plus: cannot read file data: Is a directory) from mupen64plusApi ctor --yoshi at 25a5449
fixed in 6e46cb5

Issues with TI83Hawk

IndexOutOfRangeException at BizHawk.Emulation.Cores.Calculators.TI83.ReadMemory(ushort) --yoshi at 25a5449

Building and packaging

Dependencies are being recorded (so build scripts can be made) in the GitLab mirror's snippets. On multi-arch distros, target x86_64/amd64.

For the record I've mostly focused on Manjaro (which I use), Nix/NixOS, Linux Mint, and Ubuntu. --yoshi

Bus error (core dumped) when building may be because you're out of disk space.

libblip_buf works as intended on both OSes, but the Makefile for Linux builds was hacked together (see libblip_buf issue tracker) and using the library in BizHawk was intended to be a temporary hack. The library has been rewritten as noted in #1312. Sappharad has previously attempted moving it to managed C#. ~~It may be possible to use libspeex instead.~~ they serve different purposes according to #3207 (comment) --yoshi

Don't know where else to leave these:

Debug P/Invokes (.dll/.so loading) with MONO_LOG_LEVEL=debug MONO_LOG_MASK=dll ./EmuHawkMono.sh --mono-no-redirect.
It seems libgdiplus.so doesn't support whatever the "PNG 256x256" mode of .icos is.

EmuHawk: SharpCompress implementation for Mono - TASEmulators#1430

…licks 'open firmware folder'. This mitigates the 'Folder open dialog crash' referenced in TASEmulators#1430

…shlwapi.dll calls - TASEmulators#1430

…#1430

CasualPokePlayer · 2024-09-04T19:31:09Z

@Martmists-GH your OpenGL issue should be fixed with 60fae35

For the theme issue, try setting the GTK_DATA_PREFIX environment variable to nothing (e.g. add export GTK_DATA_PREFIX= to the EmuHawkMono.sh script)

BRQuad · 2024-09-17T19:42:15Z

TAStudio unusable with Encore
Font setting not saved
Invisible buttons in the open dialog

CasualPokePlayer · 2024-09-17T19:58:51Z

Elaborate on "unusable." If you are referring to the lack of rampant automatic stating, that is intentional, you have to create markers and branches for greenzone to be created (greenzone will be created near those).
Not sure what you are referring to by "font setting"
See above comment with GTK_DATA_PREFIX.

BRQuad · 2024-09-18T19:09:25Z

In a new TAStudio project, if i click one time in the A column, the processor is saturated (2 cores/6), i can't use my PC anymore, i must disconnect myself, and then kill the mono process.
The font setting is the one in TAStudio (config > Set font).
I added export GTK_DATA_PREFIX= (without value ?) in EmuHawkMono.sh but the problem persists.

Jakeory1 · 2024-09-20T10:31:37Z

This issues has been mentioned here already:
#1430 (comment)

I just want to bring it back up to ask if there is a hotfix. Also, if there might be a way to fix this in the next version of BizHawk, this might serve as a small reminder, in case you happen to work on something related anyways.

Summary
Text drawn by a lua script has bad quality because of the way it is rendered.
This is the case for many linux distros. E.g. Mint 22 Cinnamon 6.2.9, Wayland and the latest Debian KDE Plasma.
But it was fine (iirc) in Mint 21.3 Cinnamon, Wayland.

If the comments already explained a fix, it would be great if someone could explain further!

CasualPokePlayer · 2024-09-20T10:40:25Z

Any gui string lua function outside of gui.text is handled by GDI+. On Windows this performs as you expect as GDI+ is on Windows. On Linux, this performs badly, as GDI+ is a Window specific library, so Mono reimplements it (as libgdiplus). This reimplementation is incomplete, spotty and outright buggy in some cases, and there isn't anything that can be done about that. At some point in the future, gui.pixelText could probably be easily brought into the newer ImGui hw accelerated handling, which could help here. gui.drawText is unlikely ever going to work well in the foreseeable future (unless we somehow get that working with the new system). gui.text just reuses BizHawk's OSD message system, so it is unaffected by this kind of issue.

YoshiRulz · 2024-09-21T17:44:53Z

@BRQuad The unlabelled buttons in file open dialogs is a known issue, which I believe can be fixed by finding the package which provides libgtk-x11-2.0.so.0.

BRQuad · 2024-09-23T19:08:20Z

This lib is included in libgtk2.0-0 located in /usr/lib/x86_64-linux-gnu, and it is installed.
The buttons are square, thus it would be more an icon issue than a label issue.

awsms · 2024-10-28T20:52:44Z

I'm unable to use OpenGL on the latest dev build:
System.Exception: Initialization of OpenGL Display Method failed; falling back to GDI+
I don't see any other errors (stderr.txt is empty). It worked on the bizhawk-bin package (2.9.1 build).
Looks like the program doesn't use the NVIDIA GPU at all, as it doesn't appear in nvidia-smi's output.
Setup: NVIDIA 4070 (laptop) + AMD APU, EndeavourOS
Launching bizhawk by simply running ./EmuHawkMono.sh from the dev build.

CasualPokePlayer · 2024-10-28T21:00:13Z

Does stdout.txt have anything?

awsms · 2024-10-28T21:12:59Z

Does stdout.txt have anything?

Its content:

Using SDL2 for host input (keyboard + gamepads)
Connected SDL gamepad, device index 0, instance ID 1, name Xbox 360 Controller
Disconnecting SDL gamepad, device index 0, instance ID 1, name Xbox 360 Controller
BizHawk has completed its shutdown routines, killing process...

YoshiRulz · 2024-10-28T22:34:13Z

Did you download the dev build manually or via makepkg? The build script on the AUR may be doing something that's necessary, IDK, I haven't checked on it in a while.

awsms · 2024-10-29T00:55:51Z

Did you download the dev build manually or via makepkg? The build script on the AUR may be doing something that's necessary, IDK, I haven't checked on it in a while.

Yeah I took the existing PKGBUILD, and tweaked it for the dev build.
Even if I dl the 2.9.1 release from gh, extract it, and run the shell script without any prior config, OpenGL works just fine.
But there is something wrong in the dev one for me, unfortunately there are no verbose errors :(

hhirtz · 2024-10-29T21:38:23Z

I get the same opengl error as #1430 (comment) with a AMD radeon, sway, archlinux and the 2.10-rc release tarball.

Then when GDI is used, this error happens:

X11 Error encountered: 
  Error: BadMatch (invalid parameter attributes)
  Request:     12 (0)
  Resource ID: 0x4000A4
  Serial:      6502
  Hwnd:        Hwnd, Mapped:True ClientWindow:0x4000A5, WholeWindow:0x4000A4, Zombie=False, Parent:[Hwnd, Mapped:True ClientWindow:0x4000A7, WholeWindow:0x4000A6, Zombie=False, Parent:[<null>]]
  Control:     BizHawk.Bizware.Graphics.Controls.GDIPlusControl  at System.Environment.get_StackTrace () [0x00000] in <296872a6734f443990477e3abd954b57>:0 
  at System.Windows.Forms.XplatUIX11.HandleError (System.IntPtr display, System.Windows.Forms.XErrorEvent& error_event) [0x00000] in <c87c1b414e134a3b85f757beb69a2505>:0 
  at System.Windows.Forms.XplatUIX11.XGetWindowProperty (System.IntPtr , System.IntPtr , System.IntPtr , System.IntPtr , System.IntPtr , System.Boolean , System.IntPtr , System.IntPtr& , System.Int32& , System.IntPtr& , System.IntPtr& , System.IntPtr& ) [0x00000] in <c87c1b414e134a3b85f757beb69a2505>:0 
  at System.Windows.Forms.XplatUIX11.GetActive () [0x00000] in <c87c1b414e134a3b85f757beb69a2505>:0 
  at System.Windows.Forms.XplatUI.GetActive () [0x00000] in <c87c1b414e134a3b85f757beb69a2505>:0 
  at System.Windows.Forms.Form.get_ActiveForm () [0x00000] in <c87c1b414e134a3b85f757beb69a2505>:0 
  at BizHawk.Client.EmuHawk.Input.UpdateThreadProc () [0x00000] in <04399846313242f4a77521075d3a9e24>:0 
  at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00000] in <296872a6734f443990477e3abd954b57>:0 
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <296872a6734f443990477e3abd954b57>:0 
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <296872a6734f443990477e3abd954b57>:0 
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x00000] in <296872a6734f443990477e3abd954b57>:0 
  at System.Threading.ThreadHelper.ThreadStart () [0x00000] in <296872a6734f443990477e3abd954b57>:0

On the same machine, bizhawk 2.9.1 from aur crashes on resize (any window, lua console resize will crash bizhawk for example) and freezes indefinitely so i have to killall mono or hit Ctrl-C. This happens only when a ROM is loaded. No errors in logs.

On another machine (Intel gpu+sway+archlinux) bizhawk 2.9.1 worked fine (with OpenGL at least, haven't tested GDI).

CasualPokePlayer · 2024-10-30T05:51:56Z

The GDI error I would assume is perhaps something the hottest new XWayland is getting horribly wrong, unsure what would be going wrong with OpenGL.

CasualPokePlayer · 2024-10-30T08:12:24Z

Relooking at what the core for OpenGL actually does, the exception message is actually just coming from here. This is indicating just creating an OpenGL 3.2 Core context is failing for some reason.

I've pushed a change to actually log the error, please re-check on the latest dev build.

hhirtz · 2024-10-30T09:53:35Z

Thank you, the problem was that I had a SDL_VIDEODRIVER=wayland in my environment. I changed it to SDL_VIDEODRIVER=wayland,x11 and opengl worked.

OpenGL check for version 3.2 failed, underlying exception: System.TypeInitializationException: The type initializer for 'BizHawk.Bizware.Graphics.SDL2OpenGLContext' threw an exception. ---> System.Exception: Could not init SDL video! SDL Error: wayland not available
  at BizHawk.Bizware.Graphics.SDL2OpenGLContext..cctor () [0x0004b] in <bbc4fed5ee5441da8324ddef167733e9>:0 
   --- End of inner exception stack trace ---
  at (wrapper managed-to-native) System.Object.__icall_wrapper_mono_generic_class_init(intptr)
  at BizHawk.Bizware.Graphics.OpenGLVersion.CheckVersion (System.Int32 requestedMajor, System.Int32 requestedMinor) [0x00007] in <bbc4fed5ee5441da8324ddef167733e9>:0

CasualPokePlayer · 2024-10-30T14:18:17Z

That's strange, I intentionally ensure x11 is set for that hint here, an environment variable overrides it? I would guess might need to clear out the environment variable too in BizHawk's code.

hhirtz · 2024-10-30T15:07:07Z

yeah it seems env vars have higher priority than SetHint:
https://wiki.libsdl.org/SDL2/SDL_SetHint#remarks

CasualPokePlayer · 2024-10-30T15:20:39Z

6578c85 hopefully should fix this.

ensures calls actually do something and won't be ignored due to env vars

CasualPokePlayer · 2024-11-04T01:00:26Z

@awsms To be clear, did the above commit (i.e. latest dev build) fix your issue?

awsms · 2024-11-04T06:17:11Z

@awsms To be clear, did the above commit (i.e. latest dev build) fix your issue?

Nope. However, I do have a visible error in stderr now:

OpenGL check for version 3.2 failed, underlying exception: System.Exception: Could not create SDL Window! SDL Error: Could not create GLES window surface
  at BizHawk.Bizware.Graphics.SDL2OpenGLContext..ctor (System.Int32 majorVersion, System.Int32 minorVersion, System.Boolean coreProfile) [0x0004a] in <6d86480a70a14f6c8bf4143e01cea68b>:0 
  at BizHawk.Bizware.Graphics.OpenGLVersion.CheckVersion (System.Int32 requestedMajor, System.Int32 requestedMinor) [0x00007] in <6d86480a70a14f6c8bf4143e01cea68b>:0

CasualPokePlayer · 2024-11-04T08:09:16Z

That seems very odd, it's for some reason failing here. This is also somehow after it works here, so ???

Could you try doing export SDL_VIDEO_X11_FORCE_EGL=0 before launching EmuHawkMono.sh and see if that ends up working?

awsms · 2024-11-04T19:56:11Z

Could you try doing export SDL_VIDEO_X11_FORCE_EGL=0 before launching EmuHawkMono.sh and see if that ends up working?

That worked!

CasualPokePlayer · 2024-11-07T02:24:32Z

Could you try doing export SDL_VIDEO_X11_FORCE_EGL=0 before launching EmuHawkMono.sh and see if that ends up working?

That worked!

That's odd, that's indicating that EGL seems to be initially succeeding but failing when trying to use it for a second window.

Best guess perhaps is SDL not cleaning up the window surface of the initial test window somehow interfering with this. Could you try replacing the libSDL2.so in the dll folder with this build and see if it works (without the export)? libSDL2.zip

awsms · 2024-11-07T04:39:28Z

Replacing the lib with the one you linked gave me the same error (and as expected, it works if I override the SDL_VIDEO_X11_FORCE_EGL env variable):

OpenGL check for version 3.2 failed, underlying exception: System.Exception: Could not create SDL Window! SDL Error: Could not create GLES window surface
  at BizHawk.Bizware.Graphics.SDL2OpenGLContext..ctor (System.Int32 majorVersion, System.Int32 minorVersion, System.Boolean coreProfile) [0x0004a] in <5c4f421a02a9479ab97949d87d943696>:0 
  at BizHawk.Bizware.Graphics.OpenGLVersion.CheckVersion (System.Int32 requestedMajor, System.Int32 requestedMinor) [0x00007] in <5c4f421a02a9479ab97949d87d943696>:0

CasualPokePlayer · 2024-11-07T06:31:50Z

Looking at BizHawk's code, I could perhaps actually be mislead by the issue, since the context creation might also fail in the static ctor, but it wouldn't be reported. This might be more a bug in BizHawk's fallback code, since it doesn't explicitly unload the library?

I've changed the code in the latest commit, see if a dev build here changes things?

awsms · 2024-11-07T09:51:51Z

I've changed the code in the latest commit, see if a dev build here changes things?

I can confirm you it works now!