surreal-engine: init at 2024-11-08

[hughobrien]

Description of changes

Introduce SurrealEngine

The goal of this project is to reimplement enough of the original Unreal Engine
to make the Unreal Tournament (UT99) maps playable.

Testing this package requires Unreal Tournament (1999) to be available. I used wine to run the installer.

nix-shell -p p7zip --command '7z x -out99-img "Unreal Tournament (USA) (Disc 1) (Game).7z"'
nix-shell -p bchunk --command 'bchunk ut99-img/*.bin ut99-img/*.cue ut99-img/ut99'
nix-shell -p p7zip --command '7z x -out99-fs ut99-img/ut9901.iso'
WINEPREFIX=${PWD}/ut99-wine nix-shell -p wineWowPackages.full --command "wine ut99-fs/Setup.exe"

One should also patch the installation to the version best supported by SurrealEngine

WINEPREFIX=${PWD}/ut99-wine nix-shell -p wineWowPackages.full --command "wine UTPatch436nodelta.exe"

Then run with

nix-shell -I nixpkgs=/data/nixpkgs -p surreal-engine --command "SurrealEngine ut99-wine/drive_c/UnrealTournament"

If the dialog box is blank, move your cursor around the bottom area where the buttons should be.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[hughobrien]

Re-opening this by request (closed before as it was getting stale)

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/1016/183

[OPNA2608]

(I don't know why I was elected to be a reviewer on this, but here goes)

[hughobrien]

@OPNA2608 thank you for your review, I'm implemented all your suggestions.
Forgive my tagging but since we collaborated on dosbox-x I thought you might have an interest in retro games.

[OPNA2608]

Interest yes, time ehhhh. Will try.

Please fetch & apply dpjudas/SurrealEngine#142 as a patches entry, or bump if it gets merged before you get to it, to fix building on aarch64-linux (haven't tested running yet).

Also, during configurePhase:

-- Checking for modules 'wayland-client++;wayland-client-extra++;wayland-client-unstable++;wayland-cursor++;xkbcommon>=0.5.0'
--   No package 'xkbcommon' found

Needs libxkbcommon in buildInputs to be happy with that search. Continues then with

-- Checking for modules 'wayland-client++;wayland-client-extra++;wayland-client-unstable++;wayland-cursor++;xkbcommon>=0.5.0'
--   Found wayland-client++, version 1.0.0
--   Found wayland-client-extra++, version 1.0.0
--   Found wayland-client-unstable++, version 1.0.0
--   Found wayland-cursor++, version 1.0.0
--   Found xkbcommon, version 1.7.0
Package wayland-client was not found in the pkg-config search path.
Perhaps you should add the directory containing `wayland-client.pc'
to the PKG_CONFIG_PATH environment variable
Package 'wayland-client', required by 'wayland-client++', not found
Package wayland-client was not found in the pkg-config search path.
Perhaps you should add the directory containing `wayland-client.pc'
to the PKG_CONFIG_PATH environment variable
Package 'wayland-client', required by 'wayland-client++', not found
Package wayland-client was not found in the pkg-config search path.
Perhaps you should add the directory containing `wayland-client.pc'
to the PKG_CONFIG_PATH environment variable
Package 'wayland-client', required by 'wayland-client++', not found
Package wayland-client was not found in the pkg-config search path.
Perhaps you should add the directory containing `wayland-client.pc'
to the PKG_CONFIG_PATH environment variable
Package 'wayland-client', required by 'wayland-client++', not found

Haven't checked if it's 100% happy with that arrangement already, otherwise the following would need to be added to buildInputs as well: wayland libffi

[hughobrien]

Please fetch & apply dpjudas/SurrealEngine#142 as a patches entry, or bump if it gets merged before you get to it, to fix building on aarch64-linux (haven't tested running yet).

Upstream has merged and I've updated the src.

Needs libxkbcommon in buildInputs to be happy with that search.

Added

the following would need to be added to buildInputs as well: wayland libffi

Added

Thank you for your detailed attention.

[OPNA2608]

I had to handle the UT99 setup on an x86_64 machine & copy over the resulting UnrealTournament directory. The initial dialogue window was a solid colour when using SDL's wayland backend, but pressing enter works - with the x11 backend the dialogue works under xwayland, but the game segfaults almost instantly. So aarch64 & Wayland is likely not the best setup to test this, but it's my setup. 😄

I assume it tries to force fullscreen & a resolution change, that doesn't work with my compositor. And Vulkan is not supported by my SoC, so the rendering is really slow. But it seems to work?

Will try this on a more normal setup.

[hughobrien]

Try with gamescope

gamescope --expose-wayland --backend sdl -- SurrealEngine /data/ut99-wine/drive_c/UnrealTournament

[OPNA2608]

Try with gamescope

gamescope --expose-wayland --backend sdl -- SurrealEngine /data/ut99-wine/drive_c/UnrealTournament

↪ gamescope --expose-wayland --backend sdl -- /nix/store/k4kaml0a4v1m5w1nfbzhm9pd5pb8wi11-surreal-engine-0-unstable-2024-11-04/bin/SurrealEngine ~/Games/UnrealTournament
[gamescope] [Info]  console: gamescope version 3.15.11 (gcc 13.3.0)
No CAP_SYS_NICE, falling back to regular-priority compute and threads.
Performance will be affected.
Your Wayland compositor does NOT support wp_presentation/presentation-time which is required for VK_KHR_present_wait and VK_KHR_present_id.
Please complain to your compositor vendor for support. Falling back to X11 window with less accurate present wait.
[gamescope] [Info]  vulkan: selecting physical device 'llvmpipe (LLVM 18.1.8, 128 bits)': queue family 0 (general queue family 0)
[gamescope] [Info]  vulkan: physical device does not support DRM format modifiers
[gamescope] [Error] vulkan: physical device doesn't support VK_EXT_physical_device_drm
terminate called without an active exception
fish: Job 1, 'gamescope --expose-wayland --ba…' terminated by signal SIGABRT (Abbruch)

Let's just leave this at "system/setup is struggling, but in theory it prolly works fine on ARM" 😅.

I'll give it a shot on x86_64 tonight, so we can push this PR along.

[LupertEverett]

The initial dialogue window was a solid colour when using SDL's wayland backend, but pressing enter works

Try moving the mouse around, hovering over the buttons should "force a repaint" or something.

I'm not sure why that happens honestly.

(Hi from SurrealEngine btw!)

[OPNA2608]

SIGABRTs when running on non-Wayland after building with Wayland support.

Starting program: /nix/store/xsgrhxfdm0xby8zgrjq0zwqdg3g0fzy0-surreal-engine-0-unstable-2024-11-04/bin/.SurrealEngine-wrapped /home/puna/Games/UnrealTournament
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/nix/store/3bvxjkkmwlymr0fssczhgi39c3aj1l7i-glibc-2.40-36/lib/libthread_db.so.1".
terminate called after throwing an instance of 'std::runtime_error'
  what():  Could not connect to Wayland display server via name

Program received signal SIGABRT, Aborted.
0x00007ffff71b2a9c in __pthread_kill_implementation () from /nix/store/3bvxjkkmwlymr0fssczhgi39c3aj1l7i-glibc-2.40-36/lib/libc.so.6
(gdb) bt
#0  0x00007ffff71b2a9c in __pthread_kill_implementation () from /nix/store/3bvxjkkmwlymr0fssczhgi39c3aj1l7i-glibc-2.40-36/lib/libc.so.6
#1  0x00007ffff7160576 in raise () from /nix/store/3bvxjkkmwlymr0fssczhgi39c3aj1l7i-glibc-2.40-36/lib/libc.so.6
#2  0x00007ffff7148935 in abort () from /nix/store/3bvxjkkmwlymr0fssczhgi39c3aj1l7i-glibc-2.40-36/lib/libc.so.6
#3  0x00007ffff74acc2b in __gnu_cxx::__verbose_terminate_handler() [clone .cold] () from /nix/store/s94fwp43xhzkvw8l8nqslskib99yifzi-gcc-13.3.0-lib/lib/libstdc++.so.6
#4  0x00007ffff74bc20a in __cxxabiv1::__terminate(void (*)()) () from /nix/store/s94fwp43xhzkvw8l8nqslskib99yifzi-gcc-13.3.0-lib/lib/libstdc++.so.6
#5  0x00007ffff74bc275 in std::terminate() () from /nix/store/s94fwp43xhzkvw8l8nqslskib99yifzi-gcc-13.3.0-lib/lib/libstdc++.so.6
#6  0x00007ffff74bc4c7 in __cxa_throw () from /nix/store/s94fwp43xhzkvw8l8nqslskib99yifzi-gcc-13.3.0-lib/lib/libstdc++.so.6
#7  0x00007ffff77784f0 in wayland::display_t::display_t(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) [clone .cold] () from /nix/store/rdyy1sphkn7mavllg0bswm8w1c03fpz2-waylandpp-1.0.0-lib/lib/libwayland-client++.so.1
#8  0x000000000044167e in __static_initialization_and_destruction_0 () at /nix/store/v7dbfh5n7az2lcap0z1cv4jq0bikya8p-gcc-13.3.0/include/c++/13.3.0/bits/char_traits.h:358
#9  _GLOBAL__sub_I_wayland_display_window.cpp(void) () at /build/source/Thirdparty/ZWidget/src/window/wayland/wayland_display_window.cpp:1271
#10 0x00007ffff714a3ae in __libc_start_main_impl () from /nix/store/3bvxjkkmwlymr0fssczhgi39c3aj1l7i-glibc-2.40-36/lib/libc.so.6
#11 0x00000000004417d5 in _start ()

If windowing system selection is done at compile-time and this is expected for now(?), then we should control Wayland support via a switch in the derivation.

[OPNA2608]

Try moving the mouse around, hovering over the buttons should "force a repaint" or something.

That makes it appear yeah.

Note that it paints outside of the window frame here, dunno if that's possibly an issue with the compositor.

[LupertEverett]

If windowing system selection is done at compile-time and this is expected for now(?), then we should control Wayland support via a switch in the derivation.

Windowing system selection is controllable with the ZWIDGET_DISPLAY_BACKEND environment variable. ZWidget here being the in-house GUI toolkit Surreal Engine uses.

ZWIDGET_DISPLAY_BACKEND=[Wayland/X11/SDL2]

Note that Surreal Engine main window (not the launcher) will force the SDL2 backend regardless of the option picked.

Note that it paints outside of the window frame here, dunno if that's possibly an issue with the compositor.

Does your compositor support server side decorations? ZWidget tries to use em when available. I haven't really tested it without SSDs.

I'll admit that, as the guy who wrote the code for it, the Wayland backend is pretty buggy atm, mostly due to the lack of knowledge and experience on my part.

[OPNA2608]

If windowing system selection is done at compile-time and this is expected for now(?), then we should control Wayland support via a switch in the derivation.

Windowing system selection is controllable with the ZWIDGET_DISPLAY_BACKEND environment variable. ZWidget here being the in-house GUI toolkit Surreal Engine uses.

ZWIDGET_DISPLAY_BACKEND=[Wayland/X11/SDL2]

↪ env ZWIDGET_DISPLAY_BACKEND=SDL2 SurrealEngine ~/Games/UnrealTournament
terminate called after throwing an instance of 'std::runtime_error'
  what():  Could not connect to Wayland display server via name
fish: Job 1, 'env ZWIDGET_DISPLAY_BACKEND=SDL…' terminated by signal SIGABRT (Abbruch)

Based on the stack trace from earlier

[...]
#6  0x00007ffff74bc4c7 in __cxa_throw () from /nix/store/s94fwp43xhzkvw8l8nqslskib99yifzi-gcc-13.3.0-lib/lib/libstdc++.so.6
#7  0x00007ffff77784f0 in wayland::display_t::display_t(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) [clone .cold] () from /nix/store/rdyy1sphkn7mavllg0bswm8w1c03fpz2-waylandpp-1.0.0-lib/lib/libwayland-client++.so.1
#8  0x000000000044167e in __static_initialization_and_destruction_0 () at /nix/store/v7dbfh5n7az2lcap0z1cv4jq0bikya8p-gcc-13.3.0/include/c++/13.3.0/bits/char_traits.h:358
#9  _GLOBAL__sub_I_wayland_display_window.cpp(void) () at /build/source/Thirdparty/ZWidget/src/window/wayland/wayland_display_window.cpp:1271
#10 0x00007ffff714a3ae in __libc_start_main_impl () from /nix/store/3bvxjkkmwlymr0fssczhgi39c3aj1l7i-glibc-2.40-36/lib/libc.so.6
#11 0x00000000004417d5 in _start ()

…I think the issue is the global WaylandDisplayWindow::s_waylandDisplay variable at the start of /Thirdparty/ZWidget/src/window/wayland/wayland_display_window.cpp, since that always requires running wayland::display_t() on startup no matter what backend gets selected later.

[LupertEverett]

…I think the issue is the global WaylandDisplayWindow::s_waylandDisplay variable at the start of /Thirdparty/ZWidget/src/window/wayland/wayland_display_window.cpp, since that always requires running wayland::display_t() on startup no matter what backend gets selected later.

Can you give it a try again when you have time, it should be fixed now.

[OPNA2608]

Can confirm that on dpjudas/SurrealEngine@087fa2a it now launches fine on X11. I assume the leftover issues in-game are not a packaging issue (can't change keybinds, graphical artifacts at the start of the intro, characters are displayed upside-down in the session-config-dialogue-thingy, ...). Nonetheless pretty cool so far :)

@hughobrien can you update the PR?

[LupertEverett]

I assume the leftover issues in-game are not a packaging issue (can't change keybinds, graphical artifacts at the start of the intro, characters are displayed upside-down in the session-config-dialogue-thingy, ...).

Yeah, they're bugs with the engine instead. We documented the known ones in the bugs.txt file in the repo.

Nonetheless pretty cool so far :)

Thank you! :D

[OPNA2608]

(Commit title should've been surreal-engine: init at 0-unstable-2024-11-08, I've just gone ahead & fixed this in the merge to avoid another back n forth)