Tracking issue: Linux port using dxvk / gallium nine

[q4a]

Here will be current lists of needed fixes for Linux port of engine using dxvk.

Current source code of dxvk port is here: https://github.com/storm-devs/storm-engine/tree/dxvk

Started status: there is 113 #ifdef marked with FIX_LINUX which should be replaces with proper implementations, but engine already build fine with CLANG 11 and partially show main menu background

Current status:

113 marked FIX_LINUX tasks (from most important to less):

• 19 for FIX_LINUX ID3DXEffect. Pushed workaround: CTechnique: + precompiled shaders. Check FIX_LINUX Effects
• 01 for FIX_LINUX D3DXCreateTextureFromFileA: is it used at least somehow?
• 01 for FIX_LINUX D3DXCreateTextureFromFileInMemoryEx: unused, created PR: Remove unused code #347
• 04 for FIX_LINUX HBITMAP: unused, created PR: Remove unused code #347
• 04 for FIX_LINUX Screenshot: I hope that SDL has some useful funcs for taking screenshots
• 02 for FIX_LINUX ReleaseDC: not sure, but may be important
• 04 for FIX_LINUX VirtualKey: There is dilemma: we can keep VirtualKeys for c-files in PROGRAM folder to maintain old mods or replace all VirtualKeys, because less code == less errors. Current decision - keep VirtualKeys: Replace GetKeyState and GetAsyncKeyState with core functions #328
• 04 for FIX_LINUX HINSTANCE: for correct handling window and other: Linux related changes part 1 #366
• 04 for FIX_LINUX Cursor: port all Cursor stuff to SDL
• 10 for FIX_LINUX GetTickCount: I hope that we will find old branch/changes where is this changes already made
• 03 for FIX_LINUX sentry_options: few Windows only direct calls to crashpad_handler.exe
• 01 for FIX_LINUX WideCharToMultiByte: some old conversation
• 01 for FIX_LINUX MultiByteToWideChar: some old conversation
• 08 for FIX_LINUX DirectXMath: one DirectXMath call in AVX/SSE2 ifdef. I created quick workaround (this increased 08 for FIX_LINUX DirectXMath from 2 to 8), but I'll need to replace it with this lib: https://github.com/nfrechette/rtm
• 04 for FIX_LINUX DxErr.h: few error handling, may be already in DXVK?
• 02 for FIX_LINUX _flushall: not sure
• 01 for FIX_LINUX RDTSC_*: need to test SDL_GetPerformanceCounter and use it (if it passes tests)
• 10 for FIX_LINUX __debugbreak: need to create one place with debugbreak for all platforms/archs
• 02 for FIX_LINUX GetWindowRect: I used SDL functions for Linux part, but it did not work well in Windows (why?)
• 08 for FIX_LINUX ddraw.h and amstream.h: It need only for play avi into and other videos. Will require to use ffmpeg or something like that
• 01 for FIX_LINUX ExitProcess: I used "exit(0)" for Linux port, but it is part of SailorsEditor, which should be moved from engine IMHO.
• 02 for FIX_LINUX SHGetKnownFolderPath: I used SDL_GetPrefPath for Linux port. Should I keep SHGetKnownFolderPath for compatibility?
• 02 for FIX_LINUX 7za.exe: one direct call to 7za.exe
• 01 for FIX_LINUX CreateEventA: not sure if I need to prevent multiple instances of game
• 03 for FIX_LINUX Beep: I'm fine if my pc speaker will not beep when I'm playng game)
• 17 for FIX_LINUX s_debug.h: part for debugging user mods in compiler.cpp and s_dbg_*

[q4a]

FIX_LINUX this flag currently reflects parts of the code that could be improved, but not actual problems/bugs with the game. Currently gaming on Linux should be very close to gaming on Windows, except for this list:

• Fixed. No sound.
• No debug / cheat menu by pressing F11 on LXQt / openbox. Not bug, but LXQt / openbox feature.

1. You need to replace keybind key="F11" with another key in ~/.config/openbox/*rc.xml. I replaced with keybind key="A-Return"
2. Then execute openbox --reconfigure in terminal.

• Fixed. Wrong (zero) sailors' paths on the ship. Top priority. It should be in Sailors refactoring #378
• Fixed. Flickering shadow from the main character. Top priority.
• No intro video. Low priority. This is require rewriting 1 class (libs/xinterface/src/aviplayer) and adding a dependency on libmpg123 or ffmpeg.
• Fix LifecycleDiagnosticsService and check if message "Unable to initialize lifecycle service!" still happens. Mid priority.
• No screenshots by pressing F8. Mid priority. I think that I can fix it soon, but may be with only 1 format (like png or jpeg).
• Double shadows on bridge:
  

[q4a]

Another thing about building on Ubuntu 22.04 or similar: if you have libtbb in your system, then you need to install libtbb2-dev (Source Package: tbb, not onetbb) + you need to add it in src/apps/engine/CMakeLists.txt

diff --git a/src/apps/engine/CMakeLists.txt b/src/apps/engine/CMakeLists.txt
index b710cc31..1cc1cb7c 100644
--- a/src/apps/engine/CMakeLists.txt
+++ b/src/apps/engine/CMakeLists.txt
@@ -13,7 +13,10 @@ set(SYSTEM_DEPS
     "winhttp"
 )
 else()
-set(SYSTEM_DEPS "ffi")
+set(SYSTEM_DEPS
+    "ffi"
+    "tbb"
+)
 endif()
 
 STORM_SETUP(

[q4a]

Linux ARM build

Warning: you can only build Storm engine on ARM, it's not running yet

It's possible to build Storm engine on ARM with this repo: https://github.com/q4a/storm-engine/tree/nine-arm
Build is tricky:

1. You need to install all dependencies from https://github.com/q4a/storm-engine/blob/nine-arm/.github/workflows/ci_linux.yml
   Currently it's:
   sudo apt-get update && sudo apt-get install meson libtbb-dev libstdc++-10-dev libvulkan-dev glslang-tools libsdl2-dev libgl-dev libegl-dev
   PS you can remove libtbb-dev if you don't want to use it, but you will need to remove it from src/apps/engine/CMakeLists.txt - SYSTEM_DEPS.
2. Use clang-14 for build. DON"T USE clang-11. It will crash:

PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.    Program arguments: /usr/bin/clang++-11 ...

3. At some point of running cmake will fail because of libfmod - there is no arm version of libfmod in conan. You need to put libs from fmod-armhf.tar.gz to your bin folder and run cmake angain.

There is 2 variant of build:

• Using DXVK Native. Currently ARM SBC has no good Vulkan drivers, so this is for testing using lavapipe/llvmpipe (cpu only/software implementation of vulkan driver).
• Using Gallium Nine - native D3D9 api in mesa. To use it set option(STORM_MESA_NINE ON) in storm-engine/CMakeLists.txt
  Booth is not working right now.

Running and debugging

I'm currently using:

rk3288 - 32-bit Quad-core ARM Cortex-A17 with Mali-T760 MP4
Ubuntu 22.04 armfh (Armbian)
clang version 14.0.6-1~oibaf~j
Qt Creator 6.0.2
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1

DONT'T use LLDB on ARM - it's failing.
There is small test for engine test/debug, that show only simple menu: https://github.com/q4a/storm-demo
If you will test TEHO - compare/check tree of files:
sd-teho-tree.txt.tar.gz

[q4a]

Current ARM Linux status. I have 2 ARM SoC.
Low spec 32-bit ARM SoC (rk3288) with 2GB dual channel LPDDR3 - Asus Tinker Board (from april 2017):
https://en.wikipedia.org/wiki/Asus_Tinker_Board

High spec 64-bit ARM SoC (rk3588) with 16GB dual channel LPDDR4x - Radxa Rock5 5B (from october 2022):
https://wiki.radxa.com/Rock5/5B

[q4a]

Compare perfomance DXVK vs Gallium Nine on x64 Linux.
"Release" version builded from source code:
https://github.com/q4a/storm-engine/tree/fadc0a39abb9e3544ddb3afa408360d565bcce05
gcc version 11.3.0 (Ubuntu 11.3.0-1ubuntu1~22.04)
Save file: CumanaStreets.tar.gz
Game config: engine.ini_and_options.tar.gz
Script changes - only program/utils.c (I forgot about them when I tested):

-       SendMessage(objGrass,"lffffff",42666, fScale, fW,fH, fMinDist,fMaxDist, fMinLod); // #define MSG_GRASS_SET_PARAM 42666
+       SendMessage(objGrass,"lffffff",42666, fScale, fW,fH, fMinDist,1000, fMinLod); // #define MSG_GRASS_SET_PARAM 42666

Some HW info from neofetch:

OS: Lubuntu 22.04.1 LTS x86_64                                                                                                                    
Kernel: 5.15.0-60-generic                                                                                                                         
DE: LXQt 1.2.0                                                                                                                                    
WM: Openbox                                                                                                                                       
CPU: AMD A6-5200 APU (4) @ 2.000GHz                                                                                                               
GPU: AMD ATI Radeon HD 8400 / R3 Series                                                                                                           
Memory: 3705MiB / 14935MiB

glxinfo -B

name of display: :0
display: :0  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: AMD (0x1002)
    Device: KABINI (kabini, LLVM 15.0.7, DRM 3.42, 5.15.0-60-generic) (0x9830)
    Version: 22.3.5
    Accelerated: yes
[CumanaStreets.tar.gz](https://github.com/storm-devs/storm-engine/files/10781560/CumanaStreets.tar.gz)

    Video memory: 1024MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 4.6
    Max compat profile version: 4.6
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.2
Memory info (GL_ATI_meminfo):
    VBO free memory - total: 738 MB, largest block: 738 MB
    VBO free aux. memory - total: 3023 MB, largest block: 3023 MB
    Texture free memory - total: 738 MB, largest block: 738 MB
    Texture free aux. memory - total: 3023 MB, largest block: 3023 MB
    Renderbuffer free memory - total: 738 MB, largest block: 738 MB
    Renderbuffer free aux. memory - total: 3023 MB, largest block: 3023 MB
Memory info (GL_NVX_gpu_memory_info):
    Dedicated video memory: 1024 MB
    Total available memory: 4096 MB
    Currently available dedicated video memory: 738 MB
OpenGL vendor string: AMD
OpenGL renderer string: KABINI (kabini, LLVM 15.0.7, DRM 3.42, 5.15.0-60-generic)
OpenGL core profile version string: 4.6 (Core Profile) Mesa 22.3.5 - kisak-mesa PPA
OpenGL core profile shading language version string: 4.60
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 4.6 (Compatibility Profile) Mesa 22.3.5 - kisak-mesa PPA
OpenGL shading language version string: 4.60
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile

OpenGL ES profile version string: OpenGL ES 3.2 Mesa 22.3.5 - kisak-mesa PPA
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

DXVK - ​​on land at the port: 25 fps

Nine - on land at the port: 38 fps

DXVK - at sea near the port: 30 fps

Nine - at sea near the port (with tree bug): 50 fps

DXVK - on worldmap: 141 fps

Nine - on worldmap: 145 fps