error loading module in lua (socket.dll) [2.9]

[Berserker66]

Summary

dll load failed in lua

Repro

Loading either https://github.com/alttpo/sni/tree/main/lua or https://github.com/ArchipelagoMW/Archipelago/tree/main/data/lua lua files will die trying to load socket starting with BizHawk 2.9

Detailed instructions:
https://archipelago.gg/tutorial/Adventure/setup/en For Adventure Randomizer
https://archipelago.gg/tutorial/Donkey%20Kong%20Country%203/setup/en for Donkey Kong Country 3 Randomizer
https://archipelago.gg/tutorial/Final%20Fantasy/multiworld/en for Final Fantasy I Randomizer
https://archipelago.gg/tutorial/The%20Legend%20of%20Zelda/multiworld/en for The Legend of Zelda Randomizer
https://archipelago.gg/tutorial/A%20Link%20to%20the%20Past/multiworld/en For The Legend of Zelda: A Link to the Past Randomizer
https://archipelago.gg/tutorial/Links%20Awakening%20DX/setup/en for The Legend of Zelda: Link's Awakening DX Randomizer
https://archipelago.gg/tutorial/Lufia%20II%20Ancient%20Cave/setup/en for Lufia 2: Ancient Cave Randomizer
https://archipelago.gg/tutorial/Ocarina%20of%20Time/setup/en for The Legend of Zelda: Ocarina of Time Randomizer
https://archipelago.gg/tutorial/Pokemon%20Red%20and%20Blue/setup/en for POKÉMON Red and Blue Randomizer
https://archipelago.gg/tutorial/Secret%20of%20Evermore/multiworld/en for Secret of Evermore Randomizer
https://archipelago.gg/tutorial/SMZ3/multiworld/en for SMZ3 Romhack Randomizer
https://archipelago.gg/tutorial/Super%20Mario%20World/setup/en for Super Mario World Randomizer
https://archipelago.gg/tutorial/Super%20Metroid/multiworld/en for Super Metroid Randomizer

Output

(For SNI:)

Wrong Lua Core. Found NLua+Lua, was expecting LuaInterface. 
Please go to Config -> Customize -> Advanced and select Lua+LuaInterface.
Once set, restart Bizhawk.
NLua.Exceptions.LuaScriptException: error loading module 'socket.core' from file '.\socket.dll':
	The specified module could not be found.

Alternatively, if I paste the dll file into the dll folder, the entire program cashes instantly.

Host env.

• BizHawk 2.9; Win10 Pro 22H2; AMD/AMD
  Likely all platforms though.

Screenshots

Notes

This affects at least 13 Randomizers, rendering them either only locally playable (no multiworld support) to entirely unplayable, depending on their Server reliance. We're looking at either getting this working again in BizHawk, which seems to be on you guys, or updating all these guides to say BizHawk 2.8 or lower

see also #1077

[CasualPokePlayer]

Keep in mind we've updated lua to 5.4 (anything 5.1 only will no longer work). There is also only one "lua core" now, NLua+Lua (which should load dlls just fine, unlike KopiLua). You may need to do some updates to target 5.4 (googling the issue also pops up something similar: NLua/NLua#350 which is fixed by compiling the dll with the correct lua version)

[Berserker66]

The same DLL is also loaded into other emulators, which target different lua versions, so this would complicate things quite a bit.

[CasualPokePlayer]

Do any of them use lua 5.3 or lua 5.4?

EDIT: Also, the fact it mentions ".\socket.dll" indicates it did in fact find that dll and loaded up fine. It just can't find the module (due to some sort of breakage between lua versions so it can't use it, it needs a new version). I don't think this is really resolvable by us without us just putting in an old lua version and trying to be compatible with it (which is problematic due to the fact the upgrade has allowed us to do things which were not possible with the old LuaInterface+Lua 5.1)

EDIT EDIT: On that, does using a more up to date socket.dll actually result in the opposite problem of lua 5.1 emus not being usable? The breakage might only be one way.

[Berserker66]

A popular choice that also loads this dll is https://github.com/gocha/snes9x-rr/

[CasualPokePlayer]

It uses lua 5.1, so that highly suggest it is ultimately the fault of the dll not supporting lua 5.4.

[YoshiRulz]

See #2951 (comment) for workaround.

[CasualPokePlayer]

See #2951 (comment) for workaround.

I don't believe this is the issue here, I'm pretty sure that's a Linux only issue. This here is just a "this is a 5.1 dll and 5.4 requires some more stuff".

[CasualPokePlayer]

Looking closer into this, trying out a luasocket compiled with specific support for lua 5.4 seems to not work? There's probably more at play here.

[CasualPokePlayer]

From what I can tell, in 2.8 and below, the dll you guys provide simply was never actually used, as we appeared to ship luasockets internally ourselves.

Considering the bullshit just to get the normal sockets stuff to work I think just compiling Lua ourselves and shipping lua sockets could be better. Not sure what this means for Linux. It may be still better to leave them to figure things out due to the amount of different paths lua libs could be located it and it appears to at least work when compile correctly (while Windows is being a very interesting problem child here).

[YoshiRulz]

We already provide a sockets library, script authors just refuse to use it. The protocol changed 2 years ago but it's still pretty stable. With the exception of Crowd Control, which IIRC re-used the same script for multiple emulators, there's really no reason not to use the provided libraries when writing scripts for EmuHawk. Hence I want to call this #wontfix.

[CasualPokePlayer]

That exception you mention would apply here, since they're reusing the same script with other emus (snes-rr, etc). And those CrowdControl people will likely also experience breakage due to this change.

[CasualPokePlayer]

Also, if our scripts actually do have feature parity, should that then mean it should be possible to create a wrapper lua file which uses our API but exposes the more popular luasocket API?

[YoshiRulz]

Doubt it

[CasualPokePlayer]

Ok figured out the issue with using the newer luasocket, it seemingly requires that the socket/core.dll (or alternatively? the socket.dll) be placed next to the BizHawk .exe rather than next to the lua, which makes little sense here since the error does indicate it found the dll but it wasn't able to properly load it, somehow?

[CasualPokePlayer]

This seems to be something weird in package.cpath, if I replace it with an absolute path to the .dll, it works fine. If it's a relative path, it does not work fine. I think this is related to our "sandboxing" where we change the current directory to the lua path, so lua gets confused since relative pathing works different between finding files (uses cwd) and actually loading them as dlls (relative to .exe, regardless of cwd)

I think a solution here is rather replacing cpath on our side (similar to what we do with path on Linux), and make first point at our Lua dir (with an absolute directory, no relative pathery), and let script authors slap lua modules in the Lua dir. Also, probably remove the relative path ;.\?.dll since that can never work with our current directory nonsense.

[CasualPokePlayer]

With the above your scripts should now "work". Linux users will still need to install lua modules more or less "manually" (not so much a problem for them really?), but Windows users should have luasockets work out of the box like before.

Also, any other lua modules cannot rely on relative pathing like .lua files can, but this shouldn't be much an issue (just plop the dll in the lua folder and it'll work, probably)

[zig-for]

@CasualPokePlayer How was the DLL in 5f0683c compiled? Attempting to compile my own 5.4 compatible luasockets DLL and invoke it results in random crashes inside GC functions - but your DLL appears to be ok.

[CasualPokePlayer]

Did you statically or dynamically link in lua54? You should be dynamically linking it so it doesn't try to create its own lua VM.

[zig-for]

I dynamically linked it against lua54.lib (I think...) - ironically enough, linking statically seems to Just Work but makes me uncomfortable.

[zig-for]

It's possible that I'm not properly linking, but I'm unsure what I would need to change in the solution/project file to change that.

[CasualPokePlayer]

You should be using luarocks to compile it, not any sln/project file.

[zig-for]

I tried with luarocks also, no dice. Additionally, https://github.com/pkulchenko/ZeroBraneStudio/blob/lua54/bin/clibs54/socket/core.dll fails the same way.

[CasualPokePlayer]

luarocks is how I compiled it. Maybe make sure you have the exact same dll as BizHawk (I made sure it used the same dll here).

[zig-for]

Just to ensure I'm doing the exact same thing - where did you get your .lib and headers from?

[CasualPokePlayer]

I just took them from https://sourceforge.net/projects/luabinaries/files/5.4.2 (the Win64_dll16 package in dynamic libraries) which seemed to work. I swapped in BizHawk's lua54 dll and it compiled and ran fine.

[zig-for]

I was unaware the DLL was used at all when compiling.

[zig-for]

I think I had the wrong package, hooray. Will download Win64_dll16 and try.

[zig-for]

@CasualPokePlayer that appears to fix the issue, thanks so much for the help.