gmod-lua-threaded

WIP
A side project I work on to learn more about Lua, and It's API for my gmod-lua-shared project

This project aims to provide all necessary functions to run Lua on different threads.
The Goal is to be able to create a ILuaInterface with all of Gmod's libraries, functions and classes.

You can download the last working version(which exist for 1 month) which was tested from here
If you want the newest version, just create a fork and run the Build Project (Artifact) workflow

If you have any Issue, Request or Question you can always open an Issue for it or ask me in Discord

NOTE:
This only works on Linux because on Windows creating a ILuaInterface on another thread currently crashes the game.

Current Status

Currently my focus lies on HolyLib.
When I get to a version like 1.0 I'll get back to this and I'll probably redo most of it.

What is implemented

• Shared Table
• Better Error handling(It loved to crash on any Lua error.)

Libraries

• bit
• coroutine
• os
• package
• jit
• timer
• hammer (useful for Windows later)
• engine
• gameevent
• file
• debug (Also setlocal, setupvalue, upvalueid and upvaluejoin exists)

Unfinished/Untested

• physenv (Mostly implemented)
• • physenv.AddSurfaceData
• • physenv.GetLastSimulationTime
• resource (untested!)
• system (untested!)
• umsg (untested!)
• gmod (untested!)
• game (untested!)
• • game.AddDecal
• • game.CleanUpMap
• • game.GetGlobalCounter
• • game.GetGlobalState
• • game.GetMapNext
• • game.GetSkillLevel
• • game.GetTimeScale
• • game.GetWorld
• • game.KickID
• • game.LoadNextMap
• • game.MountGMA
• • game.RemoveRagdolls
• • game.SetGlobalCounter
• • game.SetGlobalState
• • game.SetSkillLevel
• • game.SetTimeScale
• • game.StartSpot

Classes

• Angle (untested)
• Vector (untested)

Current Issues

• resource.AddFile & resource.AddSingleFile don't seem to work.
• Idk how gmod implemented (physenv.GetLastSimulationTime)
• Look into ILuaInterface:PushVector being broken.

ToDo/Testing

• Created ILuaInterface should load all Gmod scripts/allow one to do so
• Change ILuaAction to use Enums instead of strings.
• Documentation
• Add all Gmod Enums
• Recreate all Gmod libraries
• • Implement sql.Query. (Fixes sql and cookie library)
• Add a new way to send data across threads. Maybe something like net messages but for threads?
• Look again into Error handling. Maybe show which file had an error?
• Improve Performance of LuaThreaded.SetValue and LuaThreaded.GetValue

Bugs

• Fix a memory leak related to LuaThreaded.SetValue not reusing the original key and value.
• File:Seek seems to have some issues?
• Requiring this module befor InitPostEntity was called causes a bunch of weird bugs. See LuaThreaded.ReadyThreads() for the implemented workaround.

I listed all Libraries below that are serverside. Regardless if they are implemented in lua or in c++.

• • Global(functions)
• • ai
• • ai_schedule
• • ai_task
• • baseclass
• • cleanup
• • concommand
• • constraint
• • construct
• • cookie
• • cvars
• • debugoverlay
• • drive?
• • duplicator
• • effects
• • ents
• • gamemode
• • gmsave
• • hook
• • http
• • list
• • math
• • motionsensor
• • navmesh
• • net
• • numpad
• • player
• • player_manager
• • properties
• • saverestore
• • scripted_ents
• • sound
• • sql
• • string
• • table
• • team
• • undo
• • usermessage
• • utf8
• • util
• • weapons
• • widgets
• Recreate all Gmod metatable functions
• Cleanup the code
• Make this compatible for x86-64
• Look again into Windows

API

Example usage:

require("lua_threaded")
iFace = LuaThreaded.CreateInterface() -- Creates an Interface.
iFace:InitGmod() -- Adds all Libraries and Classes
iFace:Autorun() -- Runs all Autorun scripts.

LuaThreaded

LuaThreaded.GetAllInterfaces()

Returns a table containing all Interfaces created.
[ID] = ILuaInterface (Class)

LuaThreaded.GetInterface(number ID)

Returns the Interface with the given ID.

Args:

1. ID of the given Interface

LuaThreaded.CreateInterface(boolean not_threaded = false)

Creates a new Interface.

Args:

1. If the Created Interface should run on the main thread or not.

Rets:

1. The newly created Interface.

LuaThreaded.CloseInterface(ILuaInterface interface)

Closes the given Interface.

Args:

1. The Interface to close.

LuaThreaded.Msg(string msg)

Prints a Message.
This function may be removed in the future!

Args:

1. The string to print.

LuaThreaded.GetTable()

Returns the Shared table.
NOTE: You need to call this function to get an updated version!

LuaThreaded.SetValue(any key, any value)

Sets a Value inside the shared table.

LuaThreaded.GetValue(any key)

Returns only the specified value from the shared table.

Args:

1. The Key inside the Table
2. The Value to set. Allowed Types: Number, Bool, String, Vector, Angle and Table

LuaThreaded.ReadyThreads()

Enables all threads. Until then, all Threads will just wait and do nothing. You can still create Threads and give them tasks, but they won't do them until this function is called.

This is a workaround for a crash if you try to do stuff way too soon. This needs to be called if you require the module Inside or after InitPostEntity is called.

ILuaInterface

ILuaInterface:RunString(string code)

Runs the given code

Args:

1. Code to run.

ILuaInterface:InitClasses()

Initializes all Classes.
This function may be removed in the future!

ILuaInterface:InitLibraries()

Initializes all Libraries.
This function may be removed in the future!

ILuaInterface:InitEnums()

Initializes all Enums.
This function may be removed in the future!

ILuaInterface:InitGmod()

Initializes everything for Gmod.
Internally calls InitClasses, InitLibraries and InitEnums.

ILuaInterface:LoadFunction(string sig)

Loads a function into the Interface.
NOTE: This is unsafe and can cause random crashes.
This function will be removed in the future!

Args:

1. Signature of the Function to load.

ILuaInterface:Autorun()

Loads everything like autorun does.
NOTE: It currently loves to crash. Just call RunFile with init.lua
Update: It seems like it became stable. Could be because I stopped breaking the stack.

Loads:

• includes/init.lua
• autorun/server/*.lua
• autorun/server/sensorbones/*.lua

ILuaInterface:RunFile(string path)

Runs the given File.
This function could be renamed in the future!

ILuaInterface:RunHook(string name, args ...)

Runs the given Hook.
(Calls hook.Run with the name and args)

ILuaInterface:Lock()

Locks the Thread / doesn't allow it to execute any task.
Also waits until the Thread is finished executing all tasks.

ILuaInterface:Unlock()

Unlocks the Thread / allows it to execute all tasks

Args:

1. Path to the file.
   NOTE: This will use the GAME search path!

Structure

CLuaGameCallback class

Creates the CLuaGameCallback class and implements all functions.

Files:

• CLuaGameCallback.h
• CLuaGameCallback.cpp

detours

Contains all signatures and functions we need from Gmod.

Files:

• detours.h
• detours.cpp

lua_ILuaInterface

Contains all ILuaInterface metatable functions.

Files:

• lua_ILuaInterface.h
• lua_ILuaInterface.cpp

lua_LuaThread

Contains our LuaThread table and all its functions.

Files:

• lua_LuaThread.h
• lua_LuaThread.cpp

lua_Enums

Contains our LuaThread table and all its functions.

Files:

• lua_Enums.h
• lua_Enums.cpp

library_Global

Contains all implemented Global functions.
Status: Unfinished

Files:

• library_Global.h
• library_Global.cpp

library_net

Contains all implemented net functions.
Status: Unfinished

Files:

• library_net.h
• library_net.cpp

class_vector

Contains all implemented Vector functions.
Status: Unfinished + Broken

Files:

• class_vector.h
• class_vector.cpp

lua_utils

Contains important functions PushValue, FillValue and so on.

Files:

• ILuaConVars.h (Needed for library_Global)
• lua_utils.h
• lua_utils.cpp

lua_threaded

Contains module base and manages everything for Init and Shutdown.

Files:

• lua_threaded.h
• lua_threaded.cpp

Some notes

Entities

We should block entity creation while PackEntities_Normal is running because else we might cause engine errors.

NW

For the NW System I'm going to need CLuaNetworkedVars.
CLuaNetworkedVars::FindEntityVar -- Returns a LuaNetworkedVar_t. GetNW* and GetGlobal functions are gonna require this.
CLuaNetworkedVars::SetNetworkedVar -- Give it everything it needs and it'll do the rest. Probably going to need to add a mutex to it.
Als going to need LuaNetworkedVarEnts_t and going to recreate BuildNetworkVarTable & BuildEntityNetworkVarTable
This could become useful

NW2

This is going to be pain.
Going to need g_pServerWorldTable, SetDataTableVar, IGMODDataTable, CGMODVariant.
Already got: IGMODDataTable
ToDo: Find out what the CGMODVariant is.
NOTE: Found out.

Other Stuff

Gmod umsg names:

• "LuaCmd"
• "SWEPCmd"
• "AmmoPickup"
• "WeaponPickup"
• "BreakModel"
• "CheapBreakModel"

Fix file:Read not working in mode wb.

This seems to be caused by something internal in the filesystem which also affects gmod.