Allow creating arbitrary data structures in GDScript

[aaronfranke]

Describe the project you are working on: Games that require data structures beyond the built-in types. For example, Vector4 (discussion in #629), Basis25D, complex numbers, and as of writing, Vector2i/3i (will be in core in 4.0).

Describe the problem or limitation you are having in your project: Currently, this is not possible in GDScript. You can still do the same math, but not as neatly.

Describe how this feature / enhancement will help you overcome this problem or limitation:

Arbitrary data structures in GDScript would mean that core not having any given type would not be nearly as big of a deal. It would mean that users can simply download a vector4.gd file and then they would have Vector4 available in their project, without it needing to be part of the engine.

Show a mock up screenshots/video or a flow diagram explaining how your proposal will work:

Imagine any of the above C# examples, but in GDScript.

Describe implementation detail for your proposal (in code), if possible:

There are many things that would need to be solved first in GDScript to achieve this.

• Structures need to reference other instances of itself, so GDScript needs to not have issues with cyclic errors: Some usages of class_name can produce cyclic errors unexpectedly godot#21461

• To allow for things like Vector4.ONE, structures need to have static const or readonly static var or similar depending on how it's implemented. 'static var' to go with 'static func' godot#6840

• Structures should have operator overloading, so you can specify things like multiplying a Vector4 times a float, equality checks with ==, and array accessing such that my_vector4_instance[0] == my_vector4_instance.x. Operator Overloading in GDScript godot#23488 (The issue is closed because string conversion overloading was added, but the more general issue mentioned here isn't solved).

• Of course, structures themselves need to be added. These should behave like C# or Godot's built-in types, pass-by-value (copies), no references/pointers. They would be declared with a keyword similar to class_name (struct? struct_name?), but do not extend anything. Icons might still be useful if GDScript structs were usable within VisualScript or something.

So, right now this would be a lot of work to implement, but it's certainly an idea worth discussing.

If this enhancement will not be used often, can it be worked around with a few lines of script?:

You can easily do it in C#, but not in GDScript. Of course one option is to just tell everyone to use C# if they need custom data structures, but I don't think that's an ideal solution.

Is there a reason why this should be core and not an add-on in the asset library?:

The point of this proposal is to keep bloat out of core, by allowing users to make their own structures.

Kind-of related: #209 and #22

[Calinou]

• structures need to have static const or readonly static var or similar depending on how it's implemented. 'static var' to go with 'static func' godot#6840

Last time I tried, constants can always be referenced statically in GDScript.

[Jummit]

I wouldn't add a "struct" type for this. Just do something like the custom iterators, just with operators. Then you could still use class_name to make it accessible everywhere.
For constants, just use const.

[vnen]

• Structures should have operator overloading, so you can specify things like multiplying a Vector4 times a float, equality checks with ==, and array accessing such that my_vector4_instance[0] == my_vector4_instance.x.

Main problem with this is that GDScript is dynamically typed. So we may not know what method to call until it's the time to call it. So we would need to make the check on every operation if there's a valid overload, which would make regular operations slower. I can't really find a solution for this that wouldn't impact performance for the operators without overloads.

Passing by value instead of reference is a similar issue: how can we tell if a particular variable should be passed by value or by reference? But the bigger impact would probably be creating a new GDScript instance every time you pass the value around.

For this to work, one solution would be creating a new variant type for STRUCT (as a complement to the current OBJECT) that would behave differently. That would still cost some runtime performance, but would impact less the other operations. Changes in Variant should be very well-thought however.

[SuperDIMMaX]

Uhh... i don`t know all... but how about LuaJIT?
See link...

Motivating Example: Using C Data Structures
The FFI library allows you to create and access C data structures. Of course the main use for this is for interfacing with C functions. But they can be used stand-alone, too.

maybe it be usefull for create similar. (How FFI works)
In love2d i use FFI C Struct . for create 10000000(many... don`t remember) Vec2i... very, very powerfull effect. 1 min > 15 sec. 200mb ram > 40mb...

[Calinou]

@SuperDIMMaX To my knowledge, we have no plans to integrate something like that.

[SuperDIMMaX]

@SuperDIMMaX To my knowledge, we have no plans to integrate something like that.

Not Lua, something like how FFI works. Maybe you can dinamic create C struct or create samiliar effect for perfomance, not need all feature of JIT and FFI, just for struct.
Or... LLVM...

[follower]

With regard to libffi specifically, for some reason the related issue isn't found by GitHub search but there is an existing related issue: "Include libffi #3943"

It includes a link to "Foreigner: a foreign function interface library for Godot 3.1 (WIP)" by @and3rson a.k.aand3rson/foreigner which is implemented via libffi: https://github.com/and3rson/foreigner

[aaronfranke]

@vnen: Main problem with this is that GDScript is dynamically typed. So we may not know what method to call until it's the time to call it. So we would need to make the check on every operation if there's a valid overload, which would make regular operations slower.

How does it currently work with Vector3 etc? Those built-in types have their own code for things like multiplication and array access, so why can't we allow user-defined types to have the same behavior?

[vnen]

How does it currently work with Vector3 etc? Those built-in types have their own code for things like multiplication and array access, so why can't we allow user-defined types to have the same behavior?

Those aren't free either but at least they are part of the same switch to detect the operand types, and everything is hardcoded on C++ (they don't need to look up for script functions for instance).

We could potentially check for overloads on objects, since they don't work with regular operators in general so it could have an extra check for script functions before failing. But an equality operator overload would be too much IMO because it's just a pointer comparison in objects, so they would be penalized with an extra check (that most of the time you don't need).

[aaronfranke]

Closing as a won't do, but note that some of the linked issues tracking specific limitations of trying to use classes for this use case have been solved. It's acceptable to have to use a manually-defined equality method instead of the == operator (for floating-point structs, is_equal_approx is often preferred anyway). Users with more advanced use cases that significantly benefit from lightweight structs can instead use C# or C++.

[aaronfranke]

Superseded by #7329