[3.x] Allow exporting custom resources from/to any scripting language (GDScript, VisualScript, C#, NativeScript, PluginScript)

[willnationsdev]

This PR adds the ability to export custom resources and arrays of custom resources of any language's type from an exported property defined in any language.

Closes godotengine/godot-proposals#18, at least, in Godot 3.2.

This builds on top of the commit in #44796, so that would have to be merged first. This now includes the PR content of #44796 as of the 3.x rebase. For a detailed breakdown of the features commit-by-commit, see the 4.0 version at #48201.

This is a continuation of work started by me and @JFonS and would replace his 3.2 PR which deals exclusively with GDScript (#41264).

It includes editor changes to support script classes as valid resource exports.

Because the definition of PropertyInfo objects is manual in VisualScript, NativeScript, and PluginScript, the feature works out-of-the-box for those languages.

GDScript support was already implemented by JFonS's PR, but I've incorporated those changes and more.

C# support is added as well. The [Export] attribute now supports the following forms...

// Infer hint and hint string (no changes)
[Export]

// Manually define hint and hint string. Only applies if valid hint cannot be calculated (no changes).
[Export(PropertyHint.None, "")]

// Applies custom resource name if applicable to the member's data type (new!).
// Only necessary when exporting a Resource or List<Resource> for a non-C# resource type.
[Export("ClassName")]

Demo Project:
Demo_GP22_3_2_Resources.zip

Screenshots:

GDScript usage, exported properties contracted:

GDScript usage, exported properties expanded:

Note I also have export var gd_data: GDData syntax supported as well (so you can infer the export type from the GDScript type hint).

C# usage doing the same thing:

[vitorbalbio]

This qualifies Godot as a tier one engine for data intensive games. Congrats @willnationsdev, @JFonS and all devs involved.

[willnationsdev]

@vitorbalbio I will say that, for simulation-heavy games that also require a lot of data, this alone won't really be enough. This is merely a big usability improvement for general-purpose games that want data-driven behavior. To get a significant runtime performance improvement (i.e. processing over large amounts of data), you'd have to resort to writing module code with custom CPU cache-efficient code with your own Server implementations, etc.

@AndreaCatania is doing great work in this domain as he is writing a module that introduces an ECS framework that can run alongside the Godot Engine runtime and cooperates with the scripting API. He has seen major performance improvements for any simulation logic he moves over to his module's systems, so if you need a game like that, keep an eye on his development. He has multiple customizable system pipelines defined with editor tools and script code. Time will tell if the developers ever decide to actually integrate his ECS module with the core engine (they absolutely should though - or at least, something like it).

[vitorbalbio]

@vitorbalbio I will say that, for simulation-heavy games that also require a lot of data, this alone won't really be enough. This is merely a big usability improvement for general-purpose games that want data-driven behavior. To get a significant runtime performance improvement (i.e. processing over large amounts of data), you'd have to resort to writing module code with custom CPU cache-efficient code with your own Server implementations, etc.

@AndreaCatania is doing great work in this domain as he is writing a module that introduces an ECS framework that can run alongside the Godot Engine runtime and cooperates with the scripting API. He has seen major performance improvements for any simulation logic he moves over to his module's systems, so if you need a game like that, keep an eye on his development. He has multiple customizable system pipelines defined with editor tools and script code. Time will tell if the developers ever decide to actually integrate his ECS module with the core engine (they absolutely should though).

I mean more like RPGs and Tactical games that you need to define lots of data/status for large amount of items, objects and so on. The same use cases where Scripted Objects of Unity shines.

[willnationsdev]

@vitorbalbio Ah, yeah. I really would like to see a proper spreadsheet class added to the engine/Editor to simplify working with tabular data with full static typing and all the custom editors from the Editor integrated out-of-the-box. godotengine/godot-proposals#13 That would make it much easier to just create large amounts of data for projects like those.

[willnationsdev]

Fixed a bug where variables with type hints for GDScript inner classes were mistakenly not parsing to the end of the line anymore.

[willnationsdev]

Note that someone on Reddit reported a bug with this where referencing an inner class within the same file using a relative symbol name doesn't work (even though I fixed it so the fully-qualified name in other scripts works). This is only in GDScript.

Edit: I've presumably pushed a fix for this (works in my exported project, and the Reddit user reported that he no longer has parse-time issues), so as far as I know, the problem is resolved. Please let me know if anyone experiences any issues with this.

[willnationsdev]

Fixed a bug where script classes that were part of a nested res://addons/* directory weren't showing up in the CreateDialog unless they were part of an enabled plugin. I've now changed the CreateDialog code to explicitly check for the presence of a plugin.cfg file within the directory, and if there isn't one, it will continue to display the script class in the tree of types.

Also fixed a small typo in an identifier name.

[willnationsdev]

Fixed a bug where, once you had assigned a script class to a node, there was no way to remove the script class from the node because the Change Type operation would preserve the script property value even if the node you were "changing type" to was explicitly the instance base type.

Edit: Amended the fix for this so that it can also handle switching to different engine or script classes entirely rather than just a different engine class.

[willnationsdev]

Added a warning message when explicitly removing a built-in script. Built-in scripts' inheritance cannot be determined because they are never compiled, ergo their get_base_script() method won't produce a result (at least, in GDScript). Added a WARN_PRINT_ONCE so that users are not alarmed/confused as to why the entire script was removed rather than removing just down to the script class like it otherwise should (mimicking the custom type feature).

[Ternvein]

I think it's totally needed for 3.5. It's been ages since first pull request with fixes for this issue, and at least 3 such requests was archived because authors tired of updating it for months. If this will happen here too, it will be most regrettable. And this fix really needed.

[YuriSizov]

It's been ages since first pull request with fixes for this issue, and at least 3 such requests was archived because authors tired of updating it for months.

It's the nature of doing something in the core systems, especially something big. This is a huge PR with a lot of consequences beyond just the ability to pick a custom resource in the inspector. It needs thorough review from the core team, and the core team has been pretty busy these last couple of years.

[dploeger]

@pycbouh sorry for mentioning you directly, but you recently commented and maybe have a quick comment on this:

In #48201 @willnationsdev and I are talking about me adopting the two PRs. I was think about getting one of the two ready and merged in a new PR and based on the comments there (back-)port to the other version. Would that make sense to you?

[YuriSizov]

@dploeger Feel free to open a new PR based on Will's work. You could also ask him for a write permission on his fork and that way you could push directly to the same branch on his repo and update the PR this way. If not, opening a new one is fine, we'll close the old one(s) as superseded.

You can keep Will's credit by using the co-authored-by tag in the commit messages.

[dploeger]

@pycbouh I'll do that, thanks. What about my idea to only target one version first and after that's accepted port to the other one?

[YuriSizov]

If you make it for master first, that's a preferred way to contribute any time.

[dploeger]

Gotcha.

[willnationsdev]

@dploeger Rebased this branch too, if it proves to be relevant at all.

[dploeger]

This PR will be postponed until #48201 is merged, so I don't need to do things twice.

[bfloch]

With 3.5 being almost out of the door, I hope the maintainers can comment if there is a chance for this to make it into a 3.x branch.

It would be such a welcome addition in order to round off a stable Godot version while 4.0 is being ironed out - even for existing projects, I'd pick it up immediately since without it I tend to have to write editor extensions just to be able to easily handle custom resources.

Thanks for the hard work.

[dploeger]

I'm still waiting for a review on the 4.0.0 PR. I will see, whether it makes sense to backport it then.

[quantumcode-martin]

That is my most wanted feature for 3.5.
I think Resources are way under considered and underrated, and this would bring custom resources to a new level.
It will definitely boost the usage and creation of custom resources.
I'll make a video explaining it and why it's so powerful as soon as it's available.

Will it be ready for 3.5 ?
Can we do something to speedup the integration and make it available faster ?

[h0lley]

Will it be ready for 3.5 ?

see the comment right above yours.
3.5 seems kind of unlikely at this rate, but there's still a chance I guess.

Can we do something to speedup the integration and make it available faster ?

#48201 this needs to get finalized and merged first.

[andersmmg]

Are there any updates on whether this will start to be backported again now that most of the 4.0 version is merged?

[Atlinx]

Hey all, if anyone's interested I have this PR rebased and cherry-picked onto my personal Godot 3.x branch, and I'm automatically building my custom Godot version using Github Actions here: https://github.com/Atlinx/Godot3CustomBuilds. These builds have both GDScript and C# support -- feel free to download a build if you'd like.

[akien-mga]

Hey all, if anyone's interested I have this PR rebased and cherry-picked onto my personal Godot 3.x branch, and I'm automatically building my custom Godot version using Github Actions here: Atlinx/Godot3CustomBuilds. These builds have both GDScript and C# support -- feel free to download a build if you'd like.

Linking to it directly for people who would want to backport it to their own forks: Atlinx@02d1f70

---

At the current stage, I don't expect this PR to be merged in 3.x, as it's a very significant rework that had a lot of implications in the master branch for 4.0, went through multiple iterations and several months of time and many bugfix PRs. We don't have the resources to do the same effort on the 3.x branch.

So I'll close this not to give false expectations on the likelihood of this feature being included in 3.6 or later. Anyone who absolutely needs the feature can try the rebased version linked by Atlinx in their custom builds.