Add support for global plugins/universal addons

[mrjustaguy]

Describe the project you are working on:
For this the specific project doesn't matter, what matters is that I'm using the same addon for several of my projects.

Describe the problem or limitation you are having in your project:
I see it useful for people (like me) who use a ton of the same addons across different projects because it would speed up things if it auto loaded the addons from some directory i set for the engine (much like you set the engines default project directory)

Describe the feature / enhancement and how it helps to overcome the problem or limitation:
It would be a feature that would allow for automatic addon placement in a new project, thus speeding things up for people who use a ton of addons and make a ton of projects.

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:
The idea is simple, the user can make a Directory that you label as the universal addon directory in the editor settings (much like you set a directory for default project location), so for every new project when it is created the editor will automatically put the addons (or it could ask which addons you want to put from the directory in when the project is created)

the pseudocode would look something like:

1. Grab editor setting "Universal addon" (or some such name)
2. Gets the new project directory
3. (Upon project creation, optional) popup a list of addons from the directory that can be toggled on/off
4. (in case of 2 implementation) When ok is pressed copy all of the addons toggled on
5. Grabs the addons, places them in the new project directory as an addon

If this enhancement will not be used often, can it be worked around with a few lines of script?:
It can be worked around by making a directory where you store addons and you can just copy them over to the project, so it's not that big of a deal, this would just shave off a few steps making this quicker.

Is there a reason why this should be core and not an add-on in the asset library?:
Because it's an improvement to the add-on/asset library and one couldn't make an add-on for Godot that would do this (I'm fairly sure atleast)

[arkology]

Related PR: godotengine/godot#35432

[Xrayez]

From what I've heard (mind you, just rumors), core devs planned a different approach with those plugins to be treated as regular projects (which could be nested), going for the approach inspired by the node-based system.

[maciek134]

@akien-mga asking you since you explained it nicely in the closed PR - is @Xrayez right about the approach this should take? I'd love to have this feature and I'll be happy to create a PR, but I'd like it to be mergable. Could it be as simple as treating all projects in HOME/godot/plugins as global plugins, or did you have something else in mind?

[aaronfranke]

so for every new project when it is created the editor will automatically put the addons

What about a way for addons to be used globally, but not installed in each project? For example, with https://github.com/godotengine/godot-git-plugin, it doesn't really have a purpose of being inside the project aside from that being the only supported option. This addon could be deleted from the project and the project would still work fine, so it doesn't need to be copied into the project if there was a system to allow it to exist elsewhere.

[me2beats]

could global addons just to be located somewhere, say in some "global plugins dir" then the user could specify what directory to use as global plugins dir and after that when creating or loading a project Godot enables these global plugins?

also maybe there could be some API methods like
set_global_project_dir
enable_global_project_plugin
etc

About UI implementation:
It could be a plugins list in Global Settings — Plugins, like we have it in Project Settings — Plugins
or implemented as one Plugins List (for both Project and Global plugins), but then it shouldn't be located in Project Settings menu.

I believe the ability to have global plugins would be handy.

But on the other hand, there are several concerns:

1. safety of other projects. for example, if a global plugin can delete the content of projects, then when such a plugin is activated, each open or loaded project will be deleted.
   in this case, one need to be as careful as possible when using third-party projects that use global plugins.

2. it may increase projects coupling, this can be compared to a singleton pattern, which, if used incorrectly, easily turns into an anti-pattern and only creates more problems.

[silentorb]

One benefit of global plugins is, depending on how they were implemented, they could remove the pressure to add nice-to-have plugins into the Godot core.

For other game engines such as Unreal and Unity, many features that would be considered core are actually plugins that are installed and enabled by default. Over the years Blender has transitioned to a similar structure.

If Godot supported the ability for someone to zip a collection of plugins with the Godot executable, that would allow a variety of different Godot installations to choose from, both official and third-party, similar to Linux distros.

That way a beginning developer could download an installation with a hundred enabled plugins while the Godot core remained light and minimal.

That would also give core developers the option to convert some core modules into plugins, which in turn would allow game developers to toggle those modules without compiling a custom executable.

[Calinou]

We discussed this yesterday in the Godot proposal meeting. We acknowledge it's a desired feature and could be useful, but at the same time, we're wary about uses of the "global add-ons" feature for add-ons that should clearly be stored within the project. In other words, your project might break if it relies on global add-ons but other people in your team forget to install them. We don't want to require people to scour the Internet manually to find add-ons before they can start working on a project.

To summarize, we think "global add-ons" work best for editor-only features that aren't related to your project, not features that will be exported into the final project.

That would also give core developers the option to convert some core modules into plugins, which in turn would allow game developers to toggle those modules without compiling a custom executable.

Global add-ons aren't a strict requirement for moving core features to add-ons. See godot-interpolatedcamera3d for an example.
If this add-on was to be used as a "global add-on" by someone, this could cause projects to break unexpectedly when switching PCs or working in a team. To give a counterpart to Node.js, it's the same issue as using npm install -g for project-specific dependencies 🙂

For what it's worth, the Node.js community is attempting to fight this problem by disallowing the use of globally-installed modules in the project itself. It will take a while for them to fully solve this issue, though.

[Two-Tone]

No, but to a large number of users they won't know that the once standard features are now add-ons they have to manually download and enable for each project, which will create some ugly, negative word of mouth that will be hard to fight.

…

On Fri, Oct 2, 2020 at 3:37 AM Hugo Locurcio ***@***.***> wrote: That would also give core developers the option to convert some core modules into plugins, which in turn would allow game developers to toggle those modules without compiling a custom executable. Global add-ons aren't a strict requirement for moving core features to add-ons. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#831 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABBEIS5QRSMM2L7EDH3S7U3SIWGOBANCNFSM4M7Q55OA> .

[esijg]

In other words, your project might break if it relies on global add-ons but other people in your team forget to install them. We don't want to require people to scour the Internet manually to find add-ons before they can start working on a project.

I am all in for global add ons, but as seems to be the consensus here, we need to be careful. I've had that exact same experience with PureData back in the day. I even have problems viewing my own old Pure Data projects as I've lost some of the global plugins and can't find them online anymore. :/

[dark-penguin]

Indeed, it would help greatly if "go do that in GDNative" was a better experience! And I agree that it should only be for the "editor plugins". The plugins that have anything to do with your project must be included in your project.

Thoughts about the implementation:

• Of course, plugins that go into your project should be in res://addons/ . But "editor-only" plugins should definitely not! This way, you don't have to specify specific addons in your .gitignore - this folder is in your project for a reason. And that makes a clear difference between one and the other.

• I would suggest that editor-only plugins are accessed with addons:// , or even better - with addon:// , which is the path to this addon! This would eliminate a few other problems. First, an addon won't have to hard-code its own name in every load(""), which makes it easier to rename addons (and fork them). If your project is an addon, then this should be as simple as usual with "res://" , but for addons. Second, you could put LICENSE and README.md in your root folder, which is good for Github, and users downloading your addon won't have to worry about checking the files and not installing a very important part of your addon because it would overwrite their own files. Since this addon does not go into your project, it does not have to be in res://uniquename anymore !

• Downloaded "non-global" addons should be kept in your "local library". When you start a new project, you can open your "local library" and choose which of your global addons to enable, and which of your non-global addons to install. I would even suggest that the "fourth editor button" should not open the Asset Library, but the list of your local addons instead, and there should be a button to "Download more...". Now it's the opposite way - you go to the online library first, and from there you can go to your "Plugins". This really should be the other way around - with this, you will access your local library more often than the online library.

• If you update a "non-global" addon in one of your projects, the newer version is kept in your local library. It will not automatically update in all your other projects, but when you do update it, you won't have to download it again.

With this, we wouldn't even need "Favorites" implemented in the Asset Library, because you would simply download them all.

[dark-penguin]

Turns out this issue is already there since long ago: #1205
Redesigning the plugin system is not so simple, so the discussion is ongoing.

(EDIT: this is slightly different, but it's mostly the same general issue)

[Calinou]

Trying to modify a "global plugin" should indeed have a warning attached, but it should still be possible. The main reason is dogfooding. If while using a plugin, particularly one that you developed yourself, you find a bug when it's used in a specific way, being able to fix it in-place, in the current project, instead of trying to create an MVP and fix it there, is incredibly helpful for development.

If you have to do this, I think it's best to copy the add-on locally and have the local add-on override the global add-on in a given project. This makes sure your patched add-on is committed to version control, as you don't want to lose your work if you move your project to another machine (your patched version may not be available anywhere online).

[rsubtil]

I definitely believe global and project plugins should have a very clear separation. Global plugins should only run in the editor, while project plugins run on both the editor and the final project, the way it is currently.

but how to know if the plugin is "editor-only" and will not affect the game?

From the developer side, I don't see a problem here; just test whether a candidate global plugin can be used without interfering with a project. If it turns out to be needed, then it's a project plugin and needs to be copied to the project files.

From the user side though, and to avoid confusion in figuring out if a plugin can be global or not, I vote for having a dedicated category for them, requiring those plugins to be explicitly marked as global when submitted for the asset lib.

Letting users modify global plugin's code and resources from an existing project, like @SlugFiller said, is pretty helpful for plugin development and/or tweaking according to the users' needs. I don't think this should be restricted at all, although warnings are good so users are aware of the implications.

[SlugFiller]

If you have to do this, I think it's best to copy the add-on locally and have the local add-on override the global add-on in a given project. This makes sure your patched add-on is committed to version control

Quite the contrary. It makes the plugin in the version control of the project, instead of that of the plugin. If you regularly use git or mercurial for each, then having the plugin be inside the project folder actually gets in the way, because now the addon's versions are linked to those of the project, of which you could have many.

Like I said, if you modify a local version of the plugin, it makes it that much harder to then copy back the changes, and then apply them to all projects. Just because you only got burned by a bug in one project, doesn't mean it won't ever influence any of the other projects using the same plugin.

The worst case scenario is that you forgot to copy back changes, and then you hit a different bug in a different project using the same plugin, and now you have a version conflict. And because each project uses its own version control, you can't easily merge the changes. If the plugin exists in a separate place, with its own version control, this never becomes an issue.

P.S. One of the plugins I've had the most issues with this is my vector plugin. I kept a version I was pushing to GitHub, a version I was using inside a Godot project for development and unit testing of the plugin, and a version I was using in a prototype game. And each of these had different version control. And that plugin is not purely an "editor only" plugin. It imports an SVG as a PackedScene, but also needs custom elements like Vector2DFill in order for the vectors to actually work in game. The imported could (and should) be editor only, but the custom elements cannot be.

So the "only for editor plugins" thing doesn't really work from a real use case perspective.

[Sharaf5]

I think I had the same idea but from the ground #7291 (comment)
From the core itself,
that would be handy, especially for those who work on many similar projects

[AlfishSoftware]

If an editor-only (i.e. not exported with project) plugin needs to add an auto-load node to provide functionality in the debugged game, how would that work? Currently, autoload is a project setting (just like the list of enabled plugins), and since override.cfg won't work in the editor, you would need a way to merge this setting without affecting project settings in project.godot or any other versioned file in the project's repo.
(EDIT: turns out override.cfg does work for autoload in the editor, but still, this file is not plugin-specific or guaranteed to be ignored by version control)

I propose that, to aid global editor-only addons, we also also include a generic way for a plugin to change any project or editor setting, allowing to:

• set an override for the whole setting (including arrays and dictionaries)
• append or prepend items into an array setting
• merge items into a dictionary setting

Such overrides/merges could be applied by code and temporarily (not auto-persisted), so that the plugin chooses how and where to persist it on its own.

For example, it may want to save its own files or state inside .godot/ folder so it's per-project, but also per-user (i.e., not on version control). Or it could save it for all projects, inside some standard plugin-specific folder that is modifiable by the plugin, e.g. ~/.config/godot/addon_files/my_addon/. The plugin would be allowed to add any arbitrary files for its own internal use, and those would be persisted across any plugin updates.

Additionally, the user should be able to enable global plugins without this decision having to go necessarily in version control. If they need it, they can use some versioned file (e.g. README) to instruct others that a plugin is required.

[Frontrider]

If an editor-only (i.e. not exported with project) plugin needs to add an auto-load node to provide functionality in the debugged game, how would that work? Currently, autoload is a project setting (just like the list of enabled plugins), and since override.cfg won't work in the editor, you would need a way to merge this setting without affecting project settings in project.godot or any other versioned file in the project's repo.

IF the plugin needs to include stuff in the game, then it is explicitly not editor only by definition, as it has code that is needed by the game and not only the editor.

That is the other category, where you have both things that don't make sense in the export, and things that do. I'd even say that that is closer to the core of the issue that we have here (and this category is only a subset of it), as an editor only plugin is one that just filters out all of its components from the finished game.
(may even be solvable with a class level annotation that just excludes the script, and a few other specified resources from the export)

The part where it overlaps with #7375, and also projects like this one.

[AlfishSoftware]

IF the plugin needs to include stuff in the game, then it is explicitly not editor only by definition, as it has code that is needed by the game and not only the editor.

@Frontrider You didn't understand me. When I say "provide functionality in the debugged game" I mean the game that runs when you press play in the editor. It's very important to get the definition of editor-only plugin right:

• Editor-only means nothing (code, assets) is included in the exported game.
• It does not mean all code is a tool script or that no code runs in the game at all, because the plugin may need to "inject" game code to provide functionality (see below).
• In other words, it doesn't mean Engine.is_editor_hint() or @tool, it means OS.has_feature("editor") or in C# #if TOOLS (or the equivalent GDScript feature you linked). Also, putting files inside a folder that starts with a dot (so they're not exported) helps.

For example, the "Project Camera Override" feature is absolutely an editor-only feature, but it does affect the game debugged from the editor. Any editor plugin that provides a related feature or one that works similarly needs to inject game code (non-tool script). In fact the EngineDebugger API exists precisely so that a non-tool game code can communicate with the editor.

I talk about autoload because it's the only way I know of that an editor plugin can inject code into the game. If so, there needs to be a way to provide an editor-only autoload (and more generally, editor-only project settings) that:

1. Apply to debugged game, but won't be in the exported game; and
2. can be outside of version control...
3. ...without making assumptions about the structure of a specific project.

(1) can be solved by using project settings with editor feature tag (though you can only override the whole value, not merge arrays or dictionaries). (2) would be solvable with something like override.cfg if it wasn't for (3) - i.e., you can't guarantee or require that override.cfg be out of version control, you need a plugin-specific override file.
That's why I made the suggestion above.

[sammypanda]

I think this is a good idea :)

for ex i don't want everyone to have to have a discord rich presence plugin to show what they're doing in the editor since it's not related to the project in any way, it's just niceties for me (and cloning the plugin to each project i do would be a bit weird)

[Frontrider]

It does not mean all code is a tool script or that no code runs in the game at all, because the plugin may need to "inject" game code to provide functionality (see below).

Understood exactly what you mean by that explanation, thank you.
Yeah, that stuff might need its own API, just to make it clear what is happening.

[chexmo]

I dunno how is this issue going todat... but to give my grain of salt, some industry standard stuff (like decent git support for example) may depend on the implementation of this feature

[Hyrilorius]

It would be interesting if Global Plugins were sorted out before the official Godot Asset Store is released.

I think many of the existing most popular Addons would fit in this "Global" category so it would be great if anything that could be a Global Addon in the upcoming Asset Store would be a global addon from the start, to avoid a lot of confusion and compatibilty/branching issues in the future.

[dugramen]

Hey everyone. Last week I released a plugin that more or less implements global plugins, its called "globalize plugins" in the asset library.

It takes the simple approach of just copying a list of plugins into the current project, every time the project loads, keeping them updated. You just have to search and install this plugin from the asset store once per project. It's not ideal but it should suffice while we wait for an official solution

Here's the github page for more info https://github.com/dugramen/godot-globalize-plugins

[tgienger]

Hey everyone. Last week I released a plugin that more or less implements global plugins, its called "globalize plugins" in the asset library.

It takes the simple approach of just copying a list of plugins into the current project, every time the project loads, keeping them updated. You just have to search and install this plugin from the asset store once per project. It's not ideal but it should suffice while we wait for an official solution

Here's the github page for more info https://github.com/dugramen/godot-globalize-plugins

That's nice, but the point of global plugins is not being required to have multiple copies of the same thing. At least that's what I want. Symlinks work, but they're not universal. I'm storing my projects on an exFAT drive atm and they don't use links of any kind.

[dugramen]

That's nice, but the point of global plugins is not being required to have multiple copies of the same thing. At least that's what I want. Symlinks work, but they're not universal. I'm storing my projects on an exFAT drive atm and they don't use links of any kind.

I wonder if that's possible. All resource paths use the res:// prefix, which points to the running project's directory. A resource that exists outside that directory wouldn't load dependencies properly, because res:// would no longer point to the right files

What we need is a way to redirect or override what res:// points to when loading a resource. Maybe an extra option in ResourseLoader.load, or maybe a new function ResourceLoader.load_external or something?

[dugramen]

Ok so I spent the last month on a new solution, and it implements basically all the features proposed in this thread.

It's called the Godot Global Project. It implements editor-only plugins, an addon import popup once per project, automatic addon imports, and even project-manager plugins.

All you have to do is download and open the project. It adds a script to EditorSettings, so whenever the editor loads, plugins inside this project are ran. No need to install some addon for every project (like my old globalize-plugins addon), as these plugins are truly global. And no multiple copies of the same thing (for editor-only and project-manager plugins)

There's a lot going on in here, and I've tested as much as I could think of. But of course there'll be edge cases, so I welcome reporting any issues.

[aXu-AP]

@dugramen I gave it a quick try and I have to say that I'm impressed! Injecting a bit of code to the editor settings is quite clever solution, and so far I didn't find any problems. I'll definitely will use this one. Now I'm motivated to make some editor only plugins I've been planning 😁
Everything seems to be working nicely (altough I didn't test project manager plugins except for the included inspector).
Seriously, good job on this one!

[dugramen]

@aXu-AP Thanks! Hope it keeps going well, and feel free to report anything that doesn't. Also yeah, making project-manager plugins will be a bit tough / laborious until I make an "api" for it. Working on that next

[Sharaf5]

All you have to do is download and open the project. It adds a script to EditorSettings, so whenever the editor loads, plugins inside this project are ran. No need to install some addon for every project (like my old globalize-plugins addon), as these plugins are truly global. And no multiple copies of the same thing (for editor-only and project-manager plugins)

greet of you @dugramen, I would give it a try soon
and could I sugget you create a branch for godot for that injected code
the code base of godot is working out of the box with binarry scenes so I think you find a better solution for this wornning here by directly developing the code base rather that injecting code

Binary resources, files ending in .res and .scn, won't be processed. Since they're not stored as text they can't be easily converted.

however your idea is really fancy

[aXu-AP]

@Sharaf5 if I understood it right, the .res/.scn limitation is only due resource path replacing with absolute paths, otherwise it would be fine. I don't think there's an easy way out here.

The only better solution would be to implement this in core, but it's a big feature and the idea has been around for many years without much progress on this area, so I gladly take this unofficial plugin even with minor caveats. I think this is my #1 wanted feature, as I don't see much of a future for editor only plugins without this kind of support. I don't even use the official git plugin because of the hassle of adding it into every project (and also the plugin itself making into git history for no reason). Now I can use it (edit: ah, it seems to be a gdextension, so I guess it's also off limits)

[dugramen]

@Sharaf5 So the only reason .res and .scn resources aren't processed right now, is because the project grabs the files as text, and searches for resource paths it can redirect. Binary files aren't stored as text, so I can't really do that with them.

Technically, if I did allow .res and .scn files right now, they'd mostly work out of the box. They just wouldn't be able to load anything.

I think I might be able to grab the binary, convert it to text, modify it, then convert it back to binary. I'll have to do some exploring to see how easy that is. But I don't think its a priority right now

Edit: yeah what @aXu-AP said