Plugins cannot integrate custom resources into the save and load process

[Zylann]

Godot 3.0.2

I'm having an issue with my plugin which badly affects usability right now: Zylann/godot_heightmap_plugin#4

Short version:
I haven't much problems about how to save my data itself (that would eventually be covered in #17463).
The problem I have here, is when do I save it.

Detailed version:

I made a plugin that defines a custom HTerrain node.
In order to work, a HTerrain node need to know which terrain data to use.
Currently, this data is implemented as a dummy resource for the sole purpose of interfacing with Godot's saving system. That resource only hold a path to where the big data is.
The big data is mostly composed of resources Godot already handles (most are textures, others are binaries), but cannot be just assigned by the user in the inspector because they are generated and numerous.
They must be saved under a folder, not a single resource, because there are multiple big files involved. Also, because a terrain is big, it is designed to be able to save and load asynchronously with a progress bar.

When a user hits Ctrl+S to save a new scene with a terrain inside, I need to:

• Know where to save the data, ideally create the folder next to the scene (or resource) is being saved. It is possible to embed the resource in the scene, but it is impossible to save the big data inside. So I need a path. In worst case, I can do what Unity3D does: create the folder at the root of the project using a unique name, and leave to the user the task of moving it and changing paths.
• Save asynchronously so it doesn't freeze the editor (reimport is another beast though)
• Save the path to the folder as a property of the resource, so that it won't break if the user moves that resource somewhere else (moving the data folder along with it cannot be automated in Godot so I can only expect users to at least do it themselves and reassign the new path).

Problems:

• Godot offers no API to know when a custom terrain resource gets saved. EditorPlugin has queue_save_layout and save_external_data, but they are not suitable when the terrain node is not selected. It would require the plugin to parse the whole scene to find all terrains, and there is no info where the scene gets saved anyways.
• Godot offers no explicit API to integrate asynchronous terrain saving into the process of saving a scene, so if the workaround explained above were used, it will freeze the editor until it's done and won't show a progress bar. I could use background saving when I receive the callback, but there would be no feedback in the UI and the user could close the scene or even the editor before it completes.
• Even if Godot offered a ResourceSaver API to save the resource, what would happen if the resource is embedded in the scene? In that case there is no file, so how would I get to save the big data if Godot doesn't use ResourceSaver? It's fine for the resource to be embedded since it's just contains a path, but not getting a callback to set that path and write the big data gets me back to square one...

Note that so far I described a process involving a Resource. The reason I keep it is, the only thing Godot can save must be a Resource. But as you saw, what I want is not really a custom resource format, but a way to save many files, which are already supported in Godot, organized in a specific way, under a folder, as the user saves the scene.

The current workaround I have is to not save the terrain on Ctrl+S. Instead, I have a separate menu when the node is selected, on which you can "Save terrain" and "Load terrain", however this is not intuitive at all.

Related to #17463.

[Calinou]

Feature and improvement proposals for the Godot Engine are now being discussed and reviewed in a dedicated Godot Improvement Proposals (GIP) (godotengine/godot-proposals) issue tracker. The GIP tracker has a detailed issue template designed so that proposals include all the relevant information to start a productive discussion and help the community assess the validity of the proposal for the engine.

The main (godotengine/godot) tracker is now solely dedicated to bug reports and Pull Requests, enabling contributors to have a better focus on bug fixing work. Therefore, we are now closing all older feature proposals on the main issue tracker.

If you are interested in this feature proposal, please open a new proposal on the GIP tracker following the given issue template (after checking that it doesn't exist already). Be sure to reference this closed issue if it includes any relevant discussion (which you are also encouraged to summarize in the new proposal). Thanks in advance!