Redesign 3D asset import workflow

[reduz]

Describe the project you are working on:
Godot
Describe the problem or limitation you are having in your project:

There are several complaints about the current 3D import workflow

• It's confusing that scenes imported can be opened but not edited
• Inheriting scenes to edit them is error prone, users put their nodes together with those of the imported scene, and if they are renamed or moved, their changes go away.
• Configuring what is imported is difficult, the import settings are too cramped together in the import dock, and there is no way to select what happens to imported assets, specifically on a per resource (mesh, material, etc) basis.
• Imported scenes generally have scale issues (root is scaled), so putting user content in there can be easily distorted.

Describe the feature / enhancement and how it helps to overcome the problem or limitation:

The idea is to do some fundamental changes to the workflow to make it more streamlined and less confusing.

• Fist of all, editing (opening) the imported scene should no longer be allowed. This is confusing and serves no real purpose.
• Double clicking an imported scene will open a window where this scene is shown and previewed, and all import aspects of it can be configured, node by node and resource by resource, such as generation of collision, generation of lightmap, bake mode, generation of LODs, whether you want to use the material that comes with the file or an externally supplied one, etc. All tweaks on how the scene is imported will go in this new dialog, moved from the import dock. The import dock will just have a button "Edit settings" and it will open this same window.
• Offering the user to "inherit" the scene should no longer exist. As we have seen, editing an inherited scene that comes from the 3D DCC is prone to making mistakes. While we will offer features to compensate for these (see further points), the idea is to still discourage inheriting. If users still want to do it, they can still do it from the scene menu (new inherited scene), but it won't be facilitated anymore.
• Instead, to use imported scene in projects, users will be encouraged to instantiate them in their own scenes. Instancing is preferred to inheritance because its just simpler (and still allows editing if required), but as user added nodes happen on the new scene and not the inherited scene less problems are prone to happen, because the imported scene is no longer the root node.
• Additionally, for inherited or instantiated scene, if a node was renamed or removed in the original imported file and it contained local modifications, "Dummy" nodes will be created containing the transform and the local modifications, so they can be adapted to whatever the scene has changed.

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:

After a scene is imported, it won't be possible to open it anymore. Doubleclicking it will open its import settings instead. Additionally, the Import dock for this scene will allow opening these import settings, which replace most of the existing ones.

The above is a really bad mockup, but the idea is that you can edit in detail what happens to every node, material, mesh, animation. The Node editor will allow what do with with each node such as:

• Determine whether or not you want to import this node (and children).
• Determine whether to create a separate importable file (put part of the scene in a separate scene file), so you can import to multiples scenes if desired.
• Select bake mode
• Turn on Unwrap UV for lightmap
• Generate static collision on import
• Generate dynamic collision on import
• Generate navmesh
• Generate impostors (for distance LOD)
  etc

The mesh editor, will have options such as:

• Edit and previsualize LODs
• Select settings such as make the mesh streamable or not, etc

The Material editor will have options such as

• Decide what to do, replace by a custom one or use this one
• Save it to a file and use, or keep built int
• etc.

The Animation editor will have options such as

• Control optimization
• Slice large animation in smaller ones
• etc

This way, instead of just opening the scene for the sake of it like before, you open an editor where you can actually check the scene and adjust and fine-tune all import settings.

added by suggestion of @Jummit

A script section can be added where users can add add one or more import scripts, and scripts can export arguments for tweaking the import process better.

If this enhancement will not be used often, can it be worked around with a few lines of script?:

No

Is there a reason why this should be core and not an add-on in the asset library?:

This is core.

[reduz]

@Jummit sounds good, added it to the proposal

[jcostello]

I like the changes but what happens if your import model generates collition with -colonly? Or if you want to remove something from the imported model for example an unwanted node?

[reduz]

@jcostello -colonly and -noimp and similar happen independently of this so they should not be affected. You will still be able to do this process from the new UI too if you wish, but both will work.

[NHodgesVFX]

I would still like the option "open anyway" as it makes it easy to directly edit node structure if i know the source file won't change. For example downloading a model of an environment i just want to import and edit it directly in godot instead of having to open it in blender then re export.

Also it would be great to have loading done on a separate thread. I think the slowest thing is importing and converting textures it would be great to do this in the background. when you work on the importer will you also be adding betsy to convert textures?

[fire]

Many of these options can be thought of as stage plugins that operate on specific things:

• Generate static collision on import.
• Generate dynamic collision on import.
• Generate navmesh.
• Generate impostors (for distance LOD).
• Merge meshes

The idea is we want to run the set of them in a particular order or even a graph.

For example:

We could script https://github.com/wojtekpil/Godot-Octahedral-Impostors as a stage step.

[reduz]

@NHodgesVFX We could put in a RMB submenu the option to "Open Read-Only" I suppose, if you reaaaaally want to do it, but this should not be easily exposed to the user to avoid confusing like now.

[samdze]

I like this fine-grained control over the import process.
I felt like the whole import dock should change a bit to better accomodate more complex import behaviours in the previous versions of Godot.

1. Maybe it's simply not listed, but I'd like to be able to import meshes as separate resources too. (you can already do that but it's really limited)
   Say I have a single gltf file with every mesh of the game and I want to use them wherever I want.

2. Can we have this behaviour below for "standard" inherited scenes too?

Additionally, for inherited or instantiated scene, if a node was renamed or removed in the original imported file and it contained local modifications, "Dummy" nodes will be created containing the transform and the local modifications, so they can be adapted to whatever the scene has changed.

3. Would be great if the imported scene previewer was a semi-full-fledged scene view, so you can navigate/rotate it.

4. I think it would be cool to let custom import scripts to add additional tabs in the import previewer and let them use custom property editors. (maybe even custom gizmos? But it may be not worth it)

[NHodgesVFX]

"Open Read-Only"

I think open directly would make more sense considering you can still edit the scene. Just in case were talking about different things here's a video showing what i do pretty often.

open file as "open anyway"
save it as a .tscn then just use that.

https://streamable.com/4jmxb5

[ee0pdt]

@samdze I like the idea of being able to create multiple scenes from a set of meshes in a single import. Kinda analogous to importing multiple 2d sprites in a single image

[reduz]

@ee0pdt, @samdze this is already supported now, but its a hassle to use and very limited, so yes it should be possible to do this better in the dialog. Let me edit add it to the proposal so its in there too,.

[RandomShaper]

Looks powerful and intuitive.

[ee0pdt]

Could we also think about changing the wording on the re-import button to be “Apply Import Settings”?

[imjp94]

The Material editor will have options such as
Decide what to do, replace by a custom one or use this one
Save it to a file and use, or keep built int
etc.

Will it be possible to specify an existing material in the project, so that imported models from different directories can share the same material

[robbertzzz]

Will it be possible to modify all properties of child nodes in an instanced scene? Right now that can be done by changing the root node type or by importing the model as separate meshes, I see no reason why that shouldn't work already...

[dreamsComeTrue]

That's sick :D Thanks for this proposal. What I would also add - is in that thumbnail model preview window - to be able to see (for example as a green-lined wireframe) generated collisions. And if Godot would support more algorithms of doing these (like bounding boxes, spheres, trimeshes, etc) - allow to switch between them and see the result in real-time.

[DiiaBlood]

I'm not a much of a C++ programmer or anything that needs a brain for it so excuse me if I'm being stupid but can't u just make and import "preset" or "option" to impost a separate .SCN file which lays out everything from the imported file as normal nodes in a scene and u can just delete the original file and have the scene only for it.

[slapin]

Would be nice to add collision-only CollisionShapes w/o bodies too.

Also it would be nice to somehow support adding nodes to groups at import time;

Also a thing which makes my life worse than it should be is lack of ability to import "metadata" associated with node.
Blender supports export of gltf with custom properties wit data which could be used for some special marking of stuff in DCC.
For example, adding quest marks would be as easy as adding Empty and add text description under it; Also it could be used to describe some properties later used in scripts. That way one will not have to maintain huge json/csv table with metadata and models in DCC separately, which will get out of sync or mistyped easily.

[hgouveia]

Could be possible to import as a separated thread without blocking the editor? Specially for big files this could take a lot of time and blocks the editor, sometimes it shows as Not responding

Also would be nice to be able to import them vía command line without opening the editor, for the same reason, sometimes it just take very long time

[vini-guerrero]

Perhaps a merge navmesh would be interesting also, considering you could have an existing scenario and simply add new imported assets which could interact with the existing navmesh as well... like a re-bake post import.

[reduz]

Will it be possible to specify an existing material in the project, so that imported models from different directories can share the same material

yeah thats the idea

[reduz]

That's sick :D Thanks for this proposal. What I would also add - is in that thumbnail model preview window - to be able to see (for example as a green-lined wireframe) generated collisions. And if Godot would support more algorithms of doing these (like bounding boxes, spheres, trimeshes, etc) - allow to switch between them and see the result in real-time.

Yeah this is the idea, I am still not entirely sure how it will look, but the idea is that you can preview everything

[vagrantG]

Custom Processing
Having the ability to customize the import process (before and/or after the default options) to automate tweaking could be an interesting idea.
For example importing models from various sources i often see scaling issues, inverted UVs in an axis, needing to translate the objects (transform) or sometimes is better to recenter the mesh (change vertices position). Changing it from the modelling software is of course an option but a little script altering the data often times seems faster to do plus it could automate the creation of additional stuff (ex. you put a node in the asset then the import process creates something Godot/gameplay specific with that). Of course there's no way to read the data within the specific importer format (ex. read custom properties in GLTF?) but you can do tweaking with the imported nodes, materials, etc.
Another idea is use a different algorithm for generating LOD, collision mesh, etc if someone needs to, without having to change Godot. Custom script and that's it ;)
The custom scripts could work as the importing options (you see a panel with the controls to use) but it'd be nice to just duplicate script templates and tweak them from the dialog and refresh (i'm working on some editor scripts now and it's a bit slow to go around to reload the plugins every time to test them).

Remembering the import settings
I guess all the options set in the dialog will be remembered somehow every time you re-import?

I'd assume these situations in the import process:

1. importing an entire scene from a single file
2. importing an single entity (character, etc) from a single file
3. importing multiple entities from a single file (ex. multiple objects modelled in Blender in an "atlas" scene) in separate steps

I don't know if 3 can be an issue if one would have separated import options (ex. the first enable only a node, the other a different one).

Tweaking mesh/nodes/materials/etc API

I'd like to see some of this functionality around the editor itself, not just limited in that dialog. If you create things from Godot itself or tweak an entity that isn't dependent from the original import anymore i'd like to still be able to use what makes sense. It could be just an API for now to call with a plugin. Of course the main way would be doing it from the import dialog so that you don't make mistakes but if you create a level editor inside Godot it can be nice to tweak meshes without re-implementing functions ;)

Dialog size and useful info

Importing an entire scene and tweaking the options from an orbit view can be a pain. I've similar issues with the material preview above the settings, most of the time it's easier to check things from the main windows itself.
Being able to move around to see how things have been imported could be useful, plus selection and possible the names of entities could be seen.
Same thing for the opposite, a single mesh can be better seen with an orbit view so some different viewing options are useful.
Seeing the dimensions can be also a useful tool to understand if the scale is right or not.
Makes me wonder if instead of a dialog it'd be better to open a tab with custom layout: scene panel to enable/disable nodes, side panel in the tab with importing/tweaking options, toolbar with reimporting and viewing modes buttons, properties panel read-only that shows the imported data when you select objects and updates when you reimport.

[slapin]

@vagrantG hi, need to tell you that features you speak about are already there

[EzraT]

All of this sounds amazing, 4.0 is shaping up to be one beast of a release.

[vagrantG]

@slapin First point, you can make an import script and open an imported asset as a scene and tweak it but you can automate the process by creating a plugin and using it everytime manually (not ideal). Plus if the instantiation will be the main way to go you'll not be able to modify the nodes. Second one, not a proposal but me thinking. Third one is hoping the new stuff like creating impostors, LOD, etc will be still available as a manual option. Plus custom tweaking scripts functions too, why not? You can edit stuff on the fly for various reasons. Fourth, yeah now opening an asset opens a tab with the scene for sure, but the proposal is to have a similar configuration instead of the scene editing one but with different elements in the panels and scene tree use.

[FeatherAntennae]

I love the idea, could it be made for more than just 3D?

I'd love to see a universal window to tweak and preview all kind of imports.

For textures we could have live preview and side by side comparison for different compression level, show generated mipmaps, preview anisotropic and alpha, etc.

It could very well start with just the 3D, but it'd be nice to implement it in a way which would allow adding other stuff later without needing to rework too much.

If it can be made flexible enough, I'd love to have the ability to write "importer plugins" that could seamlessly plug into it for more specific needs or to improve my workflow with a specific ressource type. For example, allow things like importing a sprite sheet as an animated sprite directly, with configs and previews in the importer window.

[HeadClot]

I love the idea, could it be made for more than just 3D?

I'd love to see a universal window to tweak and preview all kind of imports.

For textures we could have live preview and side by side comparison for different compression level, show generated mipmaps, preview anisotropic and alpha, etc.

It could very well start with just the 3D, but it'd be nice to implement it in a way which would allow adding other stuff later without needing to rework too much.

If it can be made flexible enough, I'd love to have the ability to write "importer plugins" that could seamlessly plug into it for more specific needs or to improve my workflow with a specific ressource type. For example, allow things like importing a sprite sheet as an animated sprite directly, with configs and previews in the importer window.

I would just like to point out that unreal has this as well for textures and flipbooks and it is really useful for previewing assets. It would be really useful to have in Godot engine as well :)

@reduz a bit of feedback on your initial proposal under mesh editor.

The mesh editor, will have options such as:

Edit and previsualize LODs
Select settings such as make the mesh streameable or not, etc

Support for sockets in the mesh editor would be nice and would greatly streamline the work flow for making attachable items such as swords, guns, pickups, etc. in Godot engine. :)

[SaracenOne]

I really like this design, it feels a lot more intuitive and natural then the current setup. This has been mentioned a few times, but I really want to voice my support for allowing more integration of custom scripting. I have a working example of something which would benefit from this: an addon which provides a fully functional humanoid animation retargetter. Right now, due to the limitations of the import pipeline in relation to script, it only exists as a destructive post-import tool, but if the importer could have access to multiple scripts per import, as well as the ability to modify the menu's UI and preview, I could turn this tool into a general-purpose animation retargetter which everyone could use. I also have intention of providing a tool which can convert generic animation into IK points for more procedural animation.

I think from an extensibility standpoint, a focus on this import tool being modular via scripts which can be distributed as assets would serve most people's needs.

[Master-J]

The proposal looks fine overall but something concerns me a bit about how the actual resource management will be handled.

I'm currently working on a project where i need to use blendshapes and i noticed i can't have two instances of the same scene with different blendshapes values (one of the instance will always override the others).

In order to work around that, i need to open my imported 3d scenes, save it as a standalone .tscn file (as a consequence of that the inheritance will be broken, and nothing will happen if i edit the imported scene) and enable the "local to scene" parameter in the mesh's settings.

From what i understood, with this proposal it should be possible to "auto import" only a part of an imported scene, would it be possible to have something like a "make subresources local" in the import settings?

Also, what about skinned meshes, if we choose to import only the MeshInstance node without it's Skeleton parent, will the skinning data be preserved after it's imported?

[phelioz]

Don't know if this fits in right here. Been working on a 3D game for a while in Godot and one of the biggest workflow killers I have encountered is the following.

When inheriting or instancing a imported 3D asset in a scene and then reimport it again you need to open the 3D asset scene to have it update all the other open scenes that currently has the 3D asset instanced in.

This slows down the workflow when you are for example iterating over a 3D environment. Where you quickly want to see the changes that you make in your DDC (Blender, Maya, etc) in Godot.
May seem like a small thing but when you are building a large environment with allot of different assets it really slows the workflow down.

[RonanZe]

Don't know if this fits in right here. Been working on a 3D game for a while in Godot and one of the biggest workflow killers I have encountered is the following.

When inheriting or instancing a imported 3D asset in a scene and then reimport it again you need to open the 3D asset scene to have it update all the other open scenes that currently has the 3D asset instanced in.

This slows down the workflow when you are for example iterating over a 3D environment. Where you quickly want to see the changes that you make in your DDC (Blender, Maya, etc) in Godot.
May seem like a small thing but when you are building a large environment with allot of different assets it really slows the workflow down.

The way 3d asset are updating is really weird. And it's the first weakest point of 3d assets import in Godot.
It's hard to be sure that what you have export from DDC is done right or if you have made a mistake.

After exporting an asset, usually, I switch between Godot and an other program to force Godot to update. I see the update progress bar in Godot. But it's not enough.
I have to to open the asset (dae or gltf) to uncompress the .mesh files?
Then close the .tscn with the asset and reopen it.

It far from good and give a unreliability feeling

[vagrantG]

About the proposal of re-using the scene tab as import dialog, another point is being able to see custom data in the preview. If there's the inspector panel you can potentially use its plugins and see a (read-only) result of the data inserted in the elements of the imported scene. If you need to test your custom script in the import process or check if the data exported to the asset file is right it'd be a nice feature to have (without having to import, instance the scene, check there).

[Calinou]

For textures we could have live preview and side by side comparison for different compression level, show generated mipmaps, preview anisotropic and alpha, etc.

In master, the texture sampling mode (filter/mipmaps/anisotropic) is now set directly in the CanvasItemMaterial/StandardMaterial3D instead of being defined in the import settings. This allows using the same texture with different sampling properties several times in the same project.

[samdze]

The more I think about this proposal the more it seems to me like a duplication in functionality of what is already available opening an imported file (with scene view, scene tree, navigation, preview, etc.), but with better inspection capabilities in this particular case (3D asset).

Maybe it could be better to keep the fact that the imported asset opens in the main scene view like before, to reuse all the navigation functionalities (and making it very clear in the scene tree UI that manual changes to the nodes are disabled), and to create a new auto-opening dock in the bottom panel that shows the advanced import settings?

Not sure if that would dissipate the confusion around this whole "read-only imported scene" argument though.

[RonanZe]

Would be nice to support some kind of instancing system. A bridge between instances made in DCC app and Godot.
Right now, I design levels in Blender. Those level are using the same instances for walls, floors, ...

Then I use a plug that recreate some kind of "prefabs" (.tscn) based on from a prefix "_name_instance". It rebuild a scene and all the instances made with those "prefabs".

With the new design in mind, maybe it would be nice to create groups/tags for nodes in the "import setting" window.
A group can then be linked to a .tscn file and all the nodes replace by the linked scene.

[Saran-Pranavv]

Proper Texture Mapping for the imported assests, to check whether the imported assests from any 3D software looks the same in the engine.

[justinbarrett]

The main issue I have with import is animations...or shared skeleton node..
e.g I would like to be able to import a skeleton and be able to import animations individually for it(taking into consideration I will not be able to add bones etc)

or be able to import a mesh with weights assigned to the skeleton previously mentioned(without the need to import a skeleton) and deform it using that skeletons animation tracks...I hope this makes sense..

as it stands now you have to either have all your animations done before import, which is fine, but it does not allow any tweaking later without requiring a complete reimport. When doing complex node arrangements, this can be tedious and I often find myself writing all types of notes about the hierarchy and settings.

[Eoin-ONeill-Yokai]

generation of LODs

@reduz Maybe I'm confused, but my understanding was that godot currently doesn't have any LOD support?

If it's planned for the near future, will these details be stored as a mesh import detail?

[Calinou]

@Eoin-ONeill-Yokai LOD support and LOD generation are both planned for 4.0.

[Eoin-ONeill-Yokai]

@Eoin-ONeill-Yokai LOD support and LOD generation are both planned for 4.0.

Neat! In this circumstance would custom, authored LODs also be allowed to be specified in the import settings panel as well?

[AndreaCatania]

I think this is a really nice idea, and will solve many issues I'm encountering right now with the material assignment. I've an environment model, where many objects are repeated in it. Each mesh needs a visual shader material, so the only way to add it is in the 'MeshInstance'. Though, this would require assign the material to all instances of the mesh, which is a tremendous time-consuming task.
With this proposal, I'll be able to assign the material directly to the 'Mesh' resource, so that it's automatically assigned to all meshes.

The only thing that I hope, is that it will keep track of those materials assigned to the Mesh resources, otherwise for each import you have to re-configure it.

[oparisy]

I was googling at Blender => Godot assets import once more since I had such a bad experience in the past, especially with respect to a rapid prototyping workflow (frequent modifications and reimports of existing assets).

I must say I'm delighted I've stumbled upon this proposal, especially from Reduz himself. It covers all frustations I had and then some: model explorer, read-only, fine tune of the importation process, importation in multiple scenes if needed (great for assets packs).

This will be a great evolution of Godot.

[oparisy]

Also, how would this be related to meshlibs? Couldn't those benefit from those improved import options?

[RevoluPowered]

@reduz One suggestion I have is can we please make the UI for the reimport dialog expandable too?

I would have liked to have a few FBX specific options in there but only for FBX files, I did at one stage plumb it into the engine but it was very manual.

[ChildishGiant]

This is causing issues for me when trying to make a MeshLibrary because each model is imported as a scene, not a MeshInstance, so can't be included.

[akien-mga]

Implemented by godotengine/godot#47166.