Create some basic standalone tools or scripts for asset/scene conversion

[bud11]

Describe the project you are working on

A 3D action game.. really doesnt matter much more than that besides the need to prototype and iterate on bulk animations and level content frequently

Describe the problem or limitation you are having in your project

Using godot for 3D/animation asset importing can be rough. Especially for rapid prototyping and changing. Things like losing certain scene changes on reimport, or having to re-export and re-import entire animated characters with all animations just to say update one animation (which can take a very long time with some formats/softwares), arent great.
Theyre also actively limiting in some situations. For example, I wanted to import blender camera FOV/DOF keyframes, but there was no way to carry that kind of thing over, partly because Im pretty sure its not standard to any 3D format.
Theres also the fact that exporting with standard exporters sometimes gives you weird data that might need cleanup or alteration in of itself. Like useless animation tracks.

I understand there are import settings and import scripts and importer plugins and things, but these are all on the engine side, and definitely cant always do what you want. Them being on the engine side requires there to be at least some form of intermediate format to be imported in the first place, which is where limitations and potential wasted effort stem from.
Theyre an extra step in a useable import pipeline for next to no reason in the context of this proposal.

In my case, I ended up writing a scuffed blender plugin that can export animations separately directly into .tres files. Boom, one click, instant standalone animation resource in godot with whatever tracks and features I want. Dont even need to make the engine rescan or reimport.
I also tried writing one for scene/model export into .tscn files, the benefit there being immediate geometry updates, automatically assigned material slots derived from the game files, godot node/object placement from within blender.. but that started becoming really difficult with the weird underdocumented and differing internal scene formats for things like model data, collision data, navmesh data. I eventually settled on generating tscn files that references obj files but that isnt a great answer in the context of characters or anything more complex than prototype level geometry.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

I think it'd be a good idea to create some basic easy to reference scripts or executables, say with python or a headless godot instance or something, that can help generate standard scenes and resources. You'd be able to reference them in blender python scripts or maya scripts or whatever you want.
Youd feed the scripts basic data and they'd spit out scene/resource data, as if the engine imported and converted them itself. But, with the benefit of this being during your own export script of some kind, so you can feed them whatever the hell you want or make any changes you want without needing to deal with them in engine nor deal with the specific limitations of standard file formats.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Pretty much described already, but for specific api ideas, I could imagine it being close to the creation/alteration methods already provided by classes like Animation, ArrayMesh, etc. Im not totally sure.

If this enhancement will not be used often, can it be worked around with a few lines of script?

Its all technically doable without any godot-given scripts, but replicating godot scene/resource data can be difficult and documentation can be sparse. I know I struggled with arraymesh generation, with stuff like face normals being weirdly compressed and general attributes being underexplained, to the degree that Ive given up on it for now.

The one single problem I see here would be scripts like these being unable to test or register UIDs from your project, but that seems like a very minor thing that Im not even sure needs solving in a context like this.

Is there a reason why this should be core and not an add-on in the asset library?

Would massively improve the entire import pipeline, at least for 3D.

Im honestly not sure of the overall importance of a proposal like this, but its something was an actual major roadblock at the beginning of my project and something Ive had to invest a fair amount of time into solving for my project, so I think its worth mentioning.
Also not sure if its the kind of thing that should be mentioned on the godot proposals page given its bordering on external tool.

[bud11]

Just wanna clarify, the reason Im not just releasing my scripts and considering that the answer is that theyre very situational, limited, and specific to me and my project. That and I cant even complete some of them.
I also know there are addons like https://blendermarket.com/products/blender-godot-pipeline-addon, but this is specific to newer versions of blender, introduces new per-project 3d software AND engine code, and actually uses gltf underneath, which as Ive described can cause problems.

Also, anything entirely community written could and probably will become outdated with future format changes. Something like this would either be maintained by the godot team, or it'd literally be a part of the engine in some way, to the extent that it didnt need much of its own maintinence and could just rely on some of godot's internal methods.
Users would still have to write their own minimal scripts if they wanted to use whatever this would be, but they wouldnt have to worry about potentially complicated or ever changing methods of serialization.

[passivestar]

I understand there are import settings and import scripts and importer plugins and things, but these are all on the engine side, and definitely cant always do what you want. Them being on the engine side requires there to be at least some form of intermediate format to be imported in the first place

Godot's built-in blend import just launches blender with a python script that exports gltf. You should be able to execute your own logic this way doing anything you want, including writing to resources directly instead of using gltf

I can't see why this can't start as an addon to see if it's feasible. Even with gltf things can break, but if you're reading from blender's API directly you'll need to spend much more effort maintaining it because of a larger surface area

In general I think we should push gltf spec to be more useful and include more data about the scene. Right now it can't even tell you if a light has shadows enabled or not. Then it would not only serve this community, but everyone else in the industry too

[bud11]

Godot's built-in blend import just launches blender with a python script that exports gltf. You should be able to execute your own logic this way doing anything you want, including writing to resources directly instead of using gltf

Well, there is a definite strength to keeping the entire blend (or whatever 3d software) file and the final godot resources separated. I often have a lot of animations and random assets like multiple characters and props and the like in one blend file, most of which I definitely dont want to carry over or nessecarily have show up in godot right now just because I saved the file.
Id have to go out of my way to mark everything I didnt want as no-import, or Id have to save things into specific files for got, if I were using blend files. Writing some minimal custom resource export stuff on the 3D software side so you can directly export specific things in whatever way you want is absolutely preferable.
Im also using a seriously old version of blender that godot can't interact with to my understanding, and doesnt deal with gltf super well, but Im not using that as a point.

And if it just uses gltf as well, which I didnt know about, then that makes it even more useless.

In general I think we should push gltf spec to be more useful and include more data about the scene. Right now it can't even tell you if a light has shadows enabled or not. Then it would not only serve this community, but everyone else in the industry too

I think gltf and standard 3d formats like that definitely have a place, no reason not to improve them. But there is some information that just cant carry over without custom implementation. Stuff like setting shader materials, referencing scripts or creating godot objects, keyframing custom effects or whatever. Youre at least going to need to write custom export and import behavior, and while a standard format can still help you do some of the heavy lifting for things like scenes, its really not helpful for creating individual resources like animations.
Which is why I think itd be preferable if there were an official godot library that you could use for more direct and powerful heavy lifting. Then you just write your export code in your 3d software and thats it, perfectly controlled instant scenes and resources.

And really this proposal goes wider than just 3D. It'd work for any resources or nodes whatsoever.

[ydeltastar]

I think it'd be a good idea to create some basic easy to reference scripts or executables, say with python or a headless godot instance or something, that can help generate standard scenes and resources.

I can't see why this can't start as an addon to see if it's feasible. Even with gltf things can break, but if you're reading from blender's API directly you'll need to spend much more effort maintaining it because of a larger surface area

Wasn't this the purpose of the Blender ESCN exporter? But it looks like it's abandoned.

Which is why I think itd be preferable if there were an official godot library that you could use for more direct and powerful heavy lifting. Then you just write your export code in your 3d software and thats it, perfectly controlled instant resources.

DCCs are too different from each other, that's why formats like glTF are used as a middle ground. Still, everything besides regular vertex data never works out of the box on any engine. But Godot has the most stable built-in features and workflow with Blender I've seen in any game engine. Godot's scripting API is already an extensive library you can use instead of writing tscn files from scratch in another language.

Currently, I leave the bulk of the export to the glTF and Godot's built-in importers and use a custom addon to export only the metadata that it lacks with pre and post-import scripts.

Things like losing certain scene changes on reimport,

This is more a problem on the engine side than the export process. I'm hoping this starts being addressed thanks to godotengine/godot#86779. Currently, I have to do a lot of hacks like exporting switch bones and using their animation position to trigger events because callback keyframes will be lost on reimport.

[bud11]

I think it'd be a good idea to create some basic easy to reference scripts or executables, say with python or a headless godot instance or something, that can help generate standard scenes and resources.

I can't see why this can't start as an addon to see if it's feasible. Even with gltf things can break, but if you're reading from blender's API directly you'll need to spend much more effort maintaining it because of a larger surface area

Wasn't this the purpose of the Blender ESCN exporter? But it looks like it's abandoned.

Which is why I think itd be preferable if there were an official godot library that you could use for more direct and powerful heavy lifting. Then you just write your export code in your 3d software and thats it, perfectly controlled instant resources.

DCCs are too different from each other, that's why formats like glTF are used as a middle ground. Still, everything besides regular vertex data never works out of the box on any engine. But Godot has the most stable built-in features and workflow with Blender I've seen in any game engine. Godot's scripting API is already an extensive library you can use instead of writing tscn files from scratch in another language.

Currently, I leave the bulk of the export to the glTF and Godot's built-in importers and use a custom addon to export only the metadata that it lacks with pre and post-import scripts.

Things like losing certain scene changes on reimport,

This is more a problem on the engine side than the export process. I'm hoping this starts being addressed thanks to godotengine/godot#86779. Currently, I have to do a lot of hacks like exporting switch bones and using their animation position to trigger events because callback keyframes will be lost on reimport.

..yeah, great, so abandoned program specific plugins and a messy mixture of multiple scripts and addons, again both requiring code on both the dcc AND engine sides. And hacks to get around current engine problems. Not like its not doable, but I dont see how any of these responses give better or more viable solutions than what Im saying.

I do agree with you in terms of it being the best engine Ive dealt with, I appreciate the existence of plain text scenes and resources, and I remember unity being a lot worse in terms of importing. But I still think it could be slightly better, and maybe even needs to be for some projects.

Im not sure if Im communicating my point properly here. Im saying youd still need to write dcc specific export code to process the objects and data in say your blender scene, getting vertices and normals and scene properties or whatever you want, but then there'd be a library you could reference in script that let you commit that data into scenes and resources however you wanted to without needing to fret over the final serialization stage. In some cases the serialization is easy and doable, especially given the text resource formatting, but sometimes its a real pain in the ass, and you lose out on little optimizations and the like.

Its also not like youd nessecarily have to write the dcc code yourself. Those could be made into dcc godot exporter addons, but theyd be relying on this godot library instead of middle ground formats.

I dont think this specific idea is a nessecary one by any means but I think it would help a fair amount of people optimize and more quickly establish their workflows.

Honestly, I might end up do something like this myself. But if I did, Id like to have it rely on godot source somehow, so I didnt have to reimplement or maintain format related methods. Anyone know if the engine source is set up in a way where I could write say a python file that can call native godot methods from its binary, or just launch a headless godot instance for a second with some substantial way of sending data to it?

[RedMser]

I think working together with glTF is the way to go here, and not to create another custom exporter from scratch for certain edge cases. See #6883

[bud11]

I think working together with glTF is the way to go here, and not to create another custom exporter from scratch for certain edge cases. See #6883

Not a custom exporter, never said that. An easily referenceable low-effort godot provided library/script that lets you more easily write your own direct exporter without fretting over serialization details.
Regardless, I have no idea how you can call basic things like reliably exporting animations separately or with more detail, or including game-referenced scripts/materials/objects "edge cases" in the context of game development. Theyre borderline nessecities half the time.

Dont get me wrong, I see the value in working with standard formats, but there are also some big tradeoffs no matter how good the standard format is. The exporter on the dcc side has to be usable and feature-packed (blender's definitely isnt great), the importer on the engine side has to be usable and feature packed (godot has its own big import issues), and you still have to write your own code on both sides for these "edge cases" anyway, so how does it not make sense to supply a tool to make writing import code more flexible and direct?