Shader parameters specified in the inspector are not included when exporting a project in headless mode

[GullikX]

Godot version

4.0.beta2.official.f8745f2f7

System information

Void linux

Issue description

When exporting a project using --headless --export-pack, any shader parameters/uniforms specified in the inspector are not included in the exported project. All shader parameters are reverted to their default values (or null).

Steps to reproduce

Project setup:

1. Create a cube mesh.
2. Add a shader material to the cube with the following simple shader:

shader_type spatial;
render_mode unshaded;

uniform vec4 color: source_color = vec4(0.0, 1.0, 0.0, 0.0); // Green

void fragment() {
        ALBEDO = color.rgb;
}

3. In the inspector, set the 'color' shader parameter/uniform to be a red color.
4. When running the project from the editor, the cube is now red.

Export without --headless option:

rm -rf .godot
./Godot_v4.0-beta2_linux.x86_64 --editor --export-pack Linux/X11 export.zip
./Godot_v4.0-beta2_linux.x86_64 --main-pack export.zip
# Cube is red (correct behavior)

Export with --headless option:

rm -rf .godot
./Godot_v4.0-beta2_linux.x86_64 --editor --headless --export-pack Linux/X11 export_headless.zip
./Godot_v4.0-beta2_linux.x86_64 --main-pack export_headless.zip
# Cube is green (incorrect behavior)

The behavior can be reproduced on Godot 4.0 alpha 16 through Godot 4.0 beta 2.

Minimal reproduction project

ShaderParameterHeadless.zip

[GullikX]

Git bisect indicates that this bug was introduced in commit ef17c46.

[GullikX]

It seems like shader parameters get lost when scene files are converted to binary during export. A workaround is to disable 'convert text resources to binary' in the project settings.

[jayypluss]

I've had the same problem, I changed the default params of the shader to reflect what I wanted directly in the shader file.

It seems like shader parameters get lost when scene files are converted to binary during export. A workaround is to disable 'convert text resources to binary' in the project settings.

For future reference, this option is under Editor -> Export, you need to activate the Advanced Settings switch.

[cgsdev0]

I spent a lot of time investigating this tonight

It seems like this code is problematic in headless mode:

godot/editor/export/editor_export_platform.cpp

Lines 742 to 773 in 4dcf2c5

	if (type == "PackedScene") { // Its a scene.
	Ref<PackedScene> ps = ResourceLoader::load(p_path, "PackedScene", ResourceFormatLoader::CACHE_MODE_IGNORE);
	ERR_FAIL_COND_V(ps.is_null(), p_path);
	Node *node = ps->instantiate(PackedScene::GEN_EDIT_STATE_INSTANCE); // Make sure the child scene root gets the correct inheritance chain.
	ERR_FAIL_COND_V(node == nullptr, p_path);
	if (!customize_scenes_plugins.is_empty()) {
	for (Ref<EditorExportPlugin> &plugin : customize_scenes_plugins) {
	Node *customized = plugin->_customize_scene(node, p_path);
	if (customized != nullptr) {
	node = customized;
	modified = true;
	}
	}
	}
	if (!customize_resources_plugins.is_empty()) {
	if (_export_customize_scene_resources(node, node, customize_resources_plugins)) {
	modified = true;
	}
	}
	if (modified || p_force_save) {
	// If modified, save it again. This is also used for TSCN -> SCN conversion on export.
	String base_file = p_path.get_file().get_basename() + ".scn"; // use SCN for saving (binary) and repack (If conversting, TSCN PackedScene representation is inefficient, so repacking is also desired).
	save_path = export_base_path.path_join("export-" + p_path.md5_text() + "-" + base_file);
	Ref<PackedScene> s;
	s.instantiate();
	s->pack(node);
	Error err = ResourceSaver::save(s, save_path);
	ERR_FAIL_COND_V_MSG(err != OK, p_path, "Unable to save export scene file to: " + save_path);
	}

My working theory (as a total godot codebase noob) is that when we instantiate the node in headless mode, certain things (i.e. shader parameters) are ignored since they're unnecessary. This makes sense when we're running the game, but not when we're exporting.

I plan to continue working on this issue, but I figured I'd leave some notes here for myself and anyone interested.

[cgsdev0]

I think I've narrowed down the problem, but I'm not really sure what a good impl for a fix looks like here.

godot/scene/resources/shader.cpp

Lines 117 to 121 in a7276f1

	void Shader::get_shader_uniform_list(List<PropertyInfo> *p_params, bool p_get_groups) const {
	_update_shader();
	List<PropertyInfo> local;
	RenderingServer::get_singleton()->get_shader_parameter_list(shader, &local);

This function gets called in the path of exporting now, which, in headless mode, ends up calling this function:

godot/servers/rendering/dummy/storage/material_storage.h

Line 69 in a7276f1

virtual void get_shader_parameter_list(RID p_shader, List<PropertyInfo> *p_param_list) const override {}

Since the headless rendering server doesn't keep track of shader parameters, they get lost when we instantiate the tree.

[davidsmith1307]

Can confirm that the issue is still present in 4.0.3 and the workaround still works.

[RGonzalezTech]

This issue is still present in 4.1 and the workaround still solves it 👍 This was affecting my GPUParticles3D as well.

[Calinou]

Since the headless rendering server doesn't keep track of shader parameters, they get lost when we instantiate the tree.

I assume the RenderingDriverDummy needs to set per-instance parameters on the CPU, so that they can be retrieved later (even if they are never required for rendering).

[PeteX]

FYI, I can confirm that this issue still exists with Godot 4.2, and the work-around still resolves the problem.

[Polemus]

FYI, I can confirm that this issue still exists with Godot 4.2, and the work-around still resolves the problem.

yup, 4.2 mono here, this is still an issue, and the workaround works as well

[QbieShay]

When folks say "the workaround still works", do they mean giving the shader the default params equal to the desired ones?

[PeteX]

@QbieShay no I mean disabling "convert text resources to binary" in the project settings.

[mghicks]

For me in 4.2.1, this issue occurs both in headless exports and from-editor exports. Disabling convert text resources to binary solves the missing shader parameter, enabling convert text to binary causes the missing param in both headless and from-editor export. I am very happy to have found this issue and the workaround after some months of wondering if this was an undocumented compatibility renderer limitation.

[Eole211]

For me in 4.2.1, this issue occurs both in headless exports and from-editor exports. Disabling convert text resources to binary solves the missing shader parameter, enabling convert text to binary causes the missing param in both headless and from-editor export. I am very happy to have found this issue and the workaround after some months of wondering if this was an undocumented compatibility renderer limitation.

It was the same for me. Maybe it happens since I did an headless export for the first time. I had to go to each of my custom shaders to make them unique each time I really needed them for an export. Finding this issue and the fix was really helpful.

[Krigu]

We are building our game with (Linux) CI Pipeline in headless mode. After hours of frustration and wondering why it worked locally but not on the exported versions of iOS and web, we were also able to solve the issue with disabling the project setting "convert text resources to binary".

According to #76121 (comment) there is no easy fix, maybe it would be helpful to mention the limitations and quirks of the headless export in the documentation?

[Tichau]

I think it needs to be reopen.
This issue is still present with v4.3.stable.mono.official [77dcf97] (the workaround still solves it).