Shader preprocessor remake #62513

reduz · 2022-06-29T09:34:04Z

Based on #61462.

Moved preprocessor to Shader and ShaderInclude
Clean up RenderingServer side
Preprocessor is separate from parser now, but it emits tokens with include location hints.
Improved ShaderEditor validation code
Added include file code completion
Added notification for all files affected by a broken include.

Images:

Error reporting on the text side:

Code completion for includes:

Bugsquad edit: Closes godotengine/godot-proposals#944

scene/resources/shader.h

servers/rendering/renderer_rd/forward_mobile/scene_shader_forward_mobile.cpp

servers/rendering/shader_language.h

servers/rendering/shader_language.cpp

clayjohn · 2022-06-29T16:12:57Z

Its working well so far from my testing. I am noticing a few issues though.

When editing a processor function (vertex, fragment etc.) in a ShaderInclude file, there is no highlighting or autocompletion for built-ins.
Stemming from "1." the shader editor reports a compile error if you edit a processor function from a ShaderInclude, yet it actually compiles and runs the shader just fine.

reduz · 2022-06-30T09:28:51Z

@clayjohn Did you try using a shader_type in the include? that should work I think.

clayjohn · 2022-06-30T15:42:10Z

@clayjohn Did you try using a shader_type in the include? that should work I think.

@reduz Just tried now. It results in an error

Chaosus · 2022-07-03T06:48:18Z

servers/rendering/shader_preprocessor.cpp

+bool ShaderPreprocessor::check_directive_before_type(Tokenizer *p_tokenizer, const String &p_directive) {
+	if (p_tokenizer->get_index() < state->shader_type_pos) {
+		set_error(vformat(RTR("`#%s` may not be defined before `shader_type`."), p_directive), p_tokenizer->get_line());
+		return false;
+	}
+	return true;
+}


This function does not needed now, since you've removed the handling of shader_type_pos (which prevent declaration of directives before shader type).

Chaosus · 2022-07-03T06:49:52Z

servers/rendering/shader_preprocessor.h

+		int include_depth = 0;
+		String current_include;
+		String current_shader_type;
+		int shader_type_pos = -1;


Remove shader_type_pos - if we don't need handling of it (to prevent preprocessor directive declaration before shader type.

Chaosus · 2022-07-03T06:52:45Z

@reduz Error handling messages seems broken:

warriormaster12 · 2022-07-20T04:06:38Z

Is it possible to backport this feature Godot 3.x?
will you implement same functionality for visual shaders?

Chaosus · 2022-07-20T04:24:07Z

Is it possible to backport this feature Godot 3.x?

will you implement same functionality for visual shaders?

This may be possible
How do you imagine it? Which nodes should be made?

warriormaster12 · 2022-07-20T05:18:24Z

@Chaosus

https://docs.unrealengine.com/4.27/en-US/RenderingAndGraphics/Materials/Functions/

Chaosus · 2022-07-20T05:30:00Z

@Chaosus

docs.unrealengine.com/4.27/en-US/RenderingAndGraphics/Materials/Functions

What exactly do you mean? - there are a lot of functions inside UE. Also, create a proposal for this please.

warriormaster12 · 2022-07-20T06:15:57Z

I mean that a developer could make custom nodes by writing first a visual shader and then including that visual shader in a bigger visual shader as function node (with user defined inputs and outputs). The expression node kinda works the way I described but currently you can only write inside it in text and you can't seperately save it to the disk for use.

I just wanted to clarify what I meant, if I still need to create a proposal then I'll do it but for sake of consistency it would be nice to see something similar to this pull request but for visual shaders.

Chaosus · 2022-07-20T06:22:17Z

I mean that a developer could make custom nodes by writing first a visual shader and then including that visual shader in a bigger visual shader as function node (with user defined inputs and outputs). The expression node kinda works the way I described but currently you can only write inside it in text and you can't seperately save it to the disk for use.

You can already use VisualShaderNodeCustom (https://docs.godotengine.org/en/latest/tutorials/plugins/editor/visual_shader_plugins.html) and create custom visual shader nodes.

warriormaster12 · 2022-07-20T07:09:42Z

Feels a bit less intuitive to create that way in my opinion .

Co-authored-by: TheOrangeDay <6472143+TheOrangeDay@users.noreply.github.com>

* Moved preprocessor to Shader and ShaderInclude * Clean up RenderingServer side * Preprocessor is separate from parser now, but it emits tokens with include location hints. * Improved ShaderEditor validation code * Added include file code completion * Added notification for all files affected by a broken include.

reduz · 2022-07-22T21:03:57Z

@akien-mga We discussed with @Chaosus that the purpose of this PR, which was redoing the preprocess design, to solve some of the fundamental problems in integrating the original PR, have been achieved and it majorly works. It may be better to just merge it as-is (its fully usable) and then @Chaosus can work on fixing the small issues in the shader compiler/preprocessor which is more his area than mine nowadays.

akien-mga

Agreed, let's merge this and improve further in follow up PRs.

Testers who found bugs are welcome to open dedicated issues to track them further.

akien-mga · 2022-07-22T22:04:44Z

Thanks to everyone involved in making this happen!

🥇 @bitsawer @fire @MagdielM @TheOrangeDay @Chaosus @clayjohn @reduz

AlfishSoftware · 2023-03-18T21:39:36Z

Oops I never noticed this update. I guess I have to update the vscode plugin's syntax for .gdshader and add .gdshaderinc (presumably the exact same syntax?).
Would be nice if you guys ping me on GDShader syntax updates; or is there somewhere I can subscribe to be notified on just those? Because I'm not actively monitoring Godot, and just happened to know of this by accident.

reduz requested review from a team as code owners June 29, 2022 09:34

Chaosus mentioned this pull request Jun 29, 2022

Adding shader preprocessor support #61462

Closed

akien-mga added feature proposal usability topic:shaders labels Jun 29, 2022

akien-mga added this to the 4.0 milestone Jun 29, 2022

akien-mga reviewed Jun 29, 2022

View reviewed changes

scene/resources/shader.h Outdated Show resolved Hide resolved

akien-mga reviewed Jun 29, 2022

View reviewed changes

servers/rendering/renderer_rd/forward_mobile/scene_shader_forward_mobile.cpp Show resolved Hide resolved

akien-mga reviewed Jun 29, 2022

View reviewed changes

servers/rendering/shader_language.h Show resolved Hide resolved

akien-mga reviewed Jun 29, 2022

View reviewed changes

servers/rendering/shader_language.cpp Show resolved Hide resolved

reduz force-pushed the shader_preprocessor_remake branch 3 times, most recently from 63a914e to af2e20a Compare June 29, 2022 15:10

Chaosus reviewed Jul 3, 2022

View reviewed changes

akien-mga mentioned this pull request Jul 7, 2022

Add built-in noise functions to shaders godotengine/godot-proposals#2443

Open

Chaosus and others added 2 commits July 22, 2022 22:51

Adding shader preprocessor support

7b94603

Co-authored-by: TheOrangeDay <6472143+TheOrangeDay@users.noreply.github.com>

reduz force-pushed the shader_preprocessor_remake branch from af2e20a to f649678 Compare July 22, 2022 21:01

akien-mga approved these changes Jul 22, 2022

View reviewed changes

akien-mga merged commit fe929d4 into godotengine:master Jul 22, 2022

clayjohn mentioned this pull request Jul 23, 2022

Built-In errors with ShaderInclude files #63337

Closed

bitsawer mentioned this pull request Aug 2, 2022

Shader Preprocessor Improvements godotengine/godot-proposals#5062

Open

6 tasks

joemicmc mentioned this pull request Aug 23, 2022

Shader include auto complete does not include folder directory #64792

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Shader preprocessor remake #62513

Shader preprocessor remake #62513

reduz commented Jun 29, 2022 •

edited by akien-mga

Loading

clayjohn commented Jun 29, 2022 •

edited

Loading

reduz commented Jun 30, 2022

clayjohn commented Jun 30, 2022

Chaosus Jul 3, 2022 •

edited

Loading

Chaosus Jul 3, 2022

Chaosus commented Jul 3, 2022

warriormaster12 commented Jul 20, 2022

Chaosus commented Jul 20, 2022

warriormaster12 commented Jul 20, 2022

Chaosus commented Jul 20, 2022 •

edited

Loading

warriormaster12 commented Jul 20, 2022

Chaosus commented Jul 20, 2022

warriormaster12 commented Jul 20, 2022

reduz commented Jul 22, 2022

akien-mga left a comment

akien-mga commented Jul 22, 2022 •

edited

Loading

AlfishSoftware commented Mar 18, 2023

Shader preprocessor remake #62513

Shader preprocessor remake #62513

Conversation

reduz commented Jun 29, 2022 • edited by akien-mga Loading

clayjohn commented Jun 29, 2022 • edited Loading

reduz commented Jun 30, 2022

clayjohn commented Jun 30, 2022

Chaosus Jul 3, 2022 • edited Loading

Choose a reason for hiding this comment

Chaosus Jul 3, 2022

Choose a reason for hiding this comment

Chaosus commented Jul 3, 2022

warriormaster12 commented Jul 20, 2022

Chaosus commented Jul 20, 2022

warriormaster12 commented Jul 20, 2022

Chaosus commented Jul 20, 2022 • edited Loading

warriormaster12 commented Jul 20, 2022

Chaosus commented Jul 20, 2022

warriormaster12 commented Jul 20, 2022

reduz commented Jul 22, 2022

akien-mga left a comment

Choose a reason for hiding this comment

akien-mga commented Jul 22, 2022 • edited Loading

AlfishSoftware commented Mar 18, 2023

reduz commented Jun 29, 2022 •

edited by akien-mga

Loading

clayjohn commented Jun 29, 2022 •

edited

Loading

Chaosus Jul 3, 2022 •

edited

Loading

Chaosus commented Jul 20, 2022 •

edited

Loading

akien-mga commented Jul 22, 2022 •

edited

Loading